# Catalog

## Summary

Grey Matter Catalog provides discovered and user-defined metadata about services to the [Grey Matter Application](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/setup/sense/application). It connects to control plane servers such as [Grey Matter Control](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/setup/fabric-1/gm-control) and [Istiod](https://istio.io/latest/docs/ops/deployment/architecture/#istiod) to collect service endpoints and telemetry and combines them with user driven configuration to create summaries about services running within a service mesh. Although it was designed to power the Grey Matter Application, this service can be leveraged as an API to develop additional visualizations of service meshes.

## Core Features

* Multi-mesh Management
* Human-friendly Service Management
* Real-time Service Telemetry Aggregation and Health Readouts

{% hint style="info" %}
Catalog deploys with OpenAPI documentation. For the most up-to-date fields and complete usage, refer to the documentation served at the root of your deployment.
{% endhint %}

## Core Concepts

### Mesh

In Grey Matter Catalog, a mesh is an abstract collection of services across one or more data planes. Catalog provides the ability to track multiple meshes, gather in-depth information on service health, and present them to applications.

| Name             | Type   | Default | Description                                                                                              |
| ---------------- | ------ | ------- | -------------------------------------------------------------------------------------------------------- |
| `mesh_id`        | String | n/a     | The unique identifier of the mesh.                                                                       |
| `mesh_type`      | String | n/a     | The type of mesh. Either `greymatter` or `istio`.                                                        |
| `name`           | String | ""      | The human-readable name of the mesh.                                                                     |
| `sessions`       | Object | {}      | A map of configurations for control plane connections. See [configuration details](#sessions).           |
| `extensions`     | Object | {}      | A map of configurations for enabling additional mesh insights. See [configuration details](#extensions). |
| `labels`         | Object | {}      | Key/value pairs used for identifying mesh attributes. See [configuration details](#labels).              |
| `external_links` | Array  | \[]     | An array of external links associated with the mesh. See [configuration details](#external-links).       |

#### `sessions`

When configuring a mesh, the `sessions` field stores a map of session objects. Each session configured will connect to a mesh control plane server.

| Name        | Type    | Default | Description                                                         |
| ----------- | ------- | ------- | ------------------------------------------------------------------- |
| `url`       | String  | n/a     | The address of the control plane management server.                 |
| `use_tls`   | Boolean | false   | Whether to connect to the control plane management server over TLS. |
| `cert_path` | String  | ""      | The path to the server certificate.                                 |
| `key_path`  | String  | ""      | The path to the server private key.                                 |
| `ca_path`   | String  | ""      | The path to the truststore.                                         |
| `node_id`   | String  | ""      | The node ID known by the control plane management server.           |
| `cluster`   | String  | ""      | The cluster name known by the control plane management server.      |
| `region`    | String  | ""      | The region to discover service instances for.                       |
| `zone`      | String  | ""      | The zone to discover service instances for.                         |
| `sub_zone`  | String  | ""      | The sub-zone to discover service instances for.                     |

**Connecting to Grey Matter Control**

[Grey Matter Control](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/setup/fabric-1/gm-control) requires specifying a `zone` for each configured session. The following example shows how to configure Grey Matter Catalog to connect to multiple control planes within the same mesh:

```javascript
{
  ...
  "sessions": {
    "control1": {
      "url": "12345.us-east-1.elb.amazonaws.com:50000",
      "use_tls": true,
      "cert_path": "/path/to/cert.crt",
      "key_path": "/path/to/cert.key",
      "ca_path": "/path/to/ca.crt",
      "zone": "us-east-1a"
    },
    "control2": {
      "url": "67890.us-east-1.elb.amazonaws.com:50000",
      "use_tls": true,
      "cert_path": "/path/to/cert2.crt",
      "key_path": "/path/to/cert2.key",
      "ca_path": "/path/to/ca2.crt",
      "zone": "us-east-1b"
    }
  }
}
```

**Connecting to Istiod**

When configuring an `istio` mesh, only one [Istiod](https://istio.io/latest/docs/ops/deployment/architecture/#istiod) session may be configured. The following example shows how to configure Grey Matter Catalog to connect to Istiod:

```javascript
{
  ...
  "sessions": {
    "default": {
      "url": "24680.us-east-1.elb.amazonaws.com:15012"
      "use_tls": true,
      "cert_path": "/path/to/cert.crt",
      "key_path": "/path/to/cert.key",
      "ca_path": "/path/to/ca.crt",
    }
  }
}
```

#### `extensions`

Grey Matter Catalog supports a number of extensions for enabling additional mesh insights beyond service discovery.

**Health Reports**

Grey Matter Catalog can connect to one or more pub/sub servers such as Redis and NATS for consuming Grey Matter Data Plane metrics throughout a Grey Matter service mesh. The metrics are observed and analyzed via active and passive health checks on each discovered service to generate service health reports that are exposed by the API.

{% hint style="info" %}
Configuring the metrics pipelines within a Grey Matter mesh requires additional steps in the Grey Matter Control API. Refer to [this guide](https://github.com/greymatter-io/gm-gitbook-sync/blob/1.7-beta/guides/configure-service-health-reports.md) for a thorough introduction to health reports in Grey Matter.
{% endhint %}

In addition to the connection settings for each supported pub/sub server (see below), the following options may be adjusted to modify Grey Matter Catalog's health reporting behavior for each mesh:

| Name                    | Type   | Default | Description                                                                                                 |
| ----------------------- | ------ | ------- | ----------------------------------------------------------------------------------------------------------- |
| `event_window_minutes`  | Number | 2       | The length of the sliding window to observe and analyze for producing health reports in real-time.          |
| `event_timeout_minutes` | Number | 1       | How long before a node in the Grey Matter Data Plane that produces no metrics should be considered offline. |

The following example shows how to configure Grey Matter Catalog to consume, observe, and analyze metrics from multiple pub/sub servers within the same mesh:

```javascript
{
  ...
  "extensions": {
    "metrics": {
      "event_window_minutes": 2,
      "event_timeout_minutes": 1,
      "sessions": {
        "redis1": {
          "client_type": "redis",
          "connection_string": "redis://username:password@redis.svc.local:6379/0",
          "cert_path": "/path/to/redis_cert.crt",
          "key_path": "/path/to/redis_cert.key",
          "ca_path": "/path/to/redis_ca.crt"
        },
        "nats1": {
          "client_type": "nats",
          "connection_string": "username:password@nats.svc.local:4222",
          "cert_path": "/path/to/nats_cert.crt",
          "key_path": "/path/to/nats_cert.key",
          "ca_path": "/path/to/nats_ca.crt"
        }
      }
    }
  }
}
```

**Anomaly Detection**

Grey Matter Catalog can aggregate data from an anomaly detection service, a server which trains on and analyzes the behavior of nodes in the Grey Matter Data Plane. This feature is currently experimental.

#### `labels`

The `labels` field allows for specifying arbitrary string key/value pairs associated with the mesh. These can be any user-defined metadata used by an operator or another application. For example, the Grey Matter Application checks for the following labels in order to connect to the [Grey Matter Control API](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/setup/fabric-1/control-api) server for a specific mesh:

```javascript
{
  ...
  "labels": {
    "configuration_url": "https://{your-ingress-url}/services/control-api/latest/v1.0/",
    "zone_key": "zone_default_zone"
  }
}
```

#### `external_links`

The `external_links` field specifies one or more links associated with the mesh.

| Name    | Type   | Default | Description       |
| ------- | ------ | ------- | ----------------- |
| `title` | String | n/a     | The link's title. |
| `url`   | String | n/a     | The link's URL.   |

#### Example Mesh Configuration

```javascript
{
  "mesh_id": "sample-mesh",
  "mesh_type": "greymatter",
  "name": "Grey Matter Service Mesh",
  "sessions": {
    "default": {
      "url": "control.default.svc:50000",
      "use_tls": true,
      "cert_path": "/certs/control/server.crt",
      "key_path": "/certs/control/server.key",
      "ca_path": "/certs/control/ca.crt"
      "cluster": "edge",
      "zone": "zone-default-zone"
    }
  },
  "extensions": {
    "metrics": {
      "sessions": {
        "default": {
          "client_type": "redis",
          "connection_string": "redis://username:password@redis.svc.local:6379/0",
          "cert_path": "/path/to/redis_cert.crt",
          "key_path": "/path/to/redis_cert.key",
          "ca_path": "/path/to/redis_ca.crt"
        }
      }
    }
  },
  "labels": {},
  "external_links": [
    {
      "title": "Grey Matter Home Page",
      "url": "https://greymatter.io"
    }
  ]
}
```

#### Example Mesh Output

When configured, retrieving a mesh from the Catalog API will return its configuration in addition to various metadata about the mesh. Below is an example of what the Catalog API exposes for a mesh. The following sample mesh is owned by Decipher, for core and demo capabilities. Its configured discovery session found instances for 7 services from Grey Matter Control (4 stable, 2 warning, and 1 down) with a total of 8 unique service instances.

```javascript
{
  "mesh_id": "sample-mesh",
  "configuration": {...}, // see above
  "statuses": {
    "default": "OK"
  },
  "instance_counts": {
    "catalog": 2,
    "control-api": 1,
    "dashboard": 1,
    "edge": 2,
    "jwt-security": 1,
    "prometheus": 1,
    "slo": 0
  },
  "metadata": {
    "owners": [
      "Decipher"
    ],
    "capabilities": [
      "Core",
      "Demo",
    ],
    "service_count": 7,
    "service_stable_count": 4,
    "service_warning_count": 2,
    "service_down_count": 1,
    "instance_count": 8
  }
}
```

### Service

Catalog tracks each service in a configured mesh by watching for instance discovery and service health updates. It also presents human-curated data for each service.

| Name                        | Type    | Default              | Description                                                                                          |
| --------------------------- | ------- | -------------------- | ---------------------------------------------------------------------------------------------------- |
| `service_id`                | String  | n/a                  | The unique identifier of the service within its mesh.                                                |
| `mesh_id`                   | String  | n/a                  | The unique identifier of the mesh the service belongs to.                                            |
| `name`                      | String  | ""                   | The human-readable name of the service.                                                              |
| `version`                   | String  | ""                   | The version of the service.                                                                          |
| `business_impact`           | String  | "low"                | Either `low`, `medium`, `high`, or `critical`, describing the service's impact on your business.     |
| `capability`                | String  | ""                   | A tag specifying the service's capability.                                                           |
| `description`               | String  | ""                   | A description of the service.                                                                        |
| `owner`                     | String  | ""                   | A tag specifying an organization, company, department, etc. to whom the service belongs.             |
| `owner_url`                 | String  | ""                   | A URL to associate with the owner.                                                                   |
| `api_endpoint`              | String  | ""                   | A URL to the service's API endpoint.                                                                 |
| `api_spec_endpoint`         | String  | ""                   | A URL to the service's API specification.                                                            |
| `documentation`             | String  | ""                   | A URL specifying the service's documentation page.                                                   |
| `enable_instance_metrics`   | Boolean | true                 | Whether to permit access to metrics for the service's instances.                                     |
| `enable_historical_metrics` | Boolean | false                | Whether to permit access to historical metrics for the service.                                      |
| `prometheus_job`            | String  | same as `service_id` | The name assigned to the service's corresponding Prometheus job for tracking its historical metrics. |
| `external_links`            | Array   | \[]                  | An array of external links associated with the service.                                              |

#### Example Service Configuration

```javascript
{
  "service_id": "fibonacci",
  "mesh_id": "zone-default-zone",
  "name": "Fibonacci",
  "version": "1.0",
  "business_impact": "medium",
  "capability": "Tutorial",
  "description": "Fibonacci returns Fibonacci numbers",
  "owner": "Decipher",
  "owner_url": "https://greymatter.io",
  "api_endpoint": "https://greymatter.io/services/fibonacci/latest",
  "api_spec_endpoint": "https://greymatter.io/services/fibonacci/latest/swagger.json",
  "documentation": "/services/fibonacci/",
  "enable_instance_metrics": true,
  "enable_historical_metrics": true,
  "prometheus_job": "fibonacci",
  "external_links": [
    {
      "title": "Grey Matter",
      "url": "https://greymatter.io"
    }
  ]
}
```

#### Example Service Output

When configured, retrieving a service from the Catalog API will return its configuration in addition to various metadata and health readouts about the service. Below is an example of what the Catalog API exposes for a service. The following service has one instance which was discovered by the `default` session (see the [mesh configuration section](#mesh)) and exists in the locality `zone-default-zone`. The service health status is considered `stable` since it has passed all health checks. The service entry also includes metadata reported by the control plane server, including instance start time and the protocol used by the service.

```javascript
{
  ... // see above
  "mesh_type": "greymatter",
  "localities": [
    "zone-default-zone"
  ],
  "instances": [
    {
      "instance_id": "1e4687d8677e028f044075400f3dbc11",
      "session": "default",
      "locality": "zone-default-zone",
      "metadata": {
        "start_time": 1616181075916
      }
    }
  ],
  "metadata": {
    "metrics_port": 8081,
    "protocols": [
      "mTLS/HTTP"
    ]
  },
  "status": "stable",
  "health": {
    "status": "stable",
    "timestamp": 1631279780,
    "sidecar": {
      "status": "stable",
      "groups": {
        "connections": [
          {
            "title": "Destroyed With Active Requests",
            "description": "No recent connections to cluster example were destroyed with active requests over the past 2 minutes",
            "metric_name": "upstream_cx_destroy_with_active_rq",
            "severity": "stable",
            "expected": "≤ 0",
            "observed": 0
          },
          {
            "title": "Failures",
            "description": "Connection failures are within acceptable limits for requests to cluster example over the past 2 minutes",
            "metric_name": "upstream_cx_connect_fail",
            "severity": "stable",
            "expected": "≤ 10",
            "observed": 0
          },
          {
            "title": "Protocol Errors",
            "description": "An unremarkable number of protocol errors have occurred while communicating with cluster example over the past 2 minutes",
            "metric_name": "upstream_cx_protocol_error",
            "severity": "stable",
            "expected": "≤ 1",
            "observed": 0
          }
        ],
        "discovery": [
          {
            "title": "Control Plane",
            "description": "The control plane reports all instances to be healthy",
            "metric_name": "discovery_controlplane_reports",
            "severity": "stable",
            "expected": 1,
            "observed": 1
          },
          {
            "title": "Health Monitor",
            "description": "All discovered instances reported metrics on time.",
            "metric_name": "discovery_healthmonitor_reports",
            "severity": "stable",
            "expected": "Less than 20 seconds ago",
            "observed": "2 seconds ago"
          },
          {
            "title": "Instances",
            "description": "All discovered instances have reported live in the last 2 minutes",
            "metric_name": "discovery_instance_reports",
            "severity": "stable",
            "expected": 1,
            "observed": 1
          }
        ],
        "requests": [
          {
            "title": "Flow Control Throttling",
            "description": "Cluster example has had no recent problems with flow control over the past 2 minutes",
            "metric_name": "upstream_flow_control_paused_reading_total",
            "severity": "stable",
            "expected": "≤ 1",
            "observed": 1
          },
          {
            "title": "Pending",
            "description": "Requests waiting for connection pool connections to cluster example are within acceptable limits over the past 2 minutes",
            "metric_name": "upstream_rq_pending_total",
            "severity": "stable",
            "expected": "≤ 10",
            "observed": 1
          },
          {
            "title": "Retries",
            "description": "Retries to cluster example are within acceptable limits over the past 2 minutes",
            "metric_name": "upstream_rq_retry",
            "severity": "stable",
            "expected": "≤ 10",
            "observed": 0
          }
        ]
      }
    },
    "outgoing": {
      "example-service": {
        "status": "stable",
        "groups": {
          "connections": [
            {
              "title": "Destroyed With Active Requests",
              "description": "No recent connections to cluster example-service were destroyed with active requests over the past 2 minutes",
              "metric_name": "upstream_cx_destroy_with_active_rq",
              "severity": "stable",
              "expected": "≤ 0",
              "observed": 0
            },
            {
              "title": "Failures",
              "description": "Connection failures are within acceptable limits for requests to cluster example-service over the past 2 minutes",
              "metric_name": "upstream_cx_connect_fail",
              "severity": "stable",
              "expected": "≤ 10",
              "observed": 0
            },
            {
              "title": "Protocol Errors",
              "description": "An unremarkable number of protocol errors have occurred while communicating with cluster example-service over the past 2 minutes",
              "metric_name": "upstream_cx_protocol_error",
              "severity": "stable",
              "expected": "≤ 1",
              "observed": 0
            }
          ],
          "requests": [
            {
              "title": "Flow Control Throttling",
              "description": "Cluster example-service has had no recent problems with flow control over the past 2 minutes",
              "metric_name": "upstream_flow_control_paused_reading_total",
              "severity": "stable",
              "expected": "≤ 1",
              "observed": 1
            },
            {
              "title": "Pending",
              "description": "Requests waiting for connection pool connections to cluster example-service are within acceptable limits over the past 2 minutes",
              "metric_name": "upstream_rq_pending_total",
              "severity": "stable",
              "expected": "≤ 10",
              "observed": 1
            },
            {
              "title": "Retries",
              "description": "Retries to cluster example-service are within acceptable limits over the past 2 minutes",
              "metric_name": "upstream_rq_retry",
              "severity": "stable",
              "expected": "≤ 10",
              "observed": 0
            }
          ]
        }
      }
    }
  }
}
```

### Metrics

Catalog also serves as a normalized passthrough to the real-time metrics for each service. These can be requested with the catalog `/metrics` endpoint (e.g `curl host:port/metrics/<instance_id>`). See [Catalog API docs](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/catalog-api) for more detail.

#### Example Metrics Output

```javascript
{
  "grey-matter-metrics-version": "1.0.0",
  "Total/requests": 25,
  "HTTP/requests": 0,
  "HTTPS/requests": 25,
  "RPC/requests": 0,
  "RPC_TLS/requests": 0,
  "route/services/catalog/1.0/metrics/GET/errors.count": 0,
  "route/services/catalog/1.0/metrics/GET/in_throughput": 0,
  "route/services/catalog/1.0/metrics/GET/out_throughput": 148004,
  "route/services/catalog/1.0/summary/GET/requests": 2,
  "route/services/catalog/1.0/summary/GET/routes": "",
  "route/services/catalog/1.0/summary/GET/status/200": 2,
  "route/services/catalog/1.0/summary/GET/status/2XX": 2,
  "route/services/catalog/1.0/summary/GET/latency_ms.avg": 0.000000,
  "route/services/catalog/1.0/summary/GET/latency_ms.count": 2,
  "route/services/catalog/1.0/summary/GET/latency_ms.max": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.min": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.sum": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p50": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p90": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p95": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p99": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p9990": 0,
  "route/services/catalog/1.0/summary/GET/latency_ms.p9999": 0,
  "route/services/catalog/1.0/summary/GET/errors.count": 0,
  "route/services/catalog/1.0/summary/GET/in_throughput": 0,
  "route/services/catalog/1.0/summary/GET/out_throughput": 10248004,
  "all/requests": 25,
  "all/routes": "",
  "all/status/200": 21,
  "all/status/500": 1,
  "all/status/304": 3,
  "all/status/2XX": 21,
  "all/status/5XX": 1,
  "all/status/3XX": 3,
  "all/latency_ms.avg": 0.000000,
  "all/latency_ms.count": 25,
  "all/latency_ms.max": 0,
  "all/latency_ms.min": 0,
  "all/latency_ms.sum": 0,
  "all/latency_ms.p50": 0,
  "all/latency_ms.p90": 0,
  "all/latency_ms.p95": 0,
  "all/latency_ms.p99": 0,
  "all/latency_ms.p9990": 0,
  "all/latency_ms.p9999": 0,
  "all/errors.count": 0,
  "all/in_throughput": 0,
  "all/out_throughput": 5022597,
  "route/services/prometheus/api/v1/GET/requests": 10,
  "route/services/prometheus/api/v1/GET/routes": "",
  "route/services/prometheus/api/v1/GET/status/200": 10,
  "route/services/prometheus/api/v1/GET/status/2XX": 10,
  "route/services/prometheus/api/v1/GET/latency_ms.avg": 0.000000,
  "route/services/prometheus/api/v1/GET/latency_ms.count": 10,
  "route/services/prometheus/api/v1/GET/latency_ms.max": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.min": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.sum": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p50": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p90": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p95": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p99": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p9990": 0,
  "route/services/prometheus/api/v1/GET/latency_ms.p9999": 0,
  "route/services/prometheus/api/v1/GET/errors.count": 0,
  "route/services/prometheus/api/v1/GET/in_throughput": 0,
  "route/services/prometheus/api/v1/GET/out_throughput": 201782,
  "route/services/slo/latest/objectives/GET/requests": 2,
  "route/services/slo/latest/objectives/GET/routes": "",
  "route/services/slo/latest/objectives/GET/status/200": 2,
  "route/services/slo/latest/objectives/GET/status/2XX": 2,
  "route/services/slo/latest/objectives/GET/latency_ms.avg": 0.000000,
  "route/services/slo/latest/objectives/GET/latency_ms.count": 2,
  "route/services/slo/latest/objectives/GET/latency_ms.max": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.min": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.sum": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p50": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p90": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p95": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p99": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p9990": 0,
  "route/services/slo/latest/objectives/GET/latency_ms.p9999": 0,
  "route/services/slo/latest/objectives/GET/errors.count": 0,
  "route/services/slo/latest/objectives/GET/in_throughput": 0,
  "route/services/slo/latest/objectives/GET/out_throughput": 2838,
  "go_metrics/runtime/num_goroutines": 20,
  "system/start_time": 1585326138311,
  "system/cpu.pct": 5.729167,
  "system/cpu_cores": 2,
  "os": "linux",
  "os_arch": "amd64",
  "system/memory/available": 6736568320,
  "system/memory/used": 1492824064,
  "system/memory/used_percent": 17.851688,
  "process/memory/used": 71762168
}
```

## Server Configuration Options

### General Environment Variables

| Name                         | Type    | Default   | Description                                                                                                                  |
| ---------------------------- | ------- | --------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `HOST`                       | String  | "0.0.0.0" | Hostname or IP address.                                                                                                      |
| `PORT`                       | Number  | 8080      | TCP port to listen on.                                                                                                       |
| `AUTHORIZED_USERS`           | String  | ""        | List of PKI Distinguished Names (DNs) approved to access metrics views in the dashboard. An empty string permits all access. |
| `CONTROL_SERVER_RETRY_DELAY` | String  | "10s"     | Delay duration for indefinite connection retries to each configured Control server.                                          |
| `METRICS_MAX_RETRIES`        | Number  | 3         | The maximum number of retries made before stopping requests to a service instance's metrics endpoint.                        |
| `METRICS_RETRY_DELAY`        | String  | "10s"     | The wait period between attempts to connect to a service instance's metrics endpoint.                                        |
| `METRICS_REQUEST_TIMEOUT`    | String  | "15s"     | The wait time to indicate a request timeout from a service instance's metrics endpoint.                                      |
| `DEBUG`                      | Boolean | false     | Whether to start the service with the debug log level.                                                                       |

### TLS

In Grey Matter service mesh deployments, Transport Layer Security (TLS) is typically handled by a [Grey Matter Sidecar](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/setup/fabric-1/grey-matter-proxy) sidecar in front of the Catalog Service. In other words, the Catalog Service is treated like any other service in the service mesh, whereby security is offloaded to the sidecar proxy. However, this service can run standalone, and in these cases the following environment variables can be set to configure TLS.

| Name               | Type    | Default | Description                             |
| ------------------ | ------- | ------- | --------------------------------------- |
| `USE_TLS`          | Boolean | false   | Enables TLS                             |
| `CA_CERT_PATH`     | String  | ""      | Path to the certificate authority file. |
| `SERVER_CERT_PATH` | String  | ""      | Path to the server certificate.         |
| `SERVER_KEY_PATH`  | String  | ""      | Path to the server certificate key.     |

### Seeding Meshes and Services

{% hint style="warning" %}
Seeding control-plane connections and services via environment variables has been deprecated as of Catalog 2.0.0. If upgrading from a previous deployment, the seed file implementation in this section should replace any environment variables that are prefixed with the following:

* `CONTROL_SERVER_<N>_`
* `SERVICE_<N>_`
  {% endhint %}

Catalog provides the option of seeding meshes and services via JSON or YAML formatted configuration file. This is a quick way to get a new Catalog deployment up and running.

| Name               | Type   | Default | Description                                               |
| ------------------ | ------ | ------- | --------------------------------------------------------- |
| `SEED_FILE_PATH`   | String | ""      | Path to the seed file.                                    |
| `SEED_FILE_FORMAT` | String | "json"  | The format of the seed file. Use either `json` or `yaml`. |

If a [persister](#persistence) is configured, any existing data from the persister will be merged with the seed data. If there is any conflicting data, the persisted data will take precedence. The merged data will then be backed up by the persister.

The following is an example JSON configuration file:

```javascript
{
  "sample-mesh": {
    "mesh_type": "greymatter",
    "name": "Grey Matter Service Mesh",
    "sessions": {
      "zone-default-zone": {
        "url": "control.default.svc:50000",
        "use_tls": true,
        "cert_path": "/certs/control/server.crt",
        "key_path": "/certs/control/server.key",
        "ca_path": "/certs/control/ca.crt"
        "cluster": "edge",
        "zone": "zone-default-zone"
      }
    },
    "extensions": {
      "metrics": {
        "sessions": {
          "default": {
            "client_type": "redis",
            "connection_string": "redis://username:password@redis.svc.local:6379/0",
            "cert_path": "/path/to/redis_cert.crt",
            "key_path": "/path/to/redis_cert.key",
            "ca_path": "/path/to/redis_ca.crt"
          }
        }
      }
    },
    "services": {
      "catalog": {
        "name": "Grey Matter Catalog",
        "version": "2.0.0",
        "business_impact": "high"
      },
      "control-api": {
        "name": "Grey Matter Control API",
        "version": "1.5.4",
        "business_impact": "critical"
      },
      "dashboard": {
        "name": "Grey Matter Dashboard",
        "version": "5.0.0",
        "business_impact": "high"
      },
      "edge": {
        "name": "Grey Matter Edge",
        "version": "1.5.1",
        "business_impact": "critical"
      },
      "jwt-security": {
        "name": "Grey Matter JWT Security",
        "version": "1.2.0",
        "business_impact": "critical"
      },
      "slo": {
        "name": "Grey Matter Service Level Objectives",
        "version": "1.2.0",
        "business_impact": "high"
      }
    }
  },
  "sample-mesh-2": {
    "mesh_type": "istio",
    "sessions": {
      "default": {
        "url": "istiod.istio-system.svc.cluster.local:15012",
        "use_tls": true,
        "cert_path": "/certs/istiod/server.crt",
        "key_path": "/certs/istiod/server.key",
        "ca_path": "/certs/istiod/ca.crt"
      }
    },
    "services": {
      "details": {},
      "ratings": {},
      "reviews": {},
      "productpage": {}
    }
  }
}
```

Here is the same configuration as the one above in YAML format:

```yaml
sample-mesh:
  mesh_type: greymatter
  name: Grey Matter Service Mesh
  sessions:
    zone-default-zone:
      url: control.default.svc:50000
      use_tls: true
      cert_path: /certs/control/server.crt
      key_path: /certs/control/server.key
      ca_path: /certs/control/ca.crt
      cluster: edge
      zone: zone-default-zone
  extensions:
    metrics:
      sessions:
        default:
          client_type: redis
          connection_string: "redis://username:password@redis.svc.local:6379/0"
          cert_path: /path/to/redis_cert.crt
          key_path: /path/to/redis_cert.key
          ca_path: /path/to/redis_ca.crt
  services:
    catalog:
      name: Grey Matter Catalog
      version: 2.0.0
      business_impact: high
    control-api:
      name: Grey Matter Control API
      version: 1.5.4
      business_impact: critical
    dashboard:
      name: Grey Matter Dashboard
      version: 5.0.0
      business_impact: high
    edge:
      name: Grey Matter Edge
      version: 1.5.1
      business_impact: critical
    jwt-security:
      name: Grey Matter JWT Security
      version: 1.2.0
      business_impact: critical
    slo:
      name: Grey Matter Service Level Objectives
      version: 1.2.0
      business_impact: high

sample-mesh-2:
  mesh_type: istio
  sessions:
    default:
      url: istiod.istio-system.svc.cluster.local:15012
      use_tls": true
      cert_path: /certs/istiod/server.crt
      key_path: /certs/istiod/server.key
      ca_path: /certs/istiod/ca.crt
  services:
    details: {}
    ratings: {}
    reviews: {}
    productpage: {}
```

### Configure Persistence with Redis

Persistence of mesh and service configuration is disabled by default, but in production environments persistence should be configured with Redis.

When the Grey Matter service mesh is deployed via our [Helm charts](https://github.com/greymatter-io/helm-charts), Catalog will already be configured to persist service configuration in an internal instance of Redis. This is our recommended approach for production deployments where Catalog may be scaled up to multiple nodes. Changes to services stored in Redis are made possible via the Catalog RESTful API.

It is also recommended to persist the data written to Redis to disk by setting the `appendonly` setting in your Redis server configuration. [See the Redis documentation for more details](https://redis.io/topics/persistence).

Below is a list of environment variables required to use Redis as the persistent store. Note, that these are automatically set when deploying with our [Helm charts](https://github.com/greymatter-io/helm-charts).

| Name                     | Type    | Default     | Description                                        |
| ------------------------ | ------- | ----------- | -------------------------------------------------- |
| `CONFIG_SOURCE`          | String  | ""          | Requires `redis` to be set as the value            |
| `REDIS_HOST`             | String  | "localhost" | Host of the Redis connection                       |
| `REDIS_PORT`             | Integer | 6379        | Port of the Redis connection                       |
| `REDIS_USE_TLS`          | Boolean | false       | Whether to connect to Redis over TLS               |
| `REDIS_CA_CERT_PATH`     | String  | ""          | Path to SSL CA on disk for connecting to Redis     |
| `REDIS_SERVER_CERT_PATH` | String  | ""          | Path to SSL Cert on disk for connecting to Redis   |
| `REDIS_SERVER_KEY_PATH=` | String  | ""          | Path to SSL Key on disk for connecting to Redis    |
| `REDIS_PASS`             | String  | ""          | Password if required by Redis                      |
| `REDIS_DB`               | Integer | 0           | Which database to use in Redis                     |
| `REDIS_MAX_RETRIES`      | String  | 5           | Max number of times to attempt to connect to Redis |
| `REDIS_RETRY_DELAY`      | String  | "1s"        | Duration between Redis connection attempts         |

## Logging

The log level of the Grey Matter Catalog service can be retrieved and dynamically changed via the following requests:

### GET `/logging`

Returns the current log level.

```bash
current log level = info
```

### PUT `/logging?level=<log-level>`

Updates the log level to the level indicated in query parameter `level`.

Level should be one of: `error`, `warn`, `info`, `debug`.

## Environment Configuration File

{% hint style="info" %}
**Note:** the following is not a typical deployment model.
{% endhint %}

If you're running the Catalog executable binary manually you can also provide a configuration file with key/value pairs.*\*\**

```
./gm-catalog --config=settings.toml
```

If there are conflicting configurations between this file and environment variables, then the environment variables will take precedence.

## Questions

{% hint style="success" %}
**Need help configuring Grey Matter Catalog?** Contact us at <https://support.greymatter.io/support/home>.
{% endhint %}