# Catalog ## Summary Grey Matter Catalog provides additional metadata about services to the [Grey Matter Dashboard](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/sense/application). It connects to [Grey Matter Control](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control)) to retrieve discovered services and combines them with user driven configuration to create summarized metadata about services running within a service mesh. Although it was designed to power the Grey Matter application, this service can be leveraged as an API to develop further visualizations of a service mesh. See our guide on [how to use Grey Matter Catalog](https://github.com/greymatter-io/gm-gitbook-sync/tree/110050957208b90f3e62ecfb102bd2f129eeb9ed/usage/application/catalog/using-the-catalog-api.md) for how to interact with this service when it's up and running. ## Core Features * Multi-zone Management * Human-friendly Service Management * Real-time Service Metrics Access {% hint style="info" %} Catalog deploys with OpenAPI documentation. For the most up-to-date fields and complete usage, refer to those docs of your deployment. {% endhint %} ## Core Concepts ### Zones In Grey Matter, each [zone](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/api/fabric-api/zone) is a unique [data plane](https://github.com/greymatter-io/gm-gitbook-sync/tree/110050957208b90f3e62ecfb102bd2f129eeb9ed/reference/reference/glossary.md#data-plane). Catalog provides the ability to track multiple zones, gather high-level information on service statuses, and present them to applications. A sample zone is shown below. This zone is owned by Decipher, for core and demo capabilities. It has 7 different [service clusters](https://github.com/greymatter-io/gm-gitbook-sync/tree/110050957208b90f3e62ecfb102bd2f129eeb9ed/reference/reference/glossary.md#service-cluster)(4 stable, 2 warning, and 1 down) with a total of 9 unique service instances. ```javascript { "zoneName": "default-zone", "owners": [ "Decipher" ], "capabilities": [ "Greymatter", "Demo" ], "clusterCount": 7, "clusterStableCount": 4, "clusterWarningCount": 2, "clusterDownCount": 1, "instanceCount": 9, "externalLinks": [ { "name": "Grey Matter", "link": "https://greymatter.io" } ] } ``` ### Service Entry Within each zone, Catalog also tracks each [service cluster](https://github.com/greymatter-io/gm-gitbook-sync/tree/110050957208b90f3e62ecfb102bd2f129eeb9ed/reference/reference/glossary.md#service-cluster) in the Fabric service mesh. Catalog watches the mesh for instances of these services, reports when they're up/down, and presents user-defined data for each. | Name | Type | Default | Description | | ------------------------- | --------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | `clusterName` | String | n/a | The unique cluster name assigned to the service within a mesh. This is an updated field from the deprecated `clusterID` field. | | `zoneName` | String | n/a | The zone to which the service belongs (this generally will be `default-zone`.) This is an updated field from the deprecated `meshID` field. | | `name` | String | "" | A display name for the catalog entry. | | `version` | String | "" | The version of the deployed service. | | `businessImpact` | String | "low" | Either `low`, `medium`, `high`, or `critical`, describing the service's impact on your business. | | `description` | String | "" | A description of the deployed service. | | `capability` | String | "" | A tag specifying the service's capability. | | `owner` | String | "" | A tag specifying an organization, company, department, etc. to whom the service belongs. | | `ownerURL` | String | "" | A URL to associate with the owner. | | `apiEndpoint` | String | "" | A URL to the service's API endpoint. | | `apiSpecEndpoint` | String | "" | A URL to the service's API specification. | | `documentation` | String | "" | A URL specifying the service's documentation page. | | `minInstances` | Number | 1 | The minimum number of running instances of the service you can expect. | | `maxInstances` | Number | 1 | The maximum number of running instances of the service you can expect. | | `enableInstanceMetrics` | Boolean | true | Whether to permit access to metrics for the service's instances. | | `metricsPort` | Number | 8081 | The port configured for the service's sidecar to expose its metrics. | | `enableHistoricalMetrics` | Boolean | false | Whether to permit access to historical metrics for the service. | | `prometheusJob` | String | same as `clusterName` | The name assigned to the service's corresponding Prometheus job for tracking its historical metrics. | | `externalLinks` | \[]String | \[] | An array of external links associated with the service. | #### Example Service Cluster ```javascript [ { "clusterName": "fibonacci", "zoneName": "zone-default-zone", "name": "Fibonacci", "version": "1.0", "businessImpact": "medium", "capability": "Tutorial", "description": "Fibonacci returns Fibonacci numbers", "owner": "Decipher", "ownerURL": "https://greymatter.io", "apiEndpoint": "https://greymatter.io/services/fibonacci/latest", "apiSpecEndpoint": "https://greymatter.io/services/fibonacci/latest/swagger.json", "documentation": "/services/fibonacci/", "minInstances": 1, "maxInstances": 2, "enableInstanceMetrics": true, "metricsPort": 8081, "enableHistoricalMetrics": true, "prometheusJob": "fibonacci", "externalLinks": [ { "name": "Grey Matter", "link": "https://greymatter.io" } ] } ] ``` ## Metrics Catalog also serves as a normalized passthrough to the real-time metrics for each service. These can be requested with the catalog `/metrics` endpoint (e.g `curl host:port/metrics//`). See the deployed swagger docs for more detail. ### Example Returned Metrics ```javascript { "grey-matter-metrics-version": "1.0.0", "Total/requests": 25, "HTTP/requests": 0, "HTTPS/requests": 25, "RPC/requests": 0, "RPC_TLS/requests": 0, "route/services/catalog/1.0/metrics/GET/errors.count": 0, "route/services/catalog/1.0/metrics/GET/in_throughput": 0, "route/services/catalog/1.0/metrics/GET/out_throughput": 148004, "route/services/catalog/1.0/summary/GET/requests": 2, "route/services/catalog/1.0/summary/GET/routes": "", "route/services/catalog/1.0/summary/GET/status/200": 2, "route/services/catalog/1.0/summary/GET/status/2XX": 2, "route/services/catalog/1.0/summary/GET/latency_ms.avg": 0.000000, "route/services/catalog/1.0/summary/GET/latency_ms.count": 2, "route/services/catalog/1.0/summary/GET/latency_ms.max": 0, "route/services/catalog/1.0/summary/GET/latency_ms.min": 0, "route/services/catalog/1.0/summary/GET/latency_ms.sum": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p50": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p90": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p95": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p99": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p9990": 0, "route/services/catalog/1.0/summary/GET/latency_ms.p9999": 0, "route/services/catalog/1.0/summary/GET/errors.count": 0, "route/services/catalog/1.0/summary/GET/in_throughput": 0, "route/services/catalog/1.0/summary/GET/out_throughput": 10248004, "all/requests": 25, "all/routes": "", "all/status/200": 21, "all/status/500": 1, "all/status/304": 3, "all/status/2XX": 21, "all/status/5XX": 1, "all/status/3XX": 3, "all/latency_ms.avg": 0.000000, "all/latency_ms.count": 25, "all/latency_ms.max": 0, "all/latency_ms.min": 0, "all/latency_ms.sum": 0, "all/latency_ms.p50": 0, "all/latency_ms.p90": 0, "all/latency_ms.p95": 0, "all/latency_ms.p99": 0, "all/latency_ms.p9990": 0, "all/latency_ms.p9999": 0, "all/errors.count": 0, "all/in_throughput": 0, "all/out_throughput": 5022597, "route/services/prometheus/api/v1/GET/requests": 10, "route/services/prometheus/api/v1/GET/routes": "", "route/services/prometheus/api/v1/GET/status/200": 10, "route/services/prometheus/api/v1/GET/status/2XX": 10, "route/services/prometheus/api/v1/GET/latency_ms.avg": 0.000000, "route/services/prometheus/api/v1/GET/latency_ms.count": 10, "route/services/prometheus/api/v1/GET/latency_ms.max": 0, "route/services/prometheus/api/v1/GET/latency_ms.min": 0, "route/services/prometheus/api/v1/GET/latency_ms.sum": 0, "route/services/prometheus/api/v1/GET/latency_ms.p50": 0, "route/services/prometheus/api/v1/GET/latency_ms.p90": 0, "route/services/prometheus/api/v1/GET/latency_ms.p95": 0, "route/services/prometheus/api/v1/GET/latency_ms.p99": 0, "route/services/prometheus/api/v1/GET/latency_ms.p9990": 0, "route/services/prometheus/api/v1/GET/latency_ms.p9999": 0, "route/services/prometheus/api/v1/GET/errors.count": 0, "route/services/prometheus/api/v1/GET/in_throughput": 0, "route/services/prometheus/api/v1/GET/out_throughput": 201782, "route/services/slo/latest/objectives/GET/requests": 2, "route/services/slo/latest/objectives/GET/routes": "", "route/services/slo/latest/objectives/GET/status/200": 2, "route/services/slo/latest/objectives/GET/status/2XX": 2, "route/services/slo/latest/objectives/GET/latency_ms.avg": 0.000000, "route/services/slo/latest/objectives/GET/latency_ms.count": 2, "route/services/slo/latest/objectives/GET/latency_ms.max": 0, "route/services/slo/latest/objectives/GET/latency_ms.min": 0, "route/services/slo/latest/objectives/GET/latency_ms.sum": 0, "route/services/slo/latest/objectives/GET/latency_ms.p50": 0, "route/services/slo/latest/objectives/GET/latency_ms.p90": 0, "route/services/slo/latest/objectives/GET/latency_ms.p95": 0, "route/services/slo/latest/objectives/GET/latency_ms.p99": 0, "route/services/slo/latest/objectives/GET/latency_ms.p9990": 0, "route/services/slo/latest/objectives/GET/latency_ms.p9999": 0, "route/services/slo/latest/objectives/GET/errors.count": 0, "route/services/slo/latest/objectives/GET/in_throughput": 0, "route/services/slo/latest/objectives/GET/out_throughput": 2838, "go_metrics/runtime/num_goroutines": 20, "system/start_time": 1585326138311, "system/cpu.pct": 5.729167, "system/cpu_cores": 2, "os": "linux", "os_arch": "amd64", "system/memory/available": 6736568320, "system/memory/used": 1492824064, "system/memory/used_percent": 17.851688, "process/memory/used": 71762168 } ``` ## Server Configuration Options ### General Environment Variables | Name | Type | Default | Description | | ---------------------------- | ------ | --------- | ---------------------------------------------------------------------------------------------------------------------------- | | `HOST` | String | "0.0.0.0" | Hostname or IP address. | | `PORT` | Number | 8080 | TCP port to listen on. | | `AUTHORIZED_USERS` | String | "" | List of PKI Distinguished Names (DNs) approved to access metrics views in the dashboard. An empty string permits all access. | | `CONTROL_SERVER_RETRY_DELAY` | String | "10s" | Delay duration for indefinite connection retries to each configured Control server. | | `INSTANCE_POLLING_INTERVAL` | String | "5s" | The interval at which health checks are made to service instances. | | `INSTANCE_MAX_SILENCE` | String | "15s" | The wait time before a non-responsive service instance is considered down. | | `METRICS_MAX_RETRIES` | Number | 3 | The maximum number of retries made before stopping requests to a service instance's metrics endpoint. | | `METRICS_RETRY_DELAY` | String | "10s" | The wait period between attempts to connect to a service instance's metrics endpoint. | | `METRICS_REQUEST_TIMEOUT` | String | "15s" | The wait time to indicate a request timeout from a service instance's metrics endpoint. | #### Configured via Environment Variables As a second option, up to 10 connections can be created to one or more Control servers using environment variables. However, note that these environment variables will only be read if no existing Zone object data is sourced from Redis or Grey Matter Data. Each environment variable for a specific control instance should have its own index number by replacing ``. For example, `CONTROL_SERVER_0_ADDRESS`, `CONTROL_SERVER_1_ADDRESS`, etc. | Name | Type | Example | Description | | ------------------------------------ | ------- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CONTROL_SERVER__ZONE_NAME` | String | "region-1" | Federated zone name used for tracking, retrieving, and updating a mesh in the Catalog. Each mesh represents a collection of service clusters. | | `CONTROL_SERVER__REQUEST_CLUSTER` | String | "edge" | The cluster used in the request made to Control server to retrieve discovered service instances (i.e. the edge node proxy). `CONTROL_SERVER__REQUEST_CLUSTER_NAME` may also be used. | | `CONTROL_SERVER__ADDRESS` | String | "localhost:50000" | Address for establishing a gRPC connection to a Control server. | | `CONTROL_SERVER__USE_TLS` | Boolean | false | Whether to connect to the Control server over TLS. | | `CONTROL_SERVER__CA_PATH` | String | "/etc/certs/ca.crt" | Path to the certificate authority file. | | `CONTROL_SERVER__CERT_PATH` | String | "/etc/certs/server.crt" | Path to the server certificate. | | `CONTROL_SERVER__KEY_PATH` | String | "/etc/certs/server.key" | Path to the server certificate key. | ### TLS In Grey Matter service mesh deployments, Transport Layer Security (TLS) is typically handled by a [Grey Matter Sidecar](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/grey-matter-proxy) sidecar in front of the Catalog Service. In other words, the Catalog Service is treated like any other service in the service mesh, whereby security is offloaded to the sidecar proxy. However, this service can run standalone, and in these cases the following environment variables can be set to configure TLS. | Name | Type | Default | Description | | ------------------ | ------- | ------- | --------------------------------------- | | `USE_TLS` | Boolean | false | Enables TLS | | `CA_CERT_PATH` | String | "" | Path to the certificate authority file. | | `SERVER_CERT_PATH` | String | "" | Path to the server certificate. | | `SERVER_KEY_PATH` | String | "" | Path to the server certificate key. | ### Persistence | Name | Type | Default | Description | | ------------------------- | ------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CONFIG_SOURCE` | String | "env" | Configures where to store and read service configuration from. Set value to `redis` to store in Redis. Set value to `gmdata` to store in Grey Matter Data. Set value to `env` to use environment variables. | | `CONFIG_POLLING_INTERVAL` | String | "3s" | Interval for checking the store for configuration changes. Only applicable when `CONFIG_SOURCE=gmdata`. | As indicated above, there are three options for configuring services known by the Catalog service - `redis`, `gmdata`, and `env`. Think about your deployment model before choosing either of these options, which are highlighted below. #### Configure Persistence with Redis When the Grey Matter service mesh is deployed via our [Helm charts](https://github.com/greymatter-io/helm-charts), Catalog will already be configured to persist service configuration in an internal instance of Redis. This is our recommended approach for production deployments where Catalog may be scaled up to multiple nodes. Changes to services stored in Redis are made possible via the Catalog RESTful API. It is also recommended to persist the data written to Redis to disk by setting the `appendonly` setting in your Redis server configuration. [See the Redis documentation for more details](https://redis.io/topics/persistence). Below is a list of environment variables required to use Redis as the persistent store. Note, that these are automatically set when deploying with our [Helm charts](https://github.com/greymatter-io/helm-charts). | Name | Type | Default | Description | | ------------------------ | ------- | ----------- | -------------------------------------------------- | | `REDIS_HOST` | String | "localhost" | Host of the Redis connection | | `REDIS_PORT` | Integer | 6379 | Port of the Redis connection | | `REDIS_USE_TLS` | Boolean | false | Whether to connect to Redis over TLS | | `REDIS_CA_CERT_PATH` | String | "" | Path to SSL CA on disk for connecting to Redis | | `REDIS_SERVER_CERT_PATH` | String | "" | Path to SSL Cert on disk for connecting to Redis | | `REDIS_SERVER_KEY_PATH=` | String | "" | Path to SSL Key on disk for connecting to Redis | | `REDIS_PASS` | String | "" | Password if required by Redis | | `REDIS_DB` | Integer | 0 | Which database to use in Redis | | `REDIS_MAX_RETRIES` | String | 5 | Max number of times to attempt to connect to Redis | | `REDIS_RETRY_DELAY` | String | "1s" | Duration between Redis connection attempts | #### Configure Persistence with Grey Matter Data A second configuration option is to use Grey Matter Data. Note that this option provides limited support for scaling up the Catalog server. | Name | Type | Default | Description | | ---------------------------------- | ------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `CLIENT_ADDRESS` | String | "" | Hostname or IP address of Grey Matter Data. | | `CLIENT_PORT` | String | "" | TCP port exposed by Grey Matter Data. | | `CLIENT_PREFIX` | String | "" | Grey Matter Data RESTful API prefix. | | `CLIENT_IDENTITY` | String | "" | Distinguished Name (DN) of JWT user associated with Catalog, see [Helm Charts](https://github.com/greymatter-io/helm-charts/greymatter.yaml) under `internal-jwt-users` for details. | | `CLIENT_EMAIL` | String | "" | Email address used as the directory name where services JSON will be persisted. | | `CLIENT_USE_TLS` | Boolean | false | Enable TLS. | | `CLIENT_CERT` | String | "" | base64 encoded certificate for TLS. | | `CLIENT_KEY` | String | "" | base64 encoded key for TLS. | | `CLIENT_TRUST` | String | "" | base64 encoded certificate for TLS. | | `INSECURE_TLS` | Boolean | false | | | `MONGO_USE_TLS` | Boolean | false | | | `GMDATA_STARTUP_DELAY` | String | "5s" | Delay at startup to allow all instances to write to Grey Matter Data. | | `GMDATA_ROOT_EVENT_NAME` | String | "world" | Name of the top level folder in Grey Matter Data where `services.json` will be persisted. | | `GMDATA_MAX_EVENTS` | Number | 1000 | | | `GMDATA_POLLING_INTERVAL` | String | "2s" | The interval at which Catalog will poll Grey Matter Data for updated services.json | | `GMDATA_MAX_RETRIES` | Number | 10 | | | `GMDATA_RETRY_DELAY` | String | "1s" | The delay for retry attempts for the initial Grey Matter Data connection. | | `GMDATA_OBJECT_POLICY_TEMPLATE` | String | `(if (contains email %s)(yield-all)(yield R X)))` | Grey Matter Data object policy template to enforce permissions on `servics.json` folder. Default policy allows read and execute for the Catalog service. | | `GMDATA_POLICY_LABEL` | String | "localuser owned" | | | `GMDATA_TOP_LEVEL_FOLDER_SECURITY` | String | `{"label":"PUBLIC//EVERYBODY","foreground":"#FFFFFF","background":"#008800"}` | | | `GMDATA_SERVICES_FILE_SECURITY` | String | `{"label":"PUBLIC//EVERYBODY","foreground":"#FFFFFF","background":"#008800"` | | #### Configure Persistence with Environment Variables The third configuration option is to statically define the metadata associated with each service via environment variables. This is a quick way to get Catalog running without having to setup Redis or Grey Matter Data but there are some pitfalls with this approach. {% hint style="danger" %} **Pitfalls of Configuring Persistence with Environment Variables** If you switch to using Redis or Grey Matter Data after using environment variables then the metadata will be stored in the database and subsequent changes to the environment variables won't take effect. You can end up creating a disconnect between configurations if you switch back and forth. If you use the Catalog RESTful API to make configuration changes and then switch back to using environment variables, you will need to keep environment variables in sync manually with those changes. {% endhint %} The following environment variables define the additional metadata associated with each service known by Catalog. Each environment variable for a specific service instance should have its own index number by replacing ``. For example, `SERVICE_0_CLUSTER_NAME`, `SERVICE_1_CLUSTER_NAME`, etc. | Name | Type | Default | Description | | --------------------------------------- | ------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SERVICE__CLUSTER_NAME` | String | "" | The name of the cluster (logical group name of service instances), provided by gm-control. | | `SERVICE__ZONE_NAME` | String | "" | The name of the zone associated with the cluster, provided by gm-control. This must match the value of `CONTROL_SERVER__ZONE_NAME` defined in Control's environment variables. | | `SERVICE__NAME` | String | "" | The name of the service to be displayed in the Grey Matter application (make this as human friendly as possible). | | `SERVICE__DESCRIPTION` | String | "" | A summary description of the service to be displayed in the Grey Matter application (make this as human friendly as possible). | | `SERVICE__VERSION` | String | "" | The semver version of the service. | | `SERVICE__DOCUMENTATION` | String | "" | A URL path to documentation relative to the root URL of the service or a full path depending on your needs. | | `SERVICE__API_ENDPOINT` | String | "" | A URL path to the API endpoint of the service. | | `SERVICE__API_SPEC_ENDPOINT` | String | "" | A URL path to the OpenAPI specification document of the service. | | `SERVICE__OWNER` | String | "" | The name of the service owner like "Decipher" or "Cool Customer". Used to sort by Owner in the Grey Matter application. | | `SERVICE__OWNER_URL` | String | "" | A URL path to the owner of the service. | | `SERVICE__CAPABILITY` | String | "" | The name of the service capability like "Storage" or "Security". Used to sort by Capability in the Grey Matter application. | | `SERVICE__BUSINESS_IMPACT` | String | "low" | A business impact rating given to the service; must be one of: critical, high, medium, or low. | | `SERVICE__PROMETHEUSJOB` | String | "" | The name of the Prometheus Job used to store and query time series data associated with this service. This **must** match the cluster name. | | `SERVICE__MIN_INSTANCES` | Number | 1 | The minimum number of instances this service should scale to. If below this number, the service will be in a warning state in the Grey Matter application. | | `SERVICE__MAX_INSTANCES` | Number | 1 | The maximum number of instances this service should scale to. If above this number, the service will be in a warning state in the Grey Matter application. | | `SERVICE__ENABLE_INSTANCE_METRICS` | Boolean | true | Enable the instance metrics view in the Grey Matter application. | | `SERVICE__ENABLE_HISTORICAL_METRICS` | Boolean | false | Enable the historical metrics view in the Grey Matter application. | | `SERVICE__METRICS_TEMPLATE` | String | "" | URL template for constructing the service's instance metrics endpoint e.g. `http://{{host}}:{{port}}/metrics`. | | `SERVICE__METRICS_PORT` | Number | 8081 | TCP port serving up service instance metrics. | | `SERVICE__EXTERNAL_LINKS` | String | "" | A JSON object with string fields "name" and "link" for linking relevant external websites in the Grey Matter application. | ### Configuration File {% hint style="info" %} **Note:** the following is not a typical deployment model. {% endhint %} If you're running the Catalog executable binary manually you can also provide a configuration file with key/value pairs.*\*\** ``` ./gm-catalog --config=settings.toml ``` If there are conflicting configurations between this file and environment variables, then the environment variables will take precedence. ## Questions {% hint style="success" %} **Need help configuring Grey Matter Catalog?** Contact us at . {% endhint %}