Catalog
Server Setup for Grey Matter Catalog
Last updated
Was this helpful?
1.2
Server Setup for Grey Matter Catalog
Last updated
Was this helpful?
Grey Matter Catalog provides additional metadata about services to the . It connects to ) to retrieve discovered services and combines them with user driven configuration to create summarized metadata about services running within a service mesh. Although it was designed to power the Grey Matter Intelligence 360 Application, this service can be leveraged as an API to develop further visualizations of a service mesh.
See our guide on for how to interact with this service when it's up and running.
Name
Type
Default
Description
HOST
String
"0.0.0.0"
Hostname or IP address.
PORT
Number
8080
TCP port to listen on.
AUTHORIZED_USERS
String
""
List of PKI Distinguished Names (DNs) approved to access metrics views in the dashboard. An empty string permits all access.
INSTANCE_POLLING_INTERVAL
String
"5s"
The interval at which health checks are made to service instances.
INSTANCE_MAX_SILENCE
String
"15s"
The wait time before a non-responsive service instance is considered down.
METRICS_MAX_RETRIES
Number
3
The maximum number of retries made before stopping requests to a service instance's metrics endpoint.
METRICS_RETRY_DELAY
String
"10s"
The wait period between attempts to connect to a service instance's metrics endpoint.
METRICS_REQUEST_TIMEOUT
String
"15s"
The wait time to indicate a request timeout from a service instance's metrics endpoint.
The Catalog service discovers service instances from Grey Matter Control and up to ten instances of Control may be configured. Each environment variable for a specific control instance should have its own index number by replacing <N>. For example, CONTROL_SERVER_0_ADDRESS, CONTROL_SERVER_1_ADDRESS, etc.
Name
Type
Default
Description
CONTROL_SERVER_<N>_ADDRESS
String
"localhost:50000"
Address for a number of Control servers.
CONTROL_SERVER_<N>_ZONE_NAME
String
"region-1"
Federated zone name used for tracking, retrieving, and updating a mesh in the Catalog. Each mesh represents a collection of service clusters.
CONTROL_SERVER_<N>_REQUEST_CLUSTER_NAME
String
"edge"
The cluster used in the request made to Control to retrieve discovered service instances (i.e. the edge node proxy).
CONTROL_SERVER_RETRY_DELAY
String
"5s"
For each Control server, a "retry delay" can be specified. If an error occurs during creation of a client connection, establishing a stream, or sending a DiscoveryRequest, Catalog will restart the connection process. This occurs indefinitely and can only be stopped by sending a DELETE request to Control's /zones endpoint with the configured server's specified zoneName.
Name
Type
Default
Description
USE_TLS
Boolean
false
Enables TLS
CA_CERT_PATH
String
""
Path to the certificate authority file.
SERVER_CERT_PATH
String
""
Path to the server certificate.
SERVER_KEY_PATH
String
""
Path to the server certificate key.
Name
Type
Default
Description
CONFIG_SOURCE
String
"env"
Configures where to store and read service configuration from. Set value to gmdata to store in Grey Matter Data. Set value to env to use environment variables.
CONFIG_POLLING_INTERVAL
String
"3s"
Interval for checking the store for configuration changes. Only applicable when CONFIG_SOURCE=gmdata.
As indicated above, there are two options for configuring services known by the Catalog service - gmdata and env. Think about your deployment model before choosing either of these options, which are highlighted below.
Name
Type
Default
Description
CLIENT_ADDRESS
String
""
Hostname or IP address of Grey Matter Data.
CLIENT_PORT
String
""
TCP port exposed by Grey Matter Data.
CLIENT_PREFIX
String
""
Grey Matter Data RESTful API prefix.
CLIENT_IDENTITY
String
""
CLIENT_EMAIL
String
""
Email address used as the directory name where services JSON will be persisted.
CLIENT_USE_TLS
Boolean
false
Enable TLS.
CLIENT_CERT
String
""
base64 encoded certificate for TLS.
CLIENT_KEY
String
""
base64 encoded key for TLS.
CLIENT_TRUST
String
""
base64 encoded certificate for TLS.
INSECURE_TLS
Boolean
false
MONGO_USE_TLS
Boolean
false
GMDATA_STARTUP_DELAY
String
"5s"
Delay at startup to allow all instances to write to Grey Matter Data.
GMDATA_ROOT_EVENT_NAME
String
"world"
Name of the top level folder in Grey Matter Data where services.json will be persisted.
GMDATA_MAX_EVENTS
Number
1000
GMDATA_POLLING_INTERVAL
String
"2s"
The interval at which Catalog will poll Grey Matter Data for updated services.json.
GMDATA_MAX_RETRIES
Number
10
GMDATA_RETRY_DELAY
String
"1s"
The delay for retry attempts for the initial Grey Matter Data connection.
GMDATA_OBJECT_POLICY_TEMPLATE
String
(if (contains email %s)(yield-all)(yield R X)))
Grey Matter Data object policy template to enforce permissions on servics.json folder. Default policy allows read and execute for the Catalog service.
GMDATA_POLICY_LABEL
String
"localuser owned"
GMDATA_TOP_LEVEL_FOLDER_SECURITY
String
{"label":"PUBLIC//EVERYBODY","foreground":"#FFFFFF","background":"#008800"}
GMDATA_SERVICES_FILE_SECURITY
String
{"label":"PUBLIC//EVERYBODY","foreground":"#FFFFFF","background":"#008800"
The second configuration option is to statically define the metadata associated with each service via environment variables. This is a quick way to get Catalog running without having to setup Grey Matter Data but there are some pitfalls with this approach.
Pitfalls of Configuring Persistence with Environment Variables
If you switch to using Grey Matter Data after using environment variables then the metadata will be stored in Grey Matter Data and subsequent changes to the environment variables won't take effect. You can end up creating a disconnect between configurations if you switch back and forth.
If you use the Catalog RESTful API to make configuration changes and then switch back to using environment variables, you will need to keep environment variables in sync manually with those changes.
The following environment variables define the additional metadata associated with each service known by Catalog. Each environment variable for a specific service instance should have its own index number by replacing <N>. For example, SERVICE_0_CLUSTER_NAME, SERVICE_1_CLUSTER_NAME, etc.
Name
Type
Default
Description
SERVICE_<N>_CLUSTER_NAME
String
""
The name of the cluster (logical group name of service instances), provided by gm-control.
SERVICE_<N>_ZONE_NAME
String
""
The name of the zone associated with the cluster, provided by gm-control. This must match the value of CONTROL_SERVER_<N>_ZONE_NAME defined in Control's environment variables.
SERVICE_<N>_NAME
String
""
The name of the service to be displayed in the Intelligence 360 Dashboard (make this as human friendly as possible).
SERVICE_<N>_VERSION
String
""
The semver version of the service.
SERVICE_<N>_OWNER
String
""
The name of the service owner like "Decipher" or "Cool Customer". Used to sort by Owner in the Intelligence 360 Dashboard.
SERVICE_<N>_CAPABILITY
String
""
The name of the service capability like "Storage" or "Security". Used to sort by Capability in the Intelligence 360 Dashboard.
SERVICE_<N>_DOCUMENTATION
String
""
A URL path to documentation relative to the root URL of the service or a full path depending on your needs.
SERVICE_<N>_PROMETHEUSJOB
String
""
The name of the Prometheus Job used to store and query time series data associated with this service. This must match the cluster name.
SERVICE_<N>_MIN_INSTANCES
Number
1
The minimum number of instances this service should scale to. If below this number, the service will be in a warning state in the Intelligence 360 Dashboard.
SERVICE_<N>_MAX_INSTANCES
Number
1
The maximum number of instances this service should scale to. If above this number, the service will be in a warning state in the Intelligence 360 Dashboard.
SERVICE_<N>_ENABLE_INSTANCE_METRICS
Boolean
true
Enable the instance metrics view in the Intelligence 360 Dashboard.
SERVICE_<N>_ENABLE_HISTORICAL_METRICS
Boolean
false
Enable the historical metrics view in the Intelligence 360 Dashboard.
SERVICE_<N>_METRICS_TEMPLATE
String
""
URL template for constructing the service's instance metrics endpoint e.g. http://{{host}}:{{port}}/metrics.
SERVICE_<N>_METRICS_PORT
Number
8081
TCP port serving up service instance metrics.
If you're running the Catalog executable binary manually you can also provide a configuration file with key/value pairs.**
If there are conflicting configurations between this file and environment variables, then the environment variables will take precedence.
In Grey Matter service mesh deployments, Transport Layer Security (TLS) is typically handled by a sidecar in front of the Catalog Service. In other words, the Catalog Service is treated like any other service in the service mesh, whereby security is offloaded to the sidecar proxy. However, this service can run standalone, and in these cases the following environment variables can be set to configure TLS.
When the Grey Matter service mesh is deployed via our , Catalog will already be configured to persist service configuration in an internal instance of Grey Matter Data. This is our recommended approach for production deployments because of the added security and change history provided by Grey Matter Data. Changes to services stored in Grey Matter Data are made possible via the Catalog RESTful API.
Below is a list of environment variables required to use Grey Matter Data as the persistent store. Note, that these are automatically set when deploying with our .
Distinguished Name (DN) of JWT user associated with Catalog, see under internal-jwt-users for details.
Need help configuring Grey Matter Catalog? Contact us at .