# Control The Control server performs service discovery in the mesh and acts as the Envoy xDS server to which all proxies connect. Service discovery can be performed through a [variety of mechanisms](https://greymatter.gitbook.io/grey-matter-documentation/1.3/usage/discovery). ## Usage The Control server runs on a number of different platforms and supports a variety of discovery mechanisms. Each one has its own configuration options, the details of which are outlined in the [usage documentation](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-base-command). Configuration can be done via environment variables or command line flags. If both are used, then CLI flags take precedence. Environment variables for global options are of one of the following forms: `GM_CONTROL_API_`, `GM_CONTROL_XDS_`, `GM_CONTROL_STATS_`, and for discovery mechanism specific types are of the form `GM_CONTROL__`, where `` is the value of the `GM_CONTROL_CMD` variable specifying service discovery. Correspondingly, the Control CLI command is of the form: ```bash gm-control -- --api. --xds. --stats. ``` See the following [configuration](#configuration) section for the common configuration options. ## Configuration Configuring the Grey Matter Control server is based on the service discovery mechanism. To configure a server, set the following desired global configuration options, **and** see the [setup documentation for the desired discovery type](#service-discovery-configuration): | Environment Variable | Description | Type | Default | Required | | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | -------- | | `GM_CONTROL_API_KEY` | The auth key for gm-control requests | `string` | `"none"` | `true` | | `GM_CONTROL_API_ZONE_NAME` | Name of default [api zone](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/api/fabric-api/zone) to create on startup | `string` | | `true` | | `GM_CONTROL_API_ZONE` | Zone key of default [api zone](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/api/fabric-api/zone) created on startup | `string` | `zone-default-zone` | `false` | | `GM_CONTROL_API_INSECURE` | Connect to control-api without certificate verification | `boolean` | `true` | `false` | | `GM_CONTROL_API_SSL` | Connect to control-api server using HTTPS | `boolean` | `false` | `false` | | `GM_CONTROL_API_SSLCERT` | If `GM_CONTROL_API_SSL` is true, the certificate file to use | `string` | `""` | `false` | | `GM_CONTROL_API_SSLKEY` | If `GM_CONTROL_API_SSL` is true, the key file to use | `string` | `""` | `false` | | `GM_CONTROL_API_HOST` | The address (host:port) of the control-api server | `string` | | `true` | | `GM_CONTROL_CMD` | Service discovery mechanism (one of: `file`, `kubernetes`, `consul`, `aws`, `ecs`) | `string` | | `true` | | `GM_CONTROL_CONSOLE_LEVEL` | Control server log level | `string` | `info` | `false` | | `GM_CONTROL_CONSOLE_FORMAT` | Control server log format (`console`, `json`) | `string` | `json` | `false` | | `GM_CONTROL_XDS_RESOLVE_DNS` | If true, control will resolve discovered hostnames to IP addresses. | `string` | `true` | `false` | | `GM_CONTROL_XDS_ENABLE_REST` | If true, enable REST api type instead of gRPC | `boolean` | `false` | `false` | | `GM_CONTROL_DIFF_IGNORE_CREATE` | If true, control will not auto-create new Clusters in the API for discovered services. | `string` | `false` | `false` | | `GM_CONTROL_XDS_ENABLE_TLS` | Enable TLS on the gRPC control server | `boolean` | `false` | `false` | | `GM_CONTROL_XDS_SERVER_CERT` | Path to the server certificate | `string` | | `false` | | `GM_CONTROL_XDS_SERVER_KEY` | Path to the server certificate key | `string` | | `false` | | `GM_CONTROL_XDS_SERVER_TRUSTS` | Comma-delimited paths to a certificate authority | `string` | `false` | `false` | | `GM_CONTROL_XDS_SERVER_AUTH_TYPE` | The policy the server will follow for TLS Client Authentication - one of `noclientcert`, `requestclientcert`, `requireanyclientcert`, `verifyclientcertifgiven`, or `requireandverifyclientcert`. Required if `GM_CONTROL_XDS_ENABLE_TLS` is true. | `string` | `false` | `false` | > Note: a full list of detailed configuration options can be found [here](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/configuration-specifics). ### Service Discovery Configuration Use the corresponding documentation to configure based on the value of `GM_CONTROL_CMD`: * [aws ec2](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-aws) * [aws ecs](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-ecs) * [consul](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-consul) * [flat file](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-file) * [kubernetes](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-kubernetes) * [marathon](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-marathon) * [xds](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-xds-only) As described [below](#multiple-server-deployment-for-mixed-service-discovery), Grey Matter supports the use of multiple Control server's for multiple service discovery types within one mesh. #### Logging There are two configurable logging options for Grey Matter Control. As described above, the log level can be specified with `GM_CONTROL_CONSOLE_LEVEL`. There is also the option to set `GM_CONTROL_CONSOLE_FORMAT` to one of `console` or `json`. It's default value is `json`. To enable [pretty logging](https://github.com/rs/zerolog#pretty-logging), set this value to `console`. ### Additional Features * [Leaderboard Logging](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/leaderboard_logging) * [Static Resource Definition](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/load_from_static_resources) * [Access Log Service Stats](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/als_stats) ## Deployment Patterns Grey Matter Control is a stateless microservice, and supports scaling to whatever size cluster is required. If a Control server goes offline, the data plane (connected mesh of Sidecars) will continue to function, but without any updates from new configuration or changed instances. When a new Control instance is brought up it will read the current state of Sidecar instances through service discovery and the current configuration for the mesh from gm-control-api to get back to the original state (or new state, if either has progressed since). {% hint style="warning" %} **\Memory Utilization Requirements** Each Control server requires \~1.25MB of memory per connected sidecar. This should be reflected in any enforced memory limits to prevent server restart due to OOM errors. {% endhint %} ### Single-Node Deployment In a single-node deployment, the one `gm-control` instance will do service discovery, read mesh configuration from `gm-control-api`, and serve data plane configuration to all Sidecars in the mesh. ### N-Node Deployment When multiple Control servers are deployed, they should be provisioned behind a Load Balancer. This will distribute the load of connected Sidecars across all instances of the service, making each one responsible for serving configs out to only a fraction of the data plane. Even with multiple Control servers, each one still independently performs service discovery and reading mesh configuration from `gm-control-api`. This allows each `gm-control` node to see the mesh in it's entirety, and to take over control of additional sidecars should a node go down. If this does happen, existing sidecars will shift their connection to a live instance and continue to receive data plane updates. ### Multiple server deployment for mixed service discovery Multiple Control servers can also be deployed to use multiple different service discovery mechanisms for a single mesh. This allows services from different environments to be discovered and interact with each other. In this use case, no load balancer will be used, because each Control server will operate independently of any other(s), connecting to the same Grey Matter Control API. ## REST Support Grey Matter Control can be configured to serve a REST api gateway by setting the `GM_CONTROL_XDS_ENABLE_REST` environment variable to `true`. This is **required** to use Amazon ECS type service discovery (see the [ECS](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/gm-control/gm-control-ecs) docs). Also note that when this is enabled, each [Grey Matter Proxy](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/fabric-1/grey-matter-proxy) attempting to connect to control will need to have both of the following environment variables set: * `PROXY_REST_DYNAMIC=true` * `XDS_PORT=50001` > Note: To change the port over which the HTTP gateway is served, you can set `GM_CONTROL_XDS_GATEWAY_ADDR` to the desired address. `XDS_PORT` set for the Grey Matter Sidecar's must then match the port for this value. ## TLS Support The Grey Matter Control gRPC server can be configured to serve with TLS. The following block demonstrates the necessary environment variables: ```bash GM_CONTROL_XDS_ENABLE_TLS=true GM_CONTROL_XDS_SERVER_CERT="/certs/server.crt" GM_CONTROL_XDS_SERVER_KEY="/certs/server.key" GM_CONTROL_XDS_SERVER_TRUSTS="/certs/ca.crt" GM_CONTROL_XDS_SERVER_AUTH_TYPE="requireandverifyclientcert" ``` Grey Matter Sidecars must also be configured to connect to control with TLS. See the ["Connect to Control"](https://greymatter.gitbook.io/grey-matter-documentation/1.3/reference/setup/grey-matter-proxy/connect-to-control#Use-TLS-\(Optional\)) documentation to learn more. ## Grey Matter Dependencies | Version | Dependency Name | Dependency Version | | ------- | --------------- | ------------------ | | 1.x | gm-control-api | 1.x | `gm-control` can stream valid instructions to envoy-compatible proxies of v1.x