# Kubernetes Discovery ## Kubernetes ### Description Grey Matter Control discovers service instances from kubernetes pods using k8s namespace and pod specifications. ### Usage To use kubernetes discovery with Grey Matter Control running within the kubernetes cluster, set the environment variables specified [here](#server-running-inside-of-kubernetes-cluster). To use kubernetes discovery with Grey Matter Control running outside of the kubernetes cluster, set the environment variables specified [here](#server-running-outside-of-kubernetes-cluster). | Environment Variable | Description | Type | Default | Required | | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------- | ------------ | -------- | | `GM_CONTROL_KUBERNETES_NAMESPACES` | Comma-delimited list of kubernetes namespaces to discover pods in | `string` | `default` | `false` | | `GM_CONTROL_KUBERNETES_PORT_NAME` | Named container port assigned to cluster instances | `string` | `http` | `false` | | `GM_CONTROL_KUBERNETES_CLUSTER_LABEL` | Kubernetes label that specifies the cluster a pod belongs to | `string` | `gm_cluster` | `false` | | `GM_CONTROL_KUBERNETES_SELECTOR` | Kubernetes label selector that selects which pods are polled | `string` | | `false` | | `GM_CONTROL_KUBERNETES_TIMEOUT` | Timeout in seconds used for Kubernetes API requests | `string` | `120` | `false` | | `GM_CONTROL_KUBERNETES_LOG_LEVEL` | Log level used for this discovery plugin | `string` | `info` | `false` | | `GM_CONTROL_KUBERNETES_KUBERNETES_HOST` | Hostname to connect to kubernetes API server. Required if the Control server is to run outside of the kubernetes cluster | `string` | | `false` | | `GM_CONTROL_KUBERNETES_CA_CERT` | Path to trusted root cert for kubernetes API server. | `string` | | `false` | | `GM_CONTROL_KUBERNETES_CLIENT_KEY` | Path to key file for Control server to authenticate itself with the kubernetes API server. | `string` | | `false` | | `GM_CONTROL_KUBERNETES_CLIENT_CERT` | Path to certificate file for Control server to authenticate itself with the kubernetes API server. | `string` | | `false` | ### Server Running Inside of Kubernetes Cluster As described [above](#usage) in the required environment variables, for a Control server running within the Kubernetes cluster it will be discovering from, there are no required variables outside of `GM_CONTROL_CMD`. However, the full set of optional variables is: ```bash # required GM_CONTROL_CMD=kubernetes # optional GM_CONTROL_KUBERNETES_NAMESPACES={} GM_CONTROL_KUBERNETES_PORT_NAME={} GM_CONTROL_KUBERNETES_CLUSTER_LABEL={} GM_CONTROL_KUBERNETES_SELECTOR={} GM_CONTROL_KUBERNETES_TIMEOUT={} GM_CONTROL_KUBERNETES_LOG_LEVEL={} ``` ### Server Running Outside of Kubernetes Cluster For a Control server running outside of the Kubernetes cluster it will be discovering from, there are several required variables: ```bash # required GM_CONTROL_CMD=kubernetes GM_CONTROL_KUBERNETES_KUBERNETES_HOST={} GM_CONTROL_KUBERNETES_CA_CERT={} GM_CONTROL_KUBERNETES_CLIENT_KEY={} GM_CONTROL_KUBERNETES_CLIENT_CERT={} # optional GM_CONTROL_KUBERNETES_NAMESPACES={} GM_CONTROL_KUBERNETES_PORT_NAME={} GM_CONTROL_KUBERNETES_CLUSTER_LABEL={} GM_CONTROL_KUBERNETES_SELECTOR={} GM_CONTROL_KUBERNETES_TIMEOUT={} GM_CONTROL_KUBERNETES_LOG_LEVEL={} ``` ### Kubernetes Specifications Grey Matter Control will discover kubernetes pod instances based on the criteria specified in the environment variables. It will watch any kubernetes namespaces specified by name in the list of `GM_CONTROL_KUBERNETES_NAMESPACES`. If this value is not set, it will watch the `default` namespace only. Only pods within the specified namespaces that also have the pod label in `GM_CONTROL_KUBERNETES_SELECTOR` will be polled for discovery. By default this value is nil, so all pods within the namespaces will polled. In each pod, all containers must be running before the pod is considered live and ready for inclusion in the API cluster's instance list. Pods that are being polled will then be checked for another pod label, specified with `GM_CONTROL_KUBERNETES_CLUSTER_LABEL` - or `gm_cluster` by default. In order to link a pod to an api cluster, the pod **must** have this label with value equal to the api cluster name that it corresponds to. For example, if using the default value and a pod has label: `gm_cluster: test-cluster`, the pod will be discovered for cluster with name `test-cluster`. Instances will be created for the cluster with name corresponding to the value of the pod label with key `gm_cluster`. The host will be the podIP of this pod, and the port will be taken from the container port on the pod with name `GM_CONTROL_KUBERNETES_PORT_NAME`. If `GM_CONTROL_KUBERNETES_PORT_NAME` is empty, the first TCP port found is used as the API instance's port. Pods with no container port are ignored. All pod labels (except for the cluster label) are attached as instance metadata. For example, for a Control server running in a Kubernetes cluster using all of the default values, configuration: ```bash GM_CONTROL_CMD=kubernetes ``` Pods in the default namespace, with pod label `gm_cluster` and value of the cluster-name and containing a container with port named `http`, will be discovered. ### Kubernetes Permissions If the Grey Matter Control server is running within the kubernetes cluster in its own pod, it will need certain permissions to poll the namespaces specified in `GM_CONTROL_KUBERNETES_NAMESPACES`. See [service accounts](#service-accounts) and [single namespace permissions](#single-namespace) if only one namespace is specified (or if the default `default` is being used), or [multiple namespace permissions](#multiple-namespaces) if you have specified a list of namespaces. #### Service Accounts The pod running the Control server must have an authorized service account granted by an admin. If access to any or all of the namespaces are revoked, Grey Matter Control will use its last known state of discovered pods. Meanwhile, it will continue attempting to connect to the internal Kubernetes APIs until access is restored. To grant access, have a cluster admin apply either the [Single Namespace](#single-namespace) or [Multiple Namespaces](#multiple-namespaces) resources (replacing the `greymatter` namespace with the namespace of the running `control` server). Then they can add the service account to the pod running Grey Matter Control as follows: ```yaml spec: serviceAccountName: control ``` **Single Namespace** This is used when Grey Matter Control is only discovering services in the same namespace it's running in. ```yaml apiVersion: v1 kind: ServiceAccount metadata: name: control namespace: greymatter --- kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: control-manager namespace: greymatter rules: - apiGroups: [""] resources: ["pods"] verbs: ["list"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: control-binding namespace: greymatter subjects: - kind: ServiceAccount name: control namespace: greymatter roleRef: kind: Role name: control-manager apiGroup: rbac.authorization.k8s.io ``` **Multiple Namespaces** This is used when Grey Matter Control will be discovering services from multiple namespaces across the cluster. ```yaml apiVersion: v1 kind: ServiceAccount metadata: name: control namespace: greymatter --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: control-manager rules: - apiGroups: [""] resources: ["pods"] verbs: ["list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: control-binding subjects: - kind: ServiceAccount name: control namespace: greymatter roleRef: kind: ClusterRole name: control-manager apiGroup: rbac.authorization.k8s.io ``` #### Command Line To use the command line, run gm-control with: ```bash gm-control kubernetes ``` **Help** For help or to list available options for kubernetes discovery using the cli, run `gm-control kubernetes --help`.