kubernetes

Kubernetes (k8s) has a number of internal APIs that are used for the complex orchestration of containers. When using k8s as the underlying orchestration platform, gm-control can utilize some of these APIs to also do easy service announcement and discovery.

Pod Label and Named Port

To discover services, two important bits of information must be configured on each deployed pod; the cluster label and the port name. Without these two, pods will be ignored by the control plane.

Cluster Label

The cluster label is a small piece of metadata attached to the pod to determine what specific service is running in this pod. All pods of the same service will be grouped together and load balanced in the mesh. The default metadata key that will be used is gm_cluster=<service_name>, but this is user configurable.

NOTE The default label can be changed by the CLI flag --cluster-label or environment variable GM_CONTROL_KUBERNETES_CLUSTER_LABEL when running gm-control

Port Name

The port name determines which named port to expose in the mesh. Since it's possible for many ports to be open for different purposes, just one must be designated for routing normal traffic. This defaults to the port named http, but is user configurable.

NOTE The default named port can be changed by the CLI flag --port-name or environment variable GM_CONTROL_KUBERNETES_PORT_NAME when running gm-control

Example Deployment

The Kubernetes Deployment below is properly setup (label and port) to be discovered by the gm-control server.

apiVersion: apps/v1
kind: Deployment
spec:
  selector:
    matchLabels:
      app: example
  replicas: 1
  template:
    metadata:
      labels:
        gm_cluster: example
    spec:
      containers:
      - name: example-service
        image: deciphernow/example-service:latest
      - name: sidecar
        image: docker.greymatter.io/release/gm-proxy:1.4.5
        imagePullPolicy: Always
        ports:
        - name: http
          containerPort: 9080
        - name: metrics
          containerPort: 8081
        env:
        - name: PROXY_DYNAMIC
          value: "true"
        - name: XDS_CLUSTER
          value: example
        - name: XDS_HOST
          value: gm-control
        - name: XDS_PORT
          value: "50000"

Service Accounts

To discover services from the internal Kubernetes APIs, the pod running the Control server must have additional permissions granted by an admin. To grant this access, have a cluster admin apply one of the following resources (replacing the greymatter namespace with the namespace of the running control server), and then add the service account to the pod running gm-control

    spec:
      serviceAccountName: control

Single Namespace

This is used when gm-control is only discovering services in the same namespace it's running in.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: control
  namespace: greymatter

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: control-manager
  namespace: greymatter
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list"]

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: control-binding
  namespace: greymatter
subjects:
- kind: ServiceAccount
  name: control
  namespace: greymatter
roleRef:
  kind: Role
  name: control-manager
  apiGroup: rbac.authorization.k8s.io

Multiple Namespaces

This is used when gm-control will be discovering services from multiple namespaces across the cluster.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: control
  namespace: greymatter

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: control-manager
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["list"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: control-binding
subjects:
- kind: ServiceAccount
  name: control
  namespace: greymatter
roleRef:
  kind: ClusterRole
  name: control-manager
  apiGroup: rbac.authorization.k8s.io