Deploy Service to Grey Matter on Kubernetes

This guide is a step by step walkthrough on deploying a new service into an existing Grey Matter deployment. This guide is for a SPIFFE/SPIRE enabled deployment. For a walkthrough on deploying a service into a non-SPIFFE/SPIRE Grey Matter setup, follow the Quickstart Launch Service to Kubernetes for a non-SPIFFE/SPIRE Deployment.

If you are looking for a quick service deployment and/or configuration guide, check out the Grey Matter templates.

Prerequisites

1. An existing Grey Matter deployment running on Kubernetes (tutorial)

2. kubectl or oc setup with access to the cluster

3. greymatter cli setup with access to the deployment

Overview

1. Launch pod with the service and sidecar

2. Create Fabric configuration for the sidecar to talk to the service

3. Create Fabric configuration for the Edge to talk to the sidecar

4. Add entry in Catalog service to display in the Intelligence 360 Applications

Steps

1. Launch pod

The service we'll launch is a simple Fibonacci service. It has one route: /fibonacci that calculates the fibonacci sequence of any integer supplied.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: fibonacci
    greymatter.io/control: fibonacci
  name: fibonacci
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fibonacci
      greymatter.io/control: fibonacci
  template:
    metadata:
      labels:
        app: fibonacci
        greymatter.io/control: fibonacci
    spec:
      containers:
        - name: fibonacci
          image: docker.greymatter.io/internal/fibonacci:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
        - name: sidecar
          image: "docker.greymatter.io/release/gm-proxy:1.4.5"
          imagePullPolicy: IfNotPresent
          ports:
            - name: metrics
              containerPort: 8081
            - name: proxy
              containerPort: 10808
          env:
            - name: ENVOY_ADMIN_LOG_PATH
              value: "/dev/stdout"
            - name: PROXY_DYNAMIC
              value: "true"
            - name: SPIRE_PATH
              value: "/run/spire/socket/agent.sock"
            - name: XDS_CLUSTER
              value: "fibonacci"
            - name: XDS_HOST
              value: "control.default.svc"
            - name: XDS_NODE_ID
              value: "default"
            - name: XDS_PORT
              value: "50000"
            - name: XDS_ZONE
              value: "zone-default-zone"
          volumeMounts:
            - name: spire-socket
              mountPath: /run/spire/socket
              readOnly: false
      imagePullSecrets:
        - name: docker.secret
      volumes:
        - name: spire-socket
          hostPath:
            path: /run/spire/socket
            type: DirectoryOrCreate

Note the spire-specific configurations to the deployment - the volume and volume mount spire-socket and the environment variable SPIRE_PATH. These are the additions that will need to be made to any deployment for a service you wish to add to the mesh with SPIFFE/SPIRE.

Save the above deployment file as deployment.yaml and launch the pod:

kubectl apply -f deployment.yaml

2. Local Routing

The next steps are to create objects in the Fabric API. These objects will create all the configuration for the Sidecar to handle requests on behalf of the deployed service.

This step creates and configures the Grey Matter objects necessary to allow the sidecar container in the deployment to route to the fibonacci container (the service itself). We will refer to this as "local" routing. The next step will configure the Edge proxy to route to the fibonacci sidecar, thus fully wiring the new service into the mesh for routing.

NOTE: This guide goes over deploying a new service and configuring it for ingress routing. To configure a service for both ingress and egress routing within the mesh, see the guide.

For each Grey Matter object created, create the local file and send them to the API with: greymatter create <object> < <file_name.json>.

Domain

The first object to create is a Grey Matter Domain the ingress domain for the fibonacci sidecar. This object does virtual host identification, but for this service we'll accept any hosts ("name": "*") that come in on the port 10080 (the port with name proxy-or value of Grey Matter Control environment variable GM_CONTROL_KUBERNETES_PORT_NAME-in the sidecar container). Note that force_https is set to true, requiring any incoming connection be https.

See the domain object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "domain_key": "fibonacci-domain",
  "name": "*",
  "port": 10808,
  "force_https": true
}

Save this file as domain.json and apply it with

greymatter create domain < domain.json

Listener

The next object is the ingress listener. This is the physical binding of the Sidecar to a host interface and port and is linked in the field domain_keys to a specific domain. The listener and domain configurations determine where the sidecar should listen for incoming connections on and what kind of connections it should accept.

The listener object is also the place to configure Grey Matter filters. See the listener object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "listener_key": "fibonacci-listener",
  "domain_keys": [
    "fibonacci-domain"
  ],
  "name": "ingress",
  "ip": "0.0.0.0",
  "port": 10808,
  "protocol": "http_auto",
  "secret": {
    "secret_key": "fibonacci.identity",
    "secret_name": "spiffe://quickstart.greymatter.io/fibonacci",
    "secret_validation_name": "spiffe://quickstart.greymatter.io",
    "subject_names": [
      "spiffe://quickstart.greymatter.io/edge"
    ],
    "ecdh_curves": [
      "X25519:P-256:P-521:P-384"
    ]
  }
}

Note the secret field. This field is required for service to service communication in a SPIFFE/SPIRE setup. This secret tells the sidecar to fetch its SVID (with ID spiffe://quickstart.greymatter.io/fibonacci) from Envoy and present it to incoming connections. It also will set a certificate validation context with match subject alternative names specifies to only accept incoming requests with SAN spiffe://quickstart.greymatter.io/edge. See the SPIRE documentation for specifics. The listener secret configuration will be important for the Edge to fibonacci cluster.

Save this file as listener.json and apply it with

greymatter create listener < listener.json

Proxy

The proxy object links a sidecar deployment to it's Grey Matter objects. The name field must match the label on the deployment (in this case greymatter.io/control) that Grey Matter Control is looking for in it's environment variable GM_CONTROL_KUBERNETES_CLUSTER_LABEL. It takes a list of domain_keys and listener_keys to link to the deployment with cluster label matching name.

See the proxy object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "proxy_key": "fibonacci-proxy",
  "domain_keys": [
    "fibonacci-domain"
  ],
  "listener_keys": [
    "fibonacci-listener"
  ],
  "name": "fibonacci",
  "listeners": null
}

Save this file as proxy.json and apply it with

greymatter create proxy < proxy.json

Local Cluster

The next object to create is a local cluster. The cluster is in charge of the egress connection from a sidecar to whatever service is located at its configured instances, and can set things like circuit breakers, health checks, and load balancing policies.

This "local" cluster will tell the sidecar where to find the fibonacci container to send requests. From the deployment above, we configured the fibonacci container at port 8080. Since the sidecar and fibonacci containers are running in the same pod, they can communicate over localhost.

See the cluster object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "cluster_key": "fibonacci-cluster",
  "name": "local",
  "instances": [
    {
      "host": "localhost",
      "port": 8080
    }
  ],
  "require_tls": false
}

Save this file as fibonacci-local-cluster.json and apply it with

greymatter create cluster < fibonacci-local-cluster.json

Local Shared Rules

The shared rules object is used to match routes to clusters. They can do some features of routes like setting retry_policies and appending response data, but they also can perform traffic splitting between clusters for operations like blue/green deployments.

This local shared rules object will be used to link the local route in the next step to the local cluster fibonacci-cluster that we just created.

See the shared rules object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "shared_rules_key": "fibonacci-local-rules",
  "name": "local",
  "default": {
    "light": [
      {
        "constraint_key": "",
        "cluster_key": "fibonacci-cluster",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ],
    "dark": null,
    "tap": null
  }
}

Save this file as fibonacci-local-rules.json and apply it with

greymatter create shared_rules < fibonacci-local-rules.json

Local Route

Routes match against requests by things like URI path, headers, cookies, or metadata and map to shared_rules. Since this service only needs to forward everything it receives to the local microservice, the setup is fairly simple.

This "local" route will link the domain (fibonacci-domain) to the local shared rules (fibonacci-local-rules) that we just created. We know that the fibonacci-local-rules object is used to link routes to the local cluster fibonacci-cluster, thus with this route object applied, the fibonacci sidecar will be configured to accept requests and route to the fibonacci service.

See the route object documentation for more information.

{
  "zone_key": "zone-default-zone",
  "domain_key": "fibonacci-domain",
  "route_key": "fibonacci-local-route",
  "path": "/",
  "prefix_rewrite": "",
  "shared_rules_key": "fibonacci-local-rules"
}

The path indicates that any request coming into the sidecar with path / should be routed to the fibonacci service. We will see in the next step when configuring edge routes that all requests from the Edge proxy to the fibonacci service will come in at this path.

Save this file as fibonacci-local-route.json and apply it with

greymatter create route < fibonacci-local-route.json

The sidecar will now be configured to properly accept requests and route to the fibonacci service. The next step will configure the Edge proxy to route to the sidecar.

3. Edge Routing

Now that the Sidecar-to-Service routing has been configured, we will set up the Edge-to-Sidecar routing because we want this service to be available to external users.

The process will take similar steps to what was done before, but we only need to create a cluster, a shared_rules object pointing at that cluster, and two routes.

Edge to Fibonacci Cluster

This cluster will handle traffic from the Edge to the Fibonacci Sidecar. The Edge has existing domain (with domain key edge), listener, and proxy much like the ones we just created for the fibonacci service. The first step to configure the Edge to fibonacci service is to create a cluster to tell it where to find the fibonacci sidecar.

{
  "zone_key": "zone-default-zone",
  "cluster_key": "edge-to-fibonacci-cluster",
  "name": "fibonacci",
  "instances": [],
  "require_tls": true,
  "secret": {
    "secret_key": "edge.identity",
    "secret_name": "spiffe://quickstart.greymatter.io/edge",
    "secret_validation_name": "spiffe://quickstart.greymatter.io",
    "subject_names": [
      "spiffe://quickstart.greymatter.io/fibonacci"
    ],
    "ecdh_curves": [
      "X25519:P-256:P-521:P-384"
    ]
  }
}

NOTE that there are several differences between this cluster and the local cluster created above:

1. The instances field is left as an empty string, whereas the fibonacci-local-cluster instances were configured. This is because Grey Matter Control will discover the fibonacci deployment and the instances array will be automatically populated from this service discovery: the instances will go up and down whenever the service scales or changes. To do this, (in the same way as described in creating the proxy object above) the name field must match the cluster label on the deployment.

2. This cluster has a secret set on it, and require_tls is true. This is because the edge proxy and the fibonacci sidecar are running in different pods so they can't connect over localhost and must use their SPIFFE SVIDs for communication.

   The secret here mirrors the one set on the fibonacci listener. As stated above, the cluster is in charge of the egress connection from a sidecar to whatever service is located at its instances.

   In this case, the secret is telling the Edge proxy to fetch its SVID (with ID spiffe://quickstart.greymatter.io/edge) from Envoy SDS and present it on its outgoing connections. It will also only accept connections that present a certificate with SAN spiffe://quickstart.greymatter.io/fibonacci. See the SPIRE documentation for specifics.

   As described in the secret configuration on the fibonacci listener, these are opposites. The request from this cluster will be accepted by the fibonacci sidecar and vice versa.

Save this file as edge-to-fibonacci-cluster.json and apply it with

greymatter create cluster < edge-to-fibonacci-cluster.json

Edge to Fibonacci Shared Rules

The edge to fibonacci shared_rules will be used to link the edge to fibonacci routes to the edge-to-fibonacci-cluster we just created.

{
  "zone_key": "zone-default-zone",
  "shared_rules_key": "edge-to-fibonacci-rules",
  "name": "edge-to-fibonacci",
  "default": {
    "light": [
      {
        "constraint_key": "",
        "cluster_key": "edge-to-fibonacci-cluster",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ],
    "dark": null,
    "tap": null
  }
}

Save this file as edge-to-fibonacci-rules.json and apply it with

greymatter create shared_rules < edge-to-fibonacci-rules.json

Edge to Fibonacci Routes

There are two route objects that need to be created to send requests through the Edge proxy to the fibonacci service. In the same way that the local route was connected to the fibonacci-domain, these routes will be connected to the edge domain, and will configure how the edge sidecar sends routes.

They are nearly identical, but one provides for the case that the path on the request to send to the fibonacci service ends in a slash, and one for the case that it doesn't.

{
  "zone_key": "zone-default-zone",
  "domain_key": "edge",
  "route_key": "edge-to-fibonacci-route",
  "path": "/services/fibonacci",
  "prefix_rewrite": "/",
  "shared_rules_key": "edge-to-fibonacci-rules"
}

This route looks for the path on a request into the edge proxy /services/fibonacci with no trailing slash, replaces that path with /, and sends the request to the cluster (edge-to-fibonacci-cluster) being pointed to by the edge-to-fibonacci-rules.

{
  "zone_key": "zone-default-zone",
  "domain_key": "edge",
  "route_key": "edge-to-fibonacci-route-slash",
  "path": "/services/fibonacci/",
  "prefix_rewrite": "/",
  "shared_rules_key": "edge-to-fibonacci-rules"
}

This route looks for the path on a request into the edge proxy /services/fibonacci/ this time with a trailing slash, replaces that path with /, and sends the request to the same cluster.

Save these files as edge-to-fibonacci-route.json and edge-to-fibonacci-route-slash.json and apply them with

greymatter create route < edge-to-fibonacci-route.json
greymatter create route < edge-to-fibonacci-route-slash.json

Once these routes are applied, the service is fully configured in the mesh! You should be able to access the service at https://{your-gm-ingress-url}:{your-gm-ingress-port}/services/fibonacci/ with response Alive. To send a request for a specific fibonacci number, https:///{your-gm-ingress-url}:{your-gm-ingress-port}/services/fibonacci/fibonacci/<number>.

If you don't know your gm ingress url and you followed the Quickstart Install Kubernetes guide, run

kubectl get svc edge

and copy the EXTERNAL-IP and port (by default the port will be 10808).

4. Catalog

The last step in deploying a service is to add the expected service entry to the Grey Matter Catalog service. This will interface with the control plane, and provide information to the Intelligence 360 Application for display.

{
  "clusterName": "fibonacci",
  "zoneName": "zone-default-zone",
  "name": "Fibonacci",
  "version": "1.0",
  "owner": "Decipher",
  "capability": "Tutorial",
  "runtime": "GO",
  "documentation": "/services/fibonacci/",
  "prometheusJob": "fibonacci",
  "minInstances": 1,
  "maxInstances": 2,
  "authorized": true,
  "enableInstanceMetrics": true,
  "enableHistoricalMetrics": true,
  "metricsPort": 8081
}

Save this file as fibonacci-catalog.json and using the quickstart certificates from the helm chart repository at ./certs, make the following post request to the catalog service:

curl -k -XPOST --cert <path>/<to>/<certs>/quickstart.crt --key <path>/<to>/<certs>/quickstart.key https:///{your-gm-ingress-url}:{your-gm-ingress-port}/services/catalog/latest/clusters -d "@fibonacci-catalog.json"

You'll see the following response if the addition was successful.

{"added": "fibonacci"}

And the service will display in the Intelligence 360 Application.

Intelligence 360 Application