# Multi-Mesh

Multiple meshes can be connected to each other by configuring Grey Matter API objects. This document walks through different methods of configuring multi-mesh, as well as related identity propagation in requests work flows across multiple meshes.

## Table of Contents

* [Configuration](#configuration)
* [Types of Connections to Another Mesh](#types-of-connections-to-another-mesh)
  * [Ingress Edge Setup](#ingress-edge-setup)
  * [Dedicated Egress Edge Setup](#dedicated-egress-edge-setup)
* [Identity Propagation](#identity-propagation)

## Configuration

To get services in one mesh to talk to services in another, a cluster should be created that points to the Host/IP(s) of the ingress edge. The following is an example of a cluster configuration that could be applied in Mesh A which points to the location of Mesh B. It also tells any proxies that try to route to this cluster what certs it should have on disk in order to connect. Note that if the ingress edge you're pointing to is behind some kind of load-balancer, you will also need to [configure an SNI](https://github.com/greymatter-io/gm-control-api/blob/master/docs/mesh/host-identification.md) so that the proxy can direct traffic to the correct upstream host.

```javascript
{
  "cluster_key": "cluster-mesh-b",
  "zone_key": "zone-default-zone",
  "name": "mesh-b",
  "instances": [
    {
      "host": "192.168.99.102",
      "port": 31581
    }
  ],
  "require_tls": true,
  "ssl_config": {
    "cert_key_pairs": [
      {
        "certificate_path": "/etc/proxy/tls/sidecar/server.crt",
        "key_path": "/etc/proxy/tls/sidecar/server.key"
      }
    ],
    "trust_file": "/etc/proxy/tls/sidecar/ca.crt",
    "sni": ""
  }
}
```

We can then tell the mesh how to route to this cluster using a `shared_rules` object. The `light` array contains a list of clusters to which requests will be sent. In this simple case we want _all_ traffic routed to the Mesh B cluster and can link to it using the `cluster_key`:

```javascript
{
  "shared_rules_key": "mesh-b-shared-rules",
  "name": "mesh-b",
  "zone_key": "zone-default-zone",
  "default": {
    "light": [
      {
        "constraint_key": "",
        "cluster_key": "cluster-mesh-b",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ]
  }
}
```

## Types of Connections to Another Mesh

### Ingress Edge Setup

Once the Mesh B cluster has been created, routes can point to it just like any other service within the mesh, as long as those service sidecars have the correct certs on disk. The following is an example of a route configuration for a service called `service-a` that uses the traffic rule shown above (`mesh-b-shared-rules`) to route to Mesh B. If a request with the path `/mesh-b/` is made to `service-a`'s proxy, it will rewrite `/mesh-b/` to a forward slash `/` and send the request along.

```javascript
{
  "route_key": "route-service-a-to-mesh-b",
  "domain_key": "domain-myservice",
  "zone_key": "zone-default-zone",
  "path": "/mesh-b/",
  "prefix_rewrite": "/",
  "shared_rules_key": "mesh-b-shared-rules",
}
```

If you wanted `service-a` in Mesh A to only route to a specific service in Mesh B, let's call it `service-b`, you could use `prefix_rewrite` to point directly to it:

```javascript
{
  "route_key": "route-service-a-to-service-b",
  "domain_key": "domain-service-a",
  "zone_key": "zone-default-zone",
  "path": "/mesh-b/",
  "prefix_rewrite": "/services/service-b/1.0",
  "shared_rules_key": "mesh-b-shared-rules",
}
```

### Dedicated Egress Edge Setup

Another way to achieve a multimesh setup is to stand up a dedicated egress edge that handles cross-mesh traffic. Instead of pointing each service to the ingress edge of the other mesh like in the above example, only the egress proxy knows about the second mesh and all services route to it instead. This is beneficial for security since the egress proxy is the only entity with the necessary credentials to connect to Mesh B and prevents the need for every sidecar to have those certs on disk. It's also a good pattern to choose if you want to monitor cross-mesh traffic.

To achieve this setup you can deploy a standalone proxy like any other service. Then add a route which points to the Mesh B cluster:

```javascript
 {
  "route_key": "route-egress-to-mesh-b",
  "domain_key": "domain-egress",
  "zone_key": "zone-default-zone",
  "path": "/",
  "shared_rules_key": "mesh-b-shared-rules",
}
```

Then update your service routes to point to the egress proxy's shared\_rules:

```javascript
{
  "route_key": "route-service-a-to-mesh-b",
  "domain_key": "domain-service-a",
  "zone_key": "zone-default-zone",
  "path": "/mesh-b/",
  "prefix_rewrite": "/",
  "shared_rules_key": "egress-shared-rules",
}
```

## Identity Propagation

The Grey Matter `inheaders` (Ingress Headers) filter should be enabled on each mesh's ingress edge in order to correctly propagate user and service identity throughout the mesh. This is configurable on the proxy object by adding it to the `active_proxy_filters` array. `gm_inheaders` also has a debug option which is helpful when looking at the proxy logs:

```javascript
{
  "proxy_key": "edge-proxy",
  "zone_key": "zone-default-zone",
  "name": "edge",
  "domain_keys": [
    "edge"
  ],
  "listener_keys": [
    "edge-listener"
  ],
  "active_proxy_filters": [
    "gm.metrics",
    "gm.inheaders"
  ],
  "proxy_filters": {
    "gm_inheaders": {
      "debug": true
    }
    ...
  }
}
```

User and service identity flows through the meshes as follows:

1. The client uses a PKI or oAuth token to hit the Mesh A edge
2. The `inheaders` filter on the Mesh A edge edge proxy grabs the `USER_DN` from the incoming headers as well as the `DN` from the SSL certificate
3. `service-a` proxy propagates the headers as the request flows through
4. When the request exits Mesh A and hits the edge proxy of Mesh B, the `inheaders` filter will check what is already set. `USER_DN` already exists so it will keep passing it along, however, it _will_ rewrite the `EXTERNAL_SYS_DN` and `SSL_CLIENT_S_DN` headers to reflect the `DN` of the last certificates in the chain. In this example, it would be the `DN` of our `server.crt` that we configured for `cluster-mesh-b`.
5. `service-b` finally receives the request. It has the `USER_DN` of the client that first initiated the request and the identity of the service that last touched the request in Mesh A.
