Traffic Splitting

In Grey Matter, traffic is routed to instances of a cluster using the "Least Request" scheme. This is helpful in balancing request load to multiple instances of identical services. When traffic needs to flow to non-identical servers, Traffic Splitting must be used. Traffic splitting routes traffic to two or more clusters with user-defined configuration rules. This is controlled within theshared_rules object, which manages network traffic to clusters.

Table of Contents

• Configuration Reference

• Weights

• Cluster Constraints

  • Currently Supported

  • Future Work

Configuration Reference

Attribute	Description	Type	Default
shared_rules.default	Defines the types of traffic the shared rules will serve to the specified upstream clusters.	JSON map of constraintType : constraint	{}

Weights

Traffic splitting controls the percentage of traffic that goes to each cluster through the weight attribute. When weights are configured, traffic is calculated for each cluster by taking weight / (sum all weights). For example, assigning weights to 1,2,3 and clusters a,b,c respectively, would route traffic to cluster a 17% of the time (1/6), cluster b 33% of the time (2/6) and cluster c 50% of the time (3/6). Assigning the weights 2,4,6 yields the same result.

A common use case is to ramp up or ramp down a new release of a service. Consider a stable release of a service example-service-v1.0.0. If a newly optimized version was released example-service-v1.1.0 it would be useful to slowly ramp up the use of version 1.1.0. Within the shared_rules object.

In the example-service's shared_rules we would update the default.light array to support multiple clusters:

{
  ...,
  "shared_rules_key": "shared-rules-example-service",
  "name": "example-service",
  "default": {
    "light": [
      {
        "constraint_key": "",
        "cluster_key": "cluster-example-service-version-1.0.0",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 9
      },
      {
        "constraint_key": "",
        "cluster_key": "cluster-example-service-version-1.1.0",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ]
  }
}

With the weights updated to 9 (v1.0.0) and 1 (v1.1.0), v1.0.0 will receive 90% of traffic where version 1.1.0 will receive 10%. Over time, if version 1.1.0 performs well, we can increment the amount of traffic it receives until it fields 100% of requests and we can remove v1.0.0. Conversely, if version 1.1.0 performs poorly, traffic can be safely diverted from the failing cluster until the issue is resolved.

Cluster Constraints

Within the shared_rules object, cluster constraints determine the type of traffic which is routed to the specified clusters. There are three main types:

Currently Supported

light is the only constraint type currently supported:

• light - Live traffic. The request is passed through to the upstream cluster normally and the response is processed back to the client. At least one cluster must be specified in this array. This is the most common constraint type.

Future Work

The following cluster constraints are supported in envoy but not within Grey Matter:

• dark - Send and forget. Instead of forwarding the request, a copy of the request is made to the upstream cluster and the response is not returned back to the client. This is useful to get an idea of how a stable service is in a deployment environment. For example, if were introduce a breaking change in the example-service API (v2.0.0), it's possible there would be issues with other services. To see what issues might arise we could specify a dark constraint:

{
  ...,
  "shared_rules_key": "shared-rules-example-service",
  "name": "example-service",
  "default": {
    "light": [
      {
        "constraint_key": "",
        "cluster_key": "cluster-example-service-version-1.0.0",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ],
    "dark": [
      {
        "constraint_key": "",
        "cluster_key": "cluster-example-service-version-2.0.0",
        "metadata": null,
        "properties": null,
        "response_data": {},
        "weight": 1
      }
    ]
  }
}

Here, version 1.0.0 will field all live requests whereas version 2.0.0 will only receive throw away requests which will not impact the stability of the deployment.dark is not currently implemented in Grey Matter. In envoy this is called "request mirroring" or "shadowing" docs.

• tap - Recorded traffic. Instead of sending the request out, a copy of the request is saved to one or more output sinks (e.g. local file). Users can configure what types of requests activate this filter using match configuration. For more information see envoy docs's on tap filters. tap is not currently implemented in Grey Matter and as of envoy version 1.12.0, it is only in a experimental phase and should not be used in production.