# Set Mesh-wide Configuration

These examples show how you can apply mesh-wide policy changes using the [greymatter](https://nexus.greymatter.io/#browse/browse:raw) cli and [gm-control-api](https://github.com/greymatter-io/gm-control-api).

## Prerequisites

1. [jq](https://stedolan.github.io/jq/) command-line JSON processor
2. [greymatter](https://github.com/greymatter-io/cli/blob/master/README.md#quickstart) setup with a running Fabric mesh.

## Mesh-Wide Configuration Changes

To apply a mesh-wide configuration change, we need to loop through each Grey Matter Object you wish to change and reapply a new configuration. This can be done by using the `greymatter list $object` and `greymatter edit $object_key` in tandem.

### Example: Setting Circuit Breakers

This example will set circuit breaker `max_connections` to `500` for all clusters in the service mesh. Create a file named `update.json`

```javascript
{
  "circuit_breakers": {
    "max_connections": 500
  }
}
```

This update will be merged in with the current values of each cluster object when run in the following snippet. It's best to run these commands a few times individually first, to make sure the `update.json` creates the expected change. In the below example, my cluster is `cluster-slo-service`.

```
greymatter get cluster cluster-slo-service > cluster-slo-service.json
jq -s '.[0] * .[1]' cluster-slo-service.json update.json > merged.json
# this is what will be appplied
cat merged.json
...
# try applying the change
greymatter edit cluster cluster-slo-service < merged.json
```

If you're satisfied, apply the change to the entire mesh:

```bash
#!/bin/sh

for key in $(greymatter list cluster | jq -r '.[] | .cluster_key'); do
  greymatter get cluster $key > $key.json
  jq -s '.[0] * .[1]' $key.json update.json > merged.json
  greymatter edit cluster $key < merged.json
  rm $key.json merged.json
done
```

#### Example: Enable Grey Matter Observables on All Proxies

Similar to the above example, create a file `update.json` with the proposed update to all `proxy` objects:

```javascript
{
  "active_proxy_filters": [
    "gm.metrics",
    "gm.observables"
  ],
  "proxy_filters": {
    "gm_observables": {
      "emitFullResponse": true,
      "useKafka": true,
      "eventTopic": "observables",
      "enforceAudit": false,
      "kafkaZKDiscover": false,
      "topic": "__REPLACE_WITH_TOPIC_NAME__",
      "kafkaServerConnection": "kafka-default.fabric.svc:9092"
    }
  }
}
```

Note that we must also supply any other currently active proxy filters (in this case just `"gm.metrics"`, since the entirety of `"active_proxy_filters"` will be overridden. Also note that we have an attribute `__REPLACE_WITH_TOPIC_NAME__`. The kafka topic is specific to each proxy, so we must replace this with the proxy name for each proxy we change.

Apply these changes with a similar script as above.

```bash
#!/bin/sh

for key in $(greymatter list proxy | jq -r '.[] | .proxy_key'); do
  greymatter get proxy $key > $key.json
  # fill in topic name with the name of the proxy
  name=$(cat $key.json | jq -r '.name')
  sed 's/__REPLACE_WITH_TOPIC_NAME__/'"$name"'/g' update.json > update-$key.json
  jq -s '.[0] * .[1]' $key.json update-$key.json > merged.json
  greymatter edit proxy $key < merged.json
  rm $key.json merged.json update-$key.json
done
```

#### Example: Configuring Objects With Specific Attributes

It's often useful to be able to update objects in the mesh only with specific attributes. We can do this by modifying our `shell` snippet above to use [jq conditionals](https://stedolan.github.io/jq/manual/#ConditionalsandComparisons). *Note: all the shell snippets in this doc can be changed to only update on specific attributes by adding the below `if` block*.

```
#!/bin/sh

for key in $(greymatter list proxy | jq -r '.[] | .proxy_key'); do
  greymatter get proxy $key > $key.json
  matches=$(cat $key.json | jq '.proxy_filters.gm_observables.eventTopic == "fabric"')
  if [ $matches = "true" ]; then
 jq -s '.[0] * .[1]' $key.json update.json > merged.json
 greymatter edit proxy $key < merged.json
  fi
  rm -rf $key.json merged.json
done
```

In the above example, we update only the proxies that have `proxy_filters.gm_observables.eventTopic` equal to `"fabric"`.

## Health Checks And Outlier Detection

Health checks and outlier detection help determine if an endpoint is healthy and our services are configured correctly. A common problem this can fix is incorrectly configured or unresponsive hosts. See [envoy docs](https://github.com/DecipherNow/gm-gitbook-sync/tree/691889198cb8c3ebe291a31161de07931571c790/guides/envoyproxy.io/learn/health-checks/README.md) for available configurations and [source code](https://github.com/greymatter-io/gm-control-api/blob/release-1.1/api/health_check.go) for api definition.

Health checks can be added the same way we updated the clusters above, as all we need to update is the configuration options for `health_checks` on each `cluster`. Change `update.json` to enable a basic health check.

```javascript
{
  "health_checks": [
    {
      "timeout_msec": 1000,
      "interval_msec": 60000,
      "interval_jitter_msec": 1000,
      "unhealthy_threshold": 3,
      "healthy_threshold": 3
    }
  ]
}
```

Change `update.json` for basic outlier detection.

```javascript
{
  "outlier_detection": {
    "consecutive_5xx": 3,
    "base_ejection_time_msec": 30000
  }
}
```

Apply this update to each `cluster` by running the snippet

```
#!/bin/sh

for key in $(greymatter list cluster | jq -r '.[] | .cluster_key'); do
  greymatter get cluster $key > $key.json
  jq -s '.[0] * .[1]' $key.json update.json > $key-merged.json
  greymatter edit cluster $key < $key-merged.json
  rm $key.json $key-merged.json
done
```

You should now see that health checks are running. Follow the logs from any given sidecar to see that health checks have been correctly enabled.

## Detect Unneeded Objects

Often there are objects "floating around" in the mesh from previous deployments or experiments. These objects can be partially detected by looping through each one and ensuring that each one was created with a name, zone\_key, or other basic properties.

```
#!/bin/sh

for key in $(greymatter list cluster | jq -r '.[] | .cluster_key'); do
  greymatter get cluster $key > $key.json
  GREYMATTER_CONSOLE_LEVEL="none"
  possibleOutlier=$(cat $key.json | jq '.name == "" or .zone_key == "" or .instances == []')
  if [ $possibleOutlier = "true" ]; then
 echo "------ POSSIBLY UNNEEDED ------ $key "
  else
   echo "--------------OK -------------- $key "
  fi
  rm -rf $key.json merged.json
done
```

The above snippet will trigger if a `zone_key`, `cluster_key`, or `instances` attributes in any cluster is not assigned.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://greymatter.gitbook.io/grey-matter-documentation/guides/setting-meshwide-config.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.