Cache Mesh Configurations With a Hosted Redis Provider

Prerequisites

• eksctl installed

• aws installed with access to the same AWS profile configured with eksctl

• jq installed

Steps

1. Launch an AWS EKS cluster

If you don't already have a running EKS cluster on AWS, run the following to start one:

eksctl create cluster \
    --name production \
    --version 1.17 \
    --nodegroup-name workers \
    --node-type m4.2xlarge \
    --nodes=2 \
    --node-ami auto \
    --region us-east-1 \
    --zones us-east-1a,us-east-1b \
    --profile default

2. Verify necessary Security Group and Subnets are created

While the EKS cluster is being created, open up another terminal.

Run the following until you are able to find a matching security group:

aws ec2 describe-security-groups | jq '.SecurityGroups | .[] | select(.Description | contains("Communication between all"))'

Next, run the command below until you are able to find several EC2 subnets created:

aws ec2 describe-subnets | jq '.Subnets | .[] | select(.Tags | .[] | contains({"Key":"alpha.eksctl.io/cluster-name"}))'

3. Create an AWS ElastiCache subnet group

In order to network your AWS ElastiCache cluster with the EKS cluster in its Virtual Private Cloud environment, you'll need to create a cache subnet group that links to each of your EKS cluster's subnets. This will launch the ElastiCache cluster in the same VPC.

Run the following to create the subnet group:

aws elasticache create-cache-subnet-group \
    --cache-subnet-group-name gm-config-subnet-group \
    --cache-subnet-group-description "Grey Matter config subnet group" \
    --subnet-ids $(aws ec2 describe-subnets | jq '.Subnets | .[] | select(.Tags | .[] | contains({"Key":"alpha.eksctl.io/cluster-name"})) | .SubnetId' | tr "\n" " " | tr '\"' ' ' | tr -s " "  | sed 's/.$//')

4. Create an AWS ElastiCache cluster

Next, run the following to create an AWS ElastiCache cluster backed with Redis 6.x, running with a single node on a small t2.micro server (you can upgrade later on in a live deployment).

aws elasticache create-cache-cluster \
  --cache-cluster-id gm-config-store \
  --num-cache-nodes 1 \
  --cache-node-type cache.t2.micro \
  --engine redis \
  --engine-version 6.x \
  --port 6379 \
  --cache-subnet-group-name gm-config-subnet-group \
  --security-group-ids $(aws ec2 describe-security-groups | jq '.SecurityGroups | .[] | select(.Description | contains("Communication between all")) .GroupId' | tr '\"' ' ')

Note the command flags specify the gm-config-subnet-group you created in the previous step as well as the same Security Group you found in step 2. This Security Group is the same one used to authorize communication between all nodes in the same VPC environment.

5. Retrieve the AWS ElastiCache cache node endpoint

You'll need to wait a while for the cluster to be created before you can access its endpoint. Run the following until the output is no longer null:

aws elasticache describe-cache-clusters \
    --cache-cluster-id gm-config-store \
    --show-cache-node-info | jq '.CacheClusters[0] | .CacheNodes[0]'

When the output prints out cache node information, copy the Endpoint.Address value (it should end with cache.amazonaws.com).

5. Configure Grey Matter to point to AWS ElastiCache

Next, if you haven't already, clone the Grey Matter Helm Charts repo.

Open the global.yaml file in the root directory and find the external_redis section. Update the disabled value to false and set host to be the endpoint address you copied in the previous step.

  external_redis:
    disabled: false
    host: '(INSERT-ENDPOINT-HOST-HERE)'

6. Launch the mesh

Proceed with the Grey Matter installation instructions from step 3 and on.

When Grey Matter is fully installed, mesh configurations will be persisted in your AWS ElastiCache cluster.

To confirm that your AWS ElastiCache cluster is being used, [log in to the AWS Management Console and view metrics in Cloudwatch](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#metricsV2:graph=%28);query='*7bAWS*2fElastiCache*2cCacheClusterId*2cCacheNodeId*7d*20gm-config-store>)

Cleanup

Follow these cleanup steps to uninstall Grey Matter.

Afterwards, run the following commands to delete the AWS ElastiCache cluster:

aws elasticache delete-cache-cluster --cache-cluster-id gm-config-store

You'll need to wait a while for AWS ElastiCache to clean up all of the cluster's underlying resources. Then run the following to delete the cache subnet group:

aws elasticache delete-cache-subnet-group --cache-subnet-group-name gm-config-subnet-group

References

• AWS CLI Command Reference: ElastiCache

• Amazon VPC's and ElastiCache Security

• Subnets and Subnet Groups