AWS Discovery

AWS EC2

Description

Grey Matter Control discovers service instances from AWS EC2 instances using cluster tags and instance filters. This collector requires AWS access credentials to utilize the AWS APIs.

Usage

To use aws ec2 discovery, set the following environment variables:

GM_CONTROL_CMD=aws
GM_CONTROL_AWS_AWS_ACCESS_KEY_ID={}
GM_CONTROL_AWS_AWS_SECRET_ACCESS_KEY={}
GM_CONTROL_AWS_AWS_REGION={}
GM_CONTROL_AWS_VPC_ID={}
GM_CONTROL_AWS_CLUSTER_TAG_NAMESPACE={}
GM_CONTROL_AWS_FILTERS={}

Environment Variable	Description	Type	Default	Required
GM_CONTROL_AWS_AWS_ACCESS_KEY_ID	AWS Access Key to access EC2 Instances	string		true
GM_CONTROL_AWS_AWS_SECRET_ACCESS_KEY	Corresponding AWS Secret Access Key	string		true
GM_CONTROL_AWS_AWS_REGION	The AWS region in which instances are running	string		true
GM_CONTROL_AWS_VPC_ID	The ID of the VPC in which instances are running	string		true
GM_CONTROL_AWS_CLUSTER_TAG_NAMESPACE	The namespace for cluster tags	string	gm:cluster	false
GM_CONTROL_AWS_FILTERS	A comma-delimited list of key/value pairs, used to specify additional EC2 Instances filters	string		false

For an ec2 instance to be discovered by Grey Matter Control, it must have a tag of a specific form, see tags for information.

Tags

For an ec2 instance to be discovered by Grey Matter Control, it will need to have a tag in the form {Key=<cluster-tag-namespace>:<cluster-name>:<port>,Value=}.

If you wish to add additional key,value pairs to the metadata for the instance, you can either modify the tag to the form {Key=<cluster-tag-namespace>:<cluster-name>:<port>:<keyValue>,Value=<valueValue>} or add tags to the instance. Any additional tags included on an ec2 that has a discovery tag in one of the forms above will also be added to the metadata of all instances in all clusters of that ec2 - these do not need to include the <cluster-tag-namespace>.

For example, if using the default value of GM_CONTROL_AWS_CLUSTER_TAG_NAMESPACE, an example ec2 instance to be discovered with cluster name test-cluster running on port 8080 should have a tag with key: "gm:cluster:test-cluster:8080" and empty value. This will create an instance in the api cluster with name test-cluster that looks like:

"instances": [
  {
    "host": "{ec2-instance-host}",
    "port": 8080,
    "metadata": [
      {
        "key": "gm_control_discovery",
        "value": "true"
      }
    ]
  }
]

Where the "host" will be the discovered ec2 instance IP.

If the tag adds a key,value pair, and looks like: {Key=gm:cluster:test-cluster:8080:testkey,Value=testvalue} or if you add a new tag {Key=testkey,Value=testvalue} to the instance with the above existing tag, the instances will look like:

"instances": [
  {
    "host": "{ec2-instance-host}",
    "port": 8080,
    "metadata": [
      {
        "key": "gm_control_discovery",
        "value": "true"
      },
      {
        "key": "testkey",
        "value": "testvalue"
      }
    ]
  }
],

Note: If multiple services are exposed on a single EC2 instance, multiple tags can be used. Ex. Tags=[{Key=gm:cluster:service1:8080,Value=},{Key=gm:cluster:service2:9090,Value=}]

Optional Filters

By default, all EC2 Instances in the VPC are examined, but additional filters for limiting the results can be specified using GM_CONTROL_AWS_FILTERS. See the --filters flag in the aws cli describe-instances docs for a list of available filters. They should be specified in they form GM_CONTROL_AWS_FILTERS=<filter>=<value>.

For example, if GM_CONTROL_AWS_FILTERS=instance-type=t2.micro, Grey Matter Control will only discover instances with instance type t2.micro.

Command Line

To use the command line, run gm-control with gm-control aws <global-flags> --aws.access-key-id={} --aws.secret-access-key={} --aws.region={} --vpc-id={}.

Help

For help or to list available options for aws ec2 discovery using the cli, run gm-control aws --help.