Ensure Variables

This filter enforces certain attributes of a request like a header, cookie, or query string and optionally moves it to another location. The filter can be configured to reject the request completely if one of these variables is not present. This is meant to act as a normalization filter that makes it easier for downstream filters to find and use variables.

The original motivation of this filter came during the implementation of OpenID Connect authentication filters. An access token which comes from Identity Provider is in a query string; while it will be in cookie when coming from a user facing application such as Dashboard. We found ourselves checking all these locations multiple times and thought "wouldn't it be nice if there was a filter that looked for these locations and copied it to one location other filters expect?". That was the original intention. In doing so, we came up with this generic solution that can be configured to suit a variety of needs that may arise.

Filter Configuration Options

Name	Type	Default	Example	Description
rules	array	n/a		A list of rules to compare to the incoming request. Rules are checked in order of appearance in the list.
rules.key	string	n/a	"Authorization"	The key to check in the incoming request
rules.location	enum[header, cookie, queryString, metadata]	header	cookie	The location to check for the specified key/value pair
rules.metadataFilter	string	""	"gm.oidc-authentication"	The name of the filter to read dynamic metadata from. Required if rules.location=metadata
rules.enforce	bool	false	true	Whether to completely reject the request if a rule doesn't match
rules.enforceResponseCode	string	403	500	The status code to send back if a rule does not match and enforce is true.
rules.removeOriginal	bool	false	true	Whether to remove the original matching key/value pair from the request.
rules.value.matchType	enum[exact, prefix, suffix, regex]	exact	regex	The type of comparison between the key/value pair and the incoming request
rules.value.matchString	string	n/a	Bearer\s+(\S+).*	The string to match against the incoming request
rules.value.copyTo	array	n/a		A list of locations where the matched variable may be copied (optional)
rules.value.copyTo.location	enum[header, cookie, queryString, metadata]	header	cookie	Location to copy the matched variable to
rules.value.copyTo.key	string	n/a	"access_token"	Name of the key where the value will be stored
rules.value.copyTo.direction	enum[default, request, response, both]	default	response	Whether to copy the variable to the request, response, or both. Default will be request or response depending on the location. See (#hiromis-section for more information.)
rules.value.copyTo.cookieOptions.httpOnly	bool	false	true	Whether the cookie being created should be httpOnly.
rules.value.copyTo.cookieOptions.secure	bool	false	true	Whether the cookie being created should only be sent to the server over HTTPS.
rules.value.copyTo.cookieOptions.maxAge	string	session	12h	The Max-Age of the new cookie. The value is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "s", "m", "h" ("ns", "us"/"µs", and "ms" are also valid but the value gets rounded to the nearest second as Max-Age is defined as "number of seconds until the cookie expires"). If maxAge is not set, the cookie will have Expires/Max-Age of Session.
rules.value.copyTo.cookieOptions.path	string	""	/ping	Path indicates a URL path that must exist in the requested URL in order to send the Cookie header. The %x2F ("/") character is considered a directory separator, and subdirectories will match as well.
rules.value.copyTo.cookieOptions.domain	string	""	localhost	Domain specifies allowed hosts to receive the cookie. If unspecified, it defaults to the host of the current document location, excluding subdomains. If Domain is specified, then subdomains are always included.

Design Decisions

Value match

This field came about when we were dealing with Authorization Bearer token. The key is Authorization and the value looks something like Bearer eyJraWQiOiJtOEpPX0l4SndNMG.... The first thing we wanted to make sure was that the value starts with Bearer and not something like Basic (for basic authorization). We also wanted to extract the actual token value before copying it to another location. So we allow the regular expression to optionally have a capturing group.

The other variation of matches come from Envoy's string matcher. When there is no value match specified by the configuration, this filter will assert that the value consists of one or more non-whitespace character using a regular expression \S+.

CopyTo field

copyTo has a field called direction. This can be thought of as "who should see this copied variable". If the answer is "the service which is receiving this request", then the direction should be request. If it is "the user or application that initiated this request", then the direction is response. You can also set it to both. Because some combinations of location and direction are not feasible, we will make some assertions.

Copying to header

This is the most straight forward of the three. It works for both directions (response and request). The one thing to note is that unlike a cookie or a query string, HTTP header key is case-insensitive (HTTP/1.1 RFC 7230, HTTP/2 RFC 7540). So by copying to the header, you will lose the case information of the key if the original location is cookie or requestString.

If no direction is set, it will default to request.

Copying to cookie

Cookies are stored in browsers, so copying to cookie will always add a Set-Cookie header to the response (even if direction is set to request). In cases such as a Bearer token or a user information token is passed around in a cookie, the service who is receiving the request may want to see what would be stored in cookie once this request is complete. You can allow this by setting the direction to both (request will also work because we will always add a cookie to response). This will only allow the service to see what will be in Set-Cookie header in the response. So if a service is expecting the Bearer token in the cookie, it will need to check both Cookie and Set-Cookie headers.

If no direction is set, it will default to response.

Copying to query string

Query strings can only be read by the service receiving this request. Hence setting the direction to response will results in a warning getting logged and have no effect.

If no direction is set, it will default to request.

Sample Configurations

The following sample configurations can be easily tested in docs/examples/ensure-variable/config.yaml. Update the configuration starting on line 38 and use the sample queries to experiment.

Enforcing a header

Assert that requests must have an Authorization: Bearer header, otherwise return a 404.

- name: gm.ensure-variables
  config:
    rules:
    - key: Authorization
      location: header
      enforce: true
      enforceResponseCode: 404
      value:
        matchType: regex
        matchString: Bearer\s+(\S+).*

Passing requests:

curl -v -H "Authorization: Bearer abc123" "localhost:8080/ping"
curl -v -H "authorization: Bearer abc123" "localhost:8080/ping"

Rejected requests:

curl -v -H "Authorization: bearer abc123" "localhost:8080/ping"
curl -v -H "authorization: Bearer" "localhost:8080/ping"

Checking a header and copying it to a cookie

Check that an Authorization header exists on the request. If it does, copy the value to an httpOnly cookie and set on the request and response (defaults to response). If not, let the request pass through (notice enforce: false.)

- name: gm.ensure-variables
  config:
    rules:
    - key: Authorization
      location: header
      enforce: false
      value:
        matchType: regex
        matchString: Bearer\s+(\S+).*
      copyTo:
      - location: cookie
        key: access_key
        direction: both
        cookieOptions:
          httpOnly: true

Requests that will set set-cookie header:

curl -v -H "Authorization: Bearer abc123" "localhost:8080/ping" # set-cookie: access_key=abc123; HttpOnly
curl -v -H "authorization: Bearer helloworld" "localhost:8080/ping" # set-cookie: access_key=helloworld; HttpOnly

Requests that will pass through without setting anything:

curl -v -H "Authorization: Bearer" "localhost:8080/ping"
curl -v -H "Authorization: 123" "localhost:8080/ping"

Enforcing a query string

Rejects all requests that don't have a query string with a key of username and a value prefixed with jane. Sends back a default 403 status code.

- name: gm.ensure-variables
  config:
    rules:
    - key: username
      location: queryString
      enforce: true
      value:
        matchType: prefix
        matchString: jane

Passing requests:

curl -v "localhost:8080/ping?username=jane.doe"
curl -v "localhost:8080/ping?username=janegoodall"

Rejected requests:

curl -v "localhost:8080/ping?name=jane.doe"
curl -v "localhost:8080/ping?username=jann"

Enforcing a cookie and then removing it

Checks for a cookie with a user_dn value that exactly matches matchString and then removes the cookie from the browser. You'll notice the following header on a successful response: set-cookie: user_dn=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0. This will remove the cookie from the user's browser by expiring it and removing the value.

- name: gm.ensure-variables
  config:
    rules:
    - key: user_dn
      location: cookie
      enforce: true
      removeOriginal: true
      value:
        matchType: exact
        matchString: C=US,ST=Virginia,L=Alexandria,O=Decipher Technology Studios,OU=Engineering,CN=*.greymatter.svc.cluster.local

Passing request:

curl -v -b "user_dn=C=US,ST=Virginia,L=Alexandria,O=Decipher Technology Studios,OU=Engineering,CN=*.greymatter.svc.cluster.local" "localhost:8080/ping"
# set-cookie: user_dn=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Max-Age=0

Rejected requests:

curl -v -b "dn=C=US,ST=Virginia,L=Alexandria,O=Decipher Technology Studios,OU=Engineering,CN=*.greymatter.svc.cluster.local" "localhost:8080/ping"

Setting multiple copyTo locations

Checks for the existence of an id_token query string. If it exists, it is copied to a response cookie with a key of userinfoCookie. It also copies this value to a header on the request and response with a key of x-userinfo.

- name: gm.ensure-variables
  config:
    rules:
    - key: id_token
      location: queryString
      enforce: true
      enforceStatusCode: 404
      copyTo:
      - location: cookie
        key: userinfoCookie
      - location: header
        key: x-userinfo
        direction: both

Passing requests:

curl -v "http://localhost:8080/ping?id_token=abc123"
# set-cookie: userinfoCookie=abc123
# x-userinfo: abc123
curl -v "http://localhost:8080/ping?id_token=somevalue"
# set-cookie: userinfoCookie=somevalue
# x-userinfo: somevalue

Rejected requests:

curl -v "http://localhost:8080/ping?id_token= "
curl -v "http://localhost:8080/ping"

Setting multiple rules

The following configuration checks:

1. An Authorization: Bearer <somevalue> header exists. If it does, it copies <somevalue> to a cookie with a key of access_key.

2. An id_token queryString exists with any value (notice how there is no value block set). If the key exists, it copies the value to a cookie with a key of userinfo.

- name: gm.ensure-variables
  config:
    rules:
    - key: Authorization
      location: header
      enforce: true
      value:
        matchType: regex
        matchString: Bearer\s+(\S+).*
      copyTo:
      - location: cookie
        key: access_key
    - key: id_token
      location: queryString
      enforce: true
      copyTo:
      - location: cookie
        key: userinfo
        cookieOptions:
          httpOnly: true
          path: "/ping"
          domain: "localhost"
          maxAge: 24h

Passing requests:

curl -v -H "Authorization: Bearer abc123" "http://localhost:8080/ping?id_token=abc123"
# set-cookie: access_key=abc123
# set-cookie: userinfo=abc123; Path=/ping; Domain=localhost; Max-Age=86400; HttpOnly
curl -v -H "Authorization: Bearer anotherkey" "http://localhost:8080/ping?id_token=anothervalue"
# set-cookie: access_key=anotherkey
# set-cookie: userinfo=anothervalue; Path=/ping; Domain=localhost; Max-Age=86400; HttpOnly

Rejected requests:

curl -v -H "Authorization: Bearer abc123" "http://localhost:8080/ping?id_token="
curl -v -H "Authorization: Bearer" "http://localhost:8080/ping?id_token=abc"