# HTTP For a list of available HTTP filters, see [HTTP Filters table](#available-filter-summary). ## Configuration Overview HTTP filters may be configured in Grey Matter Control API in one of two configuration object types: Listener and Proxy. When configured in a Listener object, the HTTP filter will be enabled only for that listener and not any other listener registered to the same Grey Matter Proxy. On the other hand, when configured in a Proxy object, the HTTP filter will be enabled for all registered listeners in a Grey Matter Proxy. ### Listener Configuration Preferred Note that the Fabric mesh's control plane gives higher priority to HTTP filter configurations in a Listener object versus its related Proxy object. In fact, HTTP filter configurations added to a Proxy object will only be applied if there are no HTTP filters **and** no Network filter configurations added to a Listener object. If filters of either type are configured in a Listener object, any filter configurations in its related Proxy object will be ignored. In a future version of the Fabric mesh, support for configuring HTTP filters in a Proxy object will likely be deprecated. ## Example Configurations ### Listener To edit the Listener object, run: ```bash greymatter edit listener ``` The following is an example of configuring an HTTP filter in a Listener object by specifying values in two of its fields -- the `active_http_filters` field and the `http_filters` field: ```javascript { "zone_key": "default-zone", "listener_key": "listener1", "name": "listener1", "domain_keys": [ "domain1" ], "ip": "0.0.0.0", "port": 8080, "protocol": "http_auto", "active_http_filters": [ "gm.inheaders", "envoy.ip_tagging" ], "http_filters": { "gm_inheaders": { "debug": true }, "envoy_ip_tagging": { "request_type": 0, "ip_tags": [ { "ip_tag_name": "my-tag", "ip_list": [ { "address_prefix": "0.0.0.0", "prefix_len": { "value": 32 } } ] } ] } } } ``` This configuration tells the corresponding Grey Matter Proxy to register and use the `gm.inheaders` filter for its `listener1` Listener only. This allows for configuring unique filter types on a per-Listener basis within a single instance of Grey Matter Proxy. Again, note that the keys in the `active_http_filters` array use a period following their prefix, while the `http_filters` object uses underscores all the way through. ### Proxy To edit the Listener object, run: ```bash greymatter edit proxy ``` The following is an example of configuring a filter in a Proxy object by specifying values in two of its fields -- the `active_proxy_filters` field and the `proxy_filters` field: ```javascript { "zone_key": "default-zone", "proxy_key": "sidecar-proxy", "name": "sidecar", "domain_keys": [ "domain1", "domain2" ], "listener_keys": [ "listener1", "listener2" ], "active_proxy_filters": [ "gm.inheaders", "envoy.ip_tagging" ], "proxy_filters": { "gm_inheaders": { "debug": true }, "envoy_ip_tagging": { "request_type": 0, "ip_tags": [ { "ip_tag_name": "my-tag", "ip_list": [ { "address_prefix": "0.0.0.0", "prefix_len": { "value": 32 } } ] } ] } } } ``` The configuration tells the corresponding Grey Matter Proxy to enable the `gm.inheaders` and `envoy.ip_tagging` filters for all its registered listeners (i.e. `listener1` and `listener2`, according to the `listener_keys` field). Note that the keys in the `active_proxy_filters` array use a period following their prefix, while the `proxy_filters` object uses underscores all the way through. ## Known Ordering Dependencies NOTES: * [Well Known Dynamic Metadata](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/advanced/well_known_dynamic_metadata): Filters can emit dynamic metadata via the setDynamicMetadata routine in the StreamInfo interface on a Connection. This metadata emitted by a filter can be consumed by other filters and useful features can be built by stacking such filters. For example, a logging filter can consume dynamic metadata from an RBAC filter to log details about runtime shadow rule behavior. Another example is where an RBAC filter permits/restricts MySQL/MongoDB operations by looking at the operational metadata emitted by the MongoDB filter. ## HTTP Filters To learn how to enable any of the Envoy HTTP filters in the Fabric mesh, refer to the [HTTP Filters configuration overview](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters). The following table lists all Grey Matter and Envoy HTTP filters that may be configured. Some general information on each filter is available; otherwise please refer to the Envoy documentation. Also, some Envoy filters have been excluded due to being marked by Envoy as experimental. Note that the **Active Filter Name** is a simplified form of the name used in Grey Matter Proxy (e.g. `envoy.header_to_metadata` is used in place of `envoy.filters.http.header_to_metadata`). ### Available Filter Summary | Filter | Active Filter Name | Guide | Envoy documentation | | ------------------------- | --------------------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Observables | `gm.observables` | [#observables](#observables) | | | Impersonation | `gm.impersonation` | [#impersonation](#impersonation) | | | Inheaders | `gm.inheaders` | [#inheaders](#inheaders) | | | ListAuth | `gm.listauth` | [#listauth](#listauth) | | | Metrics | `gm.metrics` | [#metrics](#metrics) | | | OAuth | `gm.oauth` | [#oauth](#oauth) | | | JWT Security | `gm.jwt-security` | [#jwt-security](#jwt-security) | | | Ensure Variables | `gm.ensure-variables` | [#ensure-variables](#ensure-variables) | | | OIDC Authentication | `gm.oidc-authentication` | [#oidc-authentication](#oidc-authentication) | | | OIDC Validation | `gm.oidc-validation` | [#oidc-validation](#oidc-validation) | | | Buffer | `envoy.buffer` | [#buffer](#buffer) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/buffer_filter#config-http-filters-buffer) | | CSRF | `envoy.csrf` | [#csrf](#csrf) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/csrf_filter#config-http-filters-csrf) | | External Authorization | `envoy.ext_authz` | [#external-authorization](#external-authorization) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/ext_authz_filter#config-http-filters-ext-authz) | | JWT Authentication | `envoy.jwt_authn` | [#jwt-authentication](#jwt-authentication) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/jwt_authn_filter#config-http-filters-jwt-authn) | | Fault Injection | `envoy.fault` | [#fault-injection](#fault-injection) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/fault_filter#config-http-filters-fault-injection) | | gRPC-JSON Transcoder | `envoy.grpc_json_transcoder` | [#grpc-json-transcoder](#grpc-json-transcoder) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/grpc_json_transcoder_filter#config-http-filters-grpc-json-transcoder) | | Gzip | `envoy.gzip` | [#gzip](#gzip) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/gzip_filter#config-http-filters-gzip) | | Header-to-Metadata | `envoy.header_to_metadata` | [#header-to-metadata](#header-to-metadata) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/header_to_metadata_filter#config-http-filters-header-to-metadata) | | Health Check | `envoy.health_check` | [#health-check](#health-check) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/health_check_filter#config-http-filters-health-check) | | IP Tagging | `envoy.ip_tagging` | [#ip-tagging](#ip-tagging) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/ip_tagging_filter#config-http-filters-ip-tagging) | | Lua | `envoy.lua` | [#lua](#lua) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/lua_filter#config-http-filters-lua) | | Rate Limit | `envoy.rate_limit` | [#rate-limit](#rate-limit) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/rate_limit_filter#config-http-filters-rate-limit) | | Role Based Access Control | `envoy.rbac` | [#role-based-access-control](#role-based-access-control) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/rbac_filter#config-http-filters-rbac) | | Squash | `envoy.squash` | [#squash](#squash) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/squash_filter#config-http-filters-squash) | | Tap | `envoy.tap` | [#tap](#tap) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/tap_filter#config-http-filters-tap) | | gRPC HTTP1 Bridge | `envoy.grpc_http1_bridge` | [#grpc-http1-bridge](#grpc-http1-bridge) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/grpc_http1_bridge_filter#config-http-filters-grpc-bridge) | | gRPC HTTP1 Reverse Bridge | `envoy.grpc_http1_reverse_bridge` | [#grpc-http1-reverse-bridge](#grpc-http1-reverse-bridge) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/grpc_http1_reverse_bridge_filter#config-http-filters-grpc-http1-reverse-bridge) | | gRPC Stats | `envoy.grpc_stats` | [#grpc-stats](#grpc-stats) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/grpc_stats_filter#config-http-filters-grpc-stats) | | gRPC Web | `envoy.grpc_web` | [#grpc-web](#grpc-web) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/grpc_web_filter#config-http-filters-grpc-web) | | Dynamo | `envoy.dynamo` | [#dynamo](#dynamo) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/dynamodb_filter#config-http-filters-dynamo) | | Original Src | `envoy.original_src` | [#original-src](#original-src) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v3/extensions/filters/http/original_src/v3/original_src.proto) | | Dynamic Forward Proxy | `envoy.dynamic_forward_proxy` | [#dynamic-forward-proxy](#dynamic-forward-proxy) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/intro/arch_overview/http/http_proxy#arch-overview-http-dynamic-forward-proxy) | | Compressor | `envoy.compressor` | [#compressor](#compressor) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/compressor_filter#config-http-filters-compressor) | | Adaptive Concurrency | `envoy.adaptive_concurrency` | [#adaptive-concurrency](#adaptive-concurrency) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/adaptive_concurrency_filter#config-http-filters-adaptive-concurrency) | | HTTP Cache | `envoy.cache` | [#http-cache](#http-cache) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/api-v2/config/filter/http/cache/v2alpha/cache.proto) | | AWS Lambda | `envoy.aws_lambda` | [#aws-lambda](#aws-lambda) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/aws_lambda_filter#config-http-filters-aws-lambda) | | AWS Request Signing | `envoy.aws_request_signing` | [#aws-request-signing](#aws-request-signing) | [External Link](https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/aws_request_signing_filter#config-http-filters-aws-request-signing) | ### Detailed Filter Description #### Observables [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-observables) The Observables Filter configures the Proxy to emit a JSON payload with every request made to the microservice. This JSON payload contains a variety of different information about the request being made, as well as the user/system issuing the request. These Observables can then be aggregated to perform analysis like: audits, user-experience tracking, etc. Observable publishing defaults to stdout but can also be published to a Kafka topic or location on disk. #### Impersonation [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-impersonation) The ACL Impersonation Filter gives whitelist server distinguished names (DNs) the privilege to impersonate on behalf of users. #### Inheaders [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-inheaders) This filter sets up HTTP request headers on all incoming traffic for purposes of security and audits throughout the mesh. Set headers are: * USER\_DN * EXTERNAL\_SYS\_DN * SSL\_CLIENT\_S\_DN > *NOTE* This filter is primarily intended to be used only on Edge nodes. These nodes are ingress points which require header normalization before propagating the requests throughout the mesh. Turning this filter on at other points may result in extra latency and unintended behavior, and is thus discouraged. #### ListAuth [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-listauth) This filter restricts access to the proxied microservice based on the user's Distinguished Name (DN). The whitelist will allow given DNs, and the blacklist will forbid given DNs. The behavior of each list is slightly different: * whitelist allows all by default, but then allows only the specified DNs if given a non-default string. * blacklist denies none, but then denies only the specified DNs when given a non-default string. > NOTE The users DN comes from the USER\_DN host header. These headers must be supplied by the user, or can set by the gm.inheaders filter somewhere up the connection stream. #### Metrics `gm.metrics` [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-metrics) This filter sets up a local metrics server to gather and report real-time statistics for the sidecar, microservice, and host system. Optionally, this filter can serve the computed statistics in a form suitable for scraping by Prometheus. The prometheus endpoint will be hosted at {METRICS\_HOST}:{METRICS\_PORT}{METRICS\_PROMETHEUS\_URI\_PATH}, which can then be scraped directly through the supported Prometheus service discovery mechanisms. The metrics filter can also push the compiled statistics directly to AWS Cloudwatch. This allows the Grey Matter Proxy metrics to be directly used to trigger things like AutoScale actions or just for tighter monitoring directly in AWS. #### OAuth `gm.oauth` [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-oauth) Grey Matter’s sidecar proxy supports full OAuth 2.0 negotiation. #### JWT Security [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-jwt-security) On incoming requests, the gm-jwt-security filter creates a jwt token from the gm-jwt-security service "/policies" endpoint using the incoming USER\_DN. It passes the token along with the request in a jwt header. #### Ensure Variables [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-ensurevariables) This filter enforces certain attributes of a request like a header, cookie, or query string and optionally moves it to another location. The filter can be configured to reject the request completely if one of these variables is not present. This is meant to act as a normalization filter that makes it easier for downstream filters to find and use variables. #### OIDC Authentication [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-oidc-authentication) Authentication filter has all the information needed to initiate an authentication handshake with an OpenID Connect provider. It begins by checking whether a request contains an access token in a specified location (header, query string, or cookie). If it exists, it assumes that the token was verified at least once by previous filters in the chain and passes the request onto the next filter. If an access token was not found, it will check for the query token to see if there is an access code. This happens when a request is coming back from the identity provider (specified by callback URL). If the code is found, it will exchange it for a bearer token and an id token - both of them will then be stored in specified locations. If no access code was found, the authentication process gets kicked off by forwarding the user to the identity provider. #### OIDC Validation [Full Configuration](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/http/gm-oidc-validation) The oidc-validation filter handles bearer token (aka access token) validation. It can be configured to check for bearer tokens in headers, cookies, or query parameters. If it finds a token, it validates the token with the provided IdP and sets the desired user attributes into a configurable header. If the token is invalid, it strips it from the request so that it can be dealt with upstream. If enforce is set to true, the request will be rejected with a 403 (configurable). #### Buffer Delays iterating through the filter chain to wait for a fully buffered request. #### CSRF Prevents Cross-Site Request Forgery based on a route or virtual host settings. #### External Authorization Integrates with an external gRPC or HTTP service for checking whether an incoming HTTP request is authorized or not. If unauthorized, the request will be denied with a 403 Forbidden response. #### JWT Authentication Verifies JSON Web Tokens and permits forwarding of the payload to additional filters or services. #### Fault Injection Enables the injection of delays and abort requests with custom error codes in order to simulate various failure scenarios (e.g. service failures, overloads, high network latency, network partitions, etc). It can be configured for a given upstream cluster of a request and/or a set of predefined request headers. #### gRPC-JSON Transcoder Transcodes incoming HTTP requests into protobuf and proxies them to a gRPC service. This require creating an HTTP mapping for the gRPC service and providing Grey Matter Proxy with the proto descriptor of the gRPC service. gRPC responses returned to the JSON API client are encoded into JSON. #### Gzip > Note: this filter has been deprecated by the [compressor](#compressor) filter. Enables compressing dispatched data from an upstream service, given specific request headers. #### Header-to-Metadata Instructs Grey Matter Proxy to record and emit dynamic metadata based on request headers that match some configured rule, which can then be used further down in the filter chain for operations like defining load balancing behaviors. #### Health Check Supports active health checking configuration by defining distinct modes of operation (e.g. Whether to pass health check requests to an underlying service, whether to cache health check results for a certain period of time). This filter supports the following configuration options: * `headers`: a list of headers to match incoming requests on to determine if the request is a health check * `pass_through_mode`: a boolean value on weather to let health checks to continue through to the service or terminate them and respond at the sidecar level * `cache_time`: the amount of time in ms to cache the upstream response for health checks (only valid in pass through mode) * `cluster_min_healthy_percentages`: map of upstream cluster names to a value for the minimum percentage of servers in each of those clusters that must be healthy or degraded for the filter to return a `200` response (only valid in pass through mode) Note that this filter will determine which incoming requests are health checks based on the configured `headers`. **If this field is left empty, it will consider all incoming requests as health checks** (ie if you set pass through mode to false, *all* requests will terminate at the sidecar with a `503`). All health check requests configured on an upstream [cluster object](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/cluster/health-check) will have the `user-agent` header value `Envoy/HC`, thus the example below will match on this value to prevent this from happening. Example: ```javascript "envoy_health_check": { "pass_through_mode": true, "headers": [ { "name": "user-agent", "exact_match": "Envoy/HC" } ] } ``` #### IP Tagging Sets the header `x-envoy-ip-tags` with the string tags for the trusted address from `x-forwarded-for`. This provides a scalable way to compare an IP address to a large list of CIDR ranges effectively. #### Lua Injects a Lua script in-line to be run during request and response flows. A Lua script can inspect and modify headers, body, and trailers. Note that blocking operations should not be run from a Lua script here. #### Rate Limit Integrates with a global gRPC rate limiting service such as [Lyft's reference implementation](https://github.com/lyft/ratelimit) to control throughput throughout the Fabric mesh by preventing downstream hosts from overwhelming upstream clusters. Unlike the [Network Rate Limit filter](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/network#rate-limit), rate limit configurations must be specified at the route level. #### Role Based Access Control Allows or denies actions (permissions) by identified downstream clients (principals). It supports configuration using either a safe-list (ALLOW) or block-list (DENY) set of policies based on properties of the connection (IPs, ports, SSL subject) as well as the incoming request's HTTP headers. For a more in-depth guide on configuring RBAC, see [rbac.md](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/reference/api/fabric-api/filters/envoy-rbac). #### Squash Enables Envoy to integrate with Solo.io's [Squash microservices debugger](https://github.com/solo-io/squash). #### Tap > Note: currently experimental only Records all HTTP Traffic. #### gRPC HTTP1 Bridge Enables bridging an HTTP/1.1 client that doesn't support response trailers to a compliant gRPC server. #### gRPC HTTP1 Reverse Bridge Enables converting an incoming gRPC request into an HTTP/1.1 request to allow a server that does not understand HTTP/2 or gRPC to handle the request. #### gRPC Web Enables the bridging of a gRPC-Web client to a compliant gRPC server. #### gRPC Stats Enables telemetry of gRPC calls. It also detects message boundaries in streaming gRPC calls and emits the message counts for both the request and the response. #### Dynamo Enables application communication with [DynamoDB](https://aws.amazon.com/dynamodb/). #### Original Src Binds upstream connections to the original source address determined for the request. #### Compressor Enables Envoy to compress dispatched data from an upstream service upon client request. Currently only supports gzip compression. #### Dynamic Forward Proxy > Note: currently alpha and not production ready. Enables Envoy to perform the role of an HTTP proxy without prior knowledge of all configured DNS addresses. #### Adaptive Concurrency > Note: currently experimental only Dynamically adjusts the allowed number of requests that can be outstanding to all hosts in a given cluster at any time. #### HTTP Cache > Note: currently under active development Stores cacheable responses in an in-memory cache, and services to subsequent requests. > Note: **Configuration cannot be empty**. If a non-nil empty configuration is passed, the sidecar will exit. #### AWS Lambda > Note: currently under active development Triggers an AWS Lambda function from a standard HTTP/1.x or HTTP/2 request. #### AWS Request Signing > Note: currently experimental only Used to access authenticated AWS services. Uses the existing AWS Credential Provider to get the secrets used for generating required headers.