Grey Matter Fabric

Fabric is the master control plane for Grey Matter, and helps microservices focus on specific tasks.

Explore how Fabric works within Grey Matter's architecture, or configure Fabric now.

Oversight and Visibility

Fabric's user-operated dashboard helps visualize and manage the service mesh. The dashboard provides the following features:

• Status, owner, capability, and business value sortability

• Fine-grained search targeting every microservice on the mesh

• Aggregate service reporting for all service instances spawned atop the mesh

Learn more about Grey Matter's service discovery features.

Network Operations Management and Business Intelligence

Grey Matter offers the following network operations and business intelligence activities:

• User-defined service and route level management and objective capture for:

  • Aggregated Service Level Objectives (SLOs)

    • memory, CPU, latency, error and request rates

Read more about Grey Matter's business insight capabilities here.

Security and Access Control

Each microservice requires its own data access point and generates its own data, creating a lot of complexity in the network. Grey Matter governs data access management with fine-grained access policy creation and management.

Fabric's sidecar proxies run alongside each microservice to manage network requirements such as scaling, access control, and intercommunication. Its security and access control features include:

• Service-to-service and end-user authentication

  • Mutual TLS and Fine-Grained Access Policies

  • FIPS 140-2 (BoringSSL can be built in a )

Learn more about Grey Matter's security policies here.

Compatibility and Interoperability

Grey Matter is compatible with the following technologies:

• Istio Compatible Sidecar

  • Beta features (Istio Control Plane, Citadel, Pilot, Mixer, etc.)

• Consul Compatible Sidecar

Grey Matter Data

Data and Content Transfer and Distribution

Grey Matter Data is a Data Distribution Network (DDN) that provides data and content capture, store, sync, cache, move and share of any kind, to and from consumers and services anywhere.

Data's Features

Data's features include:

• Immutable, timestamped fact database

• JWT-based authentication and authorization

• High-performance, user-defined security policy

Grey Matter Sense

Finally, the direct and atmospheric telemetry generated by this constant flow of data is fed to Sense, our experimental machine learning-enabled AI layer. We designed Sense to automate and optimize enterprise network operations, cutting resource expenditures at machine speed.

Sense's Features

Sense's features include:

• Catalog

• Service Level Objectives

• Intelligence 360

All Together

Check out our architecture section for visual representations of Grey Matter's features.

Questions?

Grey Matter is a software platform that provides zero-trust policy compliance, operational insight, and infrastructure management. More concretely, it's a cloud-native infrastructure layer that improves microservice performance and simplifies service mesh operations.

Core Components

Grey Matter is composed of . Each simplifies the technical challenges associated with microservice management, such as service announcement, service discovery, instrumentation, logging, tracing, troubleshooting, encryption, and access control.

Technical Overviews on Grey Matter

Grey Matter's core features include:

• Zero-trust security

• Cloud, language, and vendor-agnostic implementations

• Automated service-level observability and management

You can use Grey Matter to improve service discovery, build or manage your service mesh, gain business insights, and add zero-trust security to your environment. You can also use Grey Matter to extend your unique use cases.

Read more about our use cases to learn what Grey Matter does.

Grey Matter is composed of Fabric, Data, and Sense. Learn how these components simplify microservice management in Architecture.

Why Grey Matter?

Grey Matter uses mesh-generated telemetry data to improve network insight and systems performance from local development, to production-scale container and hybrid cloud implementations.

Flexibility and Compatibility‌

We take our design cues from open architecture. Our implementations don't require predetermined downstream investments for existing infrastructure. Grey Matter is cloud, language, and platform agnostic.

Precision, Depth, and Security

Fabric, the platform’s core Envoy proxy-based mesh technology, generates and captures telemetry and audit data. This data powers dynamic enterprise policy compliance, and SLO creation and monitoring.

‌Grey Matter feeds each piece of telemetry and audit data generated throughout the lifetime of a service instance to Grey Matter’s Sense SLO monitoring overlay service. Because Grey Matter maintains the entire history of every event tied to the lifespan of every service on the mesh, the value of SLO monitoring for enterprise IT only grows over time.

Backed by the full lineage of service telemetry data, your team can:

• Establish historical patterns of use

• Refine SLOs to best reflect real-world performance

Focus on Zero-Trust

Grey Matter is designed with in mind. Our platform manages the activities, policies, security, auditing and data of every microservice in a multi-tenant mesh across any enterprise environment. These zero-trust enforcement features include:

• Enabling service-to-service mTLS connections

• Scheduled or on-demand key rotation

• Service cryptographic identifiers

Audit logs compliant with ICS 500-27 standards are automatically provided to your security team. Grey Matter maintains a historical pedigree of every action occurring on the system throughout its lifespan. This data can be provided to auditors and/or security analysts to assist with risk management, data loss, or breach impact mitigation.

Learn more about zero-trust , or learn how to set up zero-trust for your mesh.

Elegance and Accessibility‌‌

Intelligence 360 is Grey Matter’s single pane of glass for mesh service health monitoring, access, and control. Intelligence 360 enables service level objective (SLO) observability and dynamic control for every service within Grey Matter Fabric. Intelligence 360 visualizes aggregates, routes, and service-level SLOs for the following infrastructure data:

• Memory Utilization

• CPU Utilization

• Percentile Latencies

Enterprise Business Value

Grey Matter's telemetry data gives you insight into microservice performance and control over your system's resource use. It offers the following key telemetry backed value-add benefits.

Improve Data and Communications Management

is the control and data plane responsible for managing the mesh environment. Envoy-proxy backed communications and traffic management, policy enforcement, audit, and data governance take place atop Fabric.

Grey Matter Data‌

is a data store agnostic API designed to manage access control and zero-trust data security policy compliance. Data enables secure global data sharing backed by complex content tagging, object tracking, and security policy. Data is store agnostic, and capable of supporting multiple storage backends to include Amazon S3, CEPH, Gluster, local disk, and more.

​Improve Operations and Network Management

Grey Matter includes several open-source tools that improve quality and security, and offer low-cost and flexible ways to meet goals. With Grey Matter, network orchestration and policy management do not require expensive third-party tools and licensing fees.

Questions?

Artifacts

Grey Matter v1.2 artifacts are now available. Artifacts can be found in the staging repositories:

• Images

1.2.2

Server versions (if changed from 1.2.1):

Fabric

• gm-proxy:1.4.5

Fixed

• Fixed intermittent segfault when using websockets

1.2.1

Server versions (if changed from 1.2.0):

Fabric

• gm-proxy:1.4.4

• gm-jwt-security-gov:1.1.2

• gm-cli:1.4.2

Sense

• gm-dashboard:3.4.2

• gm-slo:1.1.5

Changed

• JWT-Security filter cache now respects token expiration and internally limits the max cache size

• Sidecar now supports FIPS-compliant builds

• Sidecar base build updated to Envoy 1.13.3

1.2

Server versions:

Fabric

• gm-proxy:1.4.2

• gm-control:1.4.2

• gm-control-api:1.4.4

Sense

• gm-dashboard:3.4.1

• gm-slo:1.1.4

• gm-catalog:1.0.7

Platform Services

• gm-data:1.1.1

Release Notes

Added

• Allow setting Envoy HTTP filters

• Allow setting Envoy Network filters

• Allow setting Network and HTTP filters on Grey Matter Listener object

Changed

• Sidecar base Envoy build updated to 1.13.1

• gm.inheaders filter will now return 403 if certificates are not present

• Update trace defaults to v2 APIs

Removed

• None

Fixed

• Control EDS resolution of instances now properly causes an update to be sent to the Sidecar

• Change internal protocol selection to USE_DOWNSTREM_PROTOCOL to fully support HTTP2

• Control server no longer overwrites defined static resources for the data plane

Known Issues

• The Catalog API requires that cluster name be unique, if you have two services with the same name and version values. Failure to do so will lead to a mismatch in the Sense Dashboard and you will not see one of the services. If the versions are unique then you can use the same name value.

• The proxy requires a route to be configured on the domain/listener in order for observables to be enabled.

Common Questions

If you're familiar with microservices, you've probably faced the following questions.

• How do I know what services are running in my infrastructure?

• How do I enable secure hybrid methodologies across multiple cloud and platform-as-a-service (PaaS) environments while leveraging existing on-prem investments?

• How do I deploy and enforce common security models?

• How do I avoid core function duplication (i.e. discoverability, scalability, observability, security, service level management, multinetwork fabric and data protection policies)?

• How do I understand how my new and existing IT investments are being used?

Explore Grey Matter's solutions to these common use cases below.

Microservice instances have dynamically assigned network locations, and these locations change due to autoscaling, failures, and upgrades.

Grey Matter's manage complex network traffic and data. By removing network communication complexities and infrastructure concerns, your team can focus on writing next-gen services in a polyglot, cloud-agnostic environment.

A is an infrastructure layer built into your application. It controls how parts of your app share data, and tracks these interactions so you can optimize communication and avoid downtime as your app grows. Unique service-level insight and control let you closely measure and manage business objectives.

Use a service mesh like Grey Matter to maximize network efficiency, free developers to innovate, and harness data to enable AIOps.

Modernize your applications and .

Grey Matter supports the scaling of containers, microservices, and serverless architectures, and the dashboard will help you optimize infrastructure, application performance and business outcomes. Grey Matter takes advantage of your network's data through the innovative application of neural net AI designed to enhance traditional network and application performance. Enhance your diagnostic capabilities, and enable predictive network operational monitoring and automated response.

While traditional security models rely on location-based trust, provides trust for all access requests regardless of location. It enforces adaptive controls and verifies trust on an ongoing basis.

Trust levels adapt to your evolving business. Our zero-trust approach helps prevent unauthorized access, contain breaches, and helps reduce the risk of an attacker's lateral movement.

Questions?

Welcome to Grey Matter documentation. This documentation covers all available features and options for Grey Matter, the intelligent hybrid mesh platform.

The Challenge

Microservice Monitoring Is Complex

A service mesh lacks monitoring and visibility. Without mesh operations visibility, SREs and DevOps engineers struggle to find root causes for problems that can happen anywhere: on the network, inside containers, or between services.

Audit Trail and Compliance Reporting

‌Decentralized systems already have reporting challenges that grow exponentially with new services, and cloud environments. To make matters worse, failure to comply with privacy protection policies could bring legal trouble, massive fines, and loss of accreditation.

Digital Twin Modeling

Digital twins are a cost-effective way to help decision-makers across a number of business use-cases, from technical operations to marketing and sales. They require systems capable of securely replicating, interacting with, and, as necessary, accurately modifying, massive historical data volumes at rapid speed. In addition, human user and customer data is subject to several PII policy compliance requirements that may require the anonymization or the outright removal of key user data immediately upon user request.

Security Attribution Replay (Chain of Evidence)

Service meshes, microservices, serverless, and containers are key elements of Mesh Application and Service Architecture (MASA) implementation. However, these capabilities present significant service-to-service communication, discovery mechanism, security layer, and audit observability challenges at scale. The volume of data generated by mesh operations coupled with the complexity of hybrid and multi-mesh operations can overwhelm traditional compliance tracking efforts.

The Solution

Instant Visibility and Discovery

Grey Matter's Intelligence 360 helps visualize service health, data-flows, and microservice interactions so you can solve performance issues fast. Intelligence 360 provides full-scope health observability and contextual awareness for the mesh.

The combination of Fabric orchestration and Intelligence 360 dynamic control lets your team check service health, making changes on the fly fixing an anomaly impacting service performance. Grey Matter also enables granular audit and policy compliance as well as rapid dependency identification, for real-time tracking of aggregate, route, and service level SLOs for Memory Utilization, CPU Utilization, Percentile Latencies, Error Rate, and Request Rate in an easy-to-understand manner.

Grey Matter’s omnidirectional mesh telemetry capture and analysis capabilities enable deep mesh audit and observability without the need for special instrumentation. Grey Matter employs a normalized and repeatable in-depth audit capture of every event on the mesh which can be used to establish a baseline for compliance reporting. Employing Kibana as a third-party dashboard, Grey Matter enables data lineage and provenance oversight via fully observable audit controls for rapid forensic detection and diagnosis of anomalies and intrusions.

Scale with Confidence

Grey Matter supports various containers, microservices, and serverless architectures so you can adopt cloud-native technologies without worry.

Isolate the Root Cause

Grey Matter records and displays every activity within the mesh for in-depth audit control, policy compliance reporting, rapid forensic detection and diagnosis of anomalies and intrusions. You can cut through the noise to find performance outliers throughout your mesh and pinpoint the root cause of your problem within seconds.

Distributed Tracing

Grey Matter offers distributed tracing to monitor service availability and performance. This data helps teams troubleshoot the mesh and improve mesh performance.

Omnidirectional Mesh Telemetry Data

Grey Matter offers in-depth observability and policy compliance management for your complex multi-environment networks. Grey Matter sidecars run alongside each microservice within the mesh, creating a controlled and trusted service edge fleet guided by dynamically established policy guidelines. These services are responsible for managing network requirements such as scaling, access control, policy compliance, audit, and service-to-service intercommunication.

Grey Matter’s omnidirectional mesh telemetry capture and analysis capabilities enable deep mesh audit and observability without the need for special instrumentation. Network micro-segmentation and policy enforcement ensure zero-trust managed security compliance.

Mesh telemetry data powers architecture-wide service level objective (SLO) policy compliance and fine-grained hybrid mesh operational control. With Grey Matter, users can dynamically set SLOs governing each policy action atop the mesh. If one approaches warning or violation, Grey Matter alerts via an intuitive single-pane-of-glass interface for rapid mitigation.

Zero-Trust Security

Grey Matter is designed to operate using a zero-trust threat model to ensure each service and transaction running within a Grey Matter enabled hybrid mesh is appropriately protected. This is supported by the normalized and repeatable in-depth audit capture of every event on the mesh which can be used to establish a baseline for compliance reporting. The concept of zero-trust is centered on a belief that enterprises do not automatically trust systems or services inside or outside its perimeters. In the case of Grey Matter, everything attempting to connect is verified before granting access. Every action is recorded and can be played back for forensic analysis.

Grey Matter records and displays the lineage and provenance of every activity conducted by every user on every object atop the mesh throughout its life-cycle. The platform makes this information fully observable and accessible for in-depth audit control, policy compliance reporting, and rapid forensic detection and diagnosis of anomalies and intrusions.

Grey Matter is purposefully designed to capture and analyze the massive volumes of telemetry, user, and operational data generated by mesh operations. With Grey Matter, every piece of network traffic, user activity, or policy-driven system action can be brought together to create digital twins of systems, networks, and even individual users or group for analysis and testing. Because Grey Matter is built with zero-trust control and audit tracing in mind, PII information can be anonymized or selectively removed in keeping with legislative requirements such as the EU General Data Protection Regulation (GDPR).

Questions?

Overview of Grey Matter's Functionality

Grey Matter is composed of Fabric, Data, and Sense. Internal to each component is a series of microservices that offers several core features. Each feature simplifies technical challenges associated with service management, such as:

• Announcement

• Discovery

• Instrumentation

Workload Distribution

The following diagram shows the workload distribution between Grey Matter's core components.

Grey Matter Fabric

Fabric powers the zero-trust hybrid service mesh, which consists of the , , , and . You can use Fabric to connect services regardless of language, framework, or runtime environment.

Secure network fabrics provide bridge points, observability, routing, policy assertion, and more between on-premise, multi-cloud, and multi-PaaS capabilities. Fabric offers workload distribution and management within a hybrid environment.

Support for Multiple Runtime Environments

Grey Matter supports multiple runtime environments with multi-mesh bridges as shown below. These environments include:

• Multiple cloud providers (i.e. AWS and Azure)

• Container management solutions (i.e., K8s, OpenShift and ECS)

• On-premise infrastructure

OSI Model Layers

Fabric operates at layers 3 (network), 4 (transport), and 7 (application) simultaneously. Providing a powerful, performant, and unified platform to run, manage, connect, and perform distributed workloads across a hybrid architecture.

Layer 3 operates at the TCP level. Responsible for transferring data “packets” from one host to another using IP addresses, TCP ports, etc., determining which route is the most suitable from source to its destination. At this level, network-segmentation is able to be performed using ABAC, RBAC, and NGAC policies set within each sidecar. More details can be found in the section.

Layer 4 coordinates data transfer between clients and hosts. Adding load balancing, rate limiting, discovery, health checks, observability, and more built on top of TCP/IP. Layer 3 and 4 alone live within the TCP/IP space and are unable to make routing decisions based on different URLs to backend systems or services. This is where layer 7 comes into the architecture.

Layer 7 sits at the top of the OSI model, interacting directly with services and applications responsible for presenting data to users. HTTP requests and responses accessing services, webpages, images, data, etc. are layer 7 actions.

The following graphic shows Fabric's basic capabilities--access, routing decisions, rate limits, health checks, discoverability, observability, proxying, network and micro-segmentation--and how they leverage all features found within each of the OSI layers described above.

Edge

Grey Matter Edge handles flowing through the mesh. Multiple edge nodes can be configured depending on throughput or regulatory requirements requiring segmented routing or security policy rules.

• Traffic flow management in and out of the hybrid mesh.

• Hybrid cloud jump points.

• Load balancing and protocol control.

Control

• Automatic discovery throughout your hybrid mesh.

• Templated static or dynamic sidecar configuration.

• Telemetry and observable collection and aggregation.

Security

Grey Matter Fabric offers the following security features:

• Verifies that tokens presented by the invoking service are trusted for such operations.

• Performs operations on behalf of a trusted third party within the Hybrid Mesh.

Sidecar

Add Grey Matter to services by deploying a sidecar proxy throughout your environment. This sidecar intercepts all network communication between microservices.

The Grey Matter Sidecar offers the following capabilities:

• Multiple protocol support.

• Observable events for all traffic and content streams.

• Filter SDK.

Once you've deployed the Grey Matter Sidecar, you can configure and manage Grey Matter with its control plane functionality.

Grey Matter Control Plane Functionality

• Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic

• Fine-grained control of traffic behavior with rich routing rules, retries, failover, and fault injection

• A policy layer and configuration API supporting access controls, rate limits and quotas

Example

The following diagram shows how the Grey Matter Sidecar would operate in a North/South traffic pattern.

Grey Matter Data

Grey Matter Data is an API that enables secure and flexible access control for your microservices. Data consists of Grey Matter Data and JWT server, and includes an API Explorer to help you manage the API.

Grey Matter Sense

Grey Matter Sense consists of four primary components: , , and .

Intelligence 360

Intelligence 360 is our user dashboard that paints a high-level picture of the service mesh. Intelligence 360 includes the following features:

• Mesh Overview

  • Running state of all services

  • Search, sort and filter options

SLO

Grey Matter Service Level Objectives (SLOs) allows users to manage objectives towards service-level agreements. These objectives can be internal to business operations or made between a company and its customers. They are generic and are valuable in more than one use case.

SLOs combine with Intelligence 360 time-series charts to visualize warning and violation thresholds for targeted performance analysis. These objectives are used even further to train Sense AI for service scaling recommendations.

Business Impact

Business Impact allows users to set metadata on services with the goal of associating how critical a service is towards the operations of a company, mission, or customer. Business Impact provides a list of values (Critical, High, Medium, Low) that correlates each service's business impact. Sense lets users of Intelligence 360 configure these values themselves, which can be used to filter and search via the mesh overview.

Catalog

acts as an interface between the data plane (network of sidecars) of the service mesh and Intelligence 360. Catalog provides a user-focused representation of the mesh.

Learn how to here.

Questions?

Need Help?

Grey Matter consists of a control plane, data plane, mesh, telemetry-powered business intelligence, and AI. It can be deployed on multiple cloud-native or legacy infrastructures without placing predetermined downstream requirements on existing investments.

Learn about the inner workings of Grey Matter's three core components: Fabric, Data, and Sense.

Grey Matter enables unified hybrid microservice deployments and hybrid/multi-cloud operations without special requirements for underlying infrastructure like containers or container platforms regardless of cloud vendor, PaaS or infrastructure.

Questions?

The Challenge

Service discovery is important but hard. Microservices constantly communicate with instances of other microservices, intensifying the volume and frequency of communications.

The Solution

Grey Matter's service-level visibility and telemetry improves service inventory and dependency analysis.

How It Works

The server supports several service discovery options and platforms.

• Control performs service discovery in the mesh and acts as the Envoy xDS server to which all proxies connect.

• Service discovery lets services communicate transparently with a member of a cluster without knowing its identity, or even how many members there are.

• When a service joins or leaves the cluster, it joins or leaves the load-balancing list.

Implementations

Grey Matter supports *several service discovery implementations, with more to come. These include a central server or servers that maintain a global view of addresses (Kubernetes keeps this central server), and clients that connect to the central server to update and retrieve addresses.

*Except for flat-file cases; we farm out that functionality to some other service discovery system.

Here is a short explanation of the workings of each of our service discovery implementations.

Kubernetes

Grey Matter's Kubernetes service discovery scheme keeps track of labels placed on pods when they're deployed. Our software uses the Kubernetes API to ask "what pods are in this namespace?" Then we use the labels to match those pods to "clusters", so services can talk to other services just by saying "send this request to some instance of this cluster". When pods die, they get removed from Kubernetes' service discovery list, and therefore from our software's list of running instances for a cluster.

Consul

For existing infrastructures already using Consul, you can easily configure Grey Matter to discover services from the existing catalog without migration to Kubernetes or any other platform. Infrastructures with services running in heterogeneous environments, or looking to leverage features such as health checking, load-balancing, or distributed service configuration management can use Consul as a central service registry from which Grey Matter will discover.

Envoy

Many cloud infrastructure use Envoy's or to keep track of instances and upstream clusters. Since Grey Matter Control uses xDS as the configuration discovery mechanism, Grey Matter inherits all Envoy configs. Moreover, any Grey Matter proxy can act as a vanilla Envoy proxy instance, and any Envoy proxy can integrate with Grey Matter Control.

File

We also support service discovery for flat-file cases.

Features

Grey Matter monitors cloud-native environments in real-time. Grey Matter can detect and address meaningful conditions in seconds.

Grey Matter Control performs service discovery and distribution of configuration in the Grey Matter service mesh. Control provides policy and configuration for all running sidecars within the Grey Matter platform.

The Control server works in conjunction with the to manage, maintain, operate, and govern the Grey Matter hybrid mesh platform.

xDS

The Control server performs service discovery in the mesh and acts as an xDS server to which all proxies connect.

Grey Matter is a zero-trust hybrid mesh platform built using open architecture and mesh app and service architecture (MASA) principles.

Each microservice within Grey Matter runs and scales independently to improve secure interoperations, resiliency, continuity of operations, and insight for your business. Our omnichannel support provides rich, fluid, and dynamic connections between people, content, devices, processes, and services.

Core Architectural Principles

Our core architecture principles support the following business needs.

Security

Designed to operate using a zero-trust threat model to ensure each service running within a Grey Matter enabled hybrid mesh is appropriately secured, observed, and managed.

Portability

Enable on-premise, multi-cloud, and multi-platform as a service (PaaS) runtime environments.

Cloud-native

Built with elasticity, high availability, and cloud computing models in mind - provides a unified mesh platform to build applications as microservices, utilize container management solutions, and dynamically orchestrate workloads across hybrid enterprise.

Web-scale

Provides a solid foundation to scale with the growth of your business. Enabling modern architectural patterns supporting rapid increase or decrease in traffic volume, maintaining business insight for effectiveness and efficiency, and aiding in the reduction of bottlenecks when the time matters.

Modular

Modular service delivery - enabling loosely coupled systems and services developed independent of each other, taking advantage of continuous delivery to achieve reliability and faster time to market.

Interoperability

Creates a secure unified zero-trust network fabric, allowing systems to interchangeably serve or receive services from other systems providing enterprises the ability to perform multi-environment segmentation and observe traffic flows between environments. Managed through a runtime environment agnostic Grey Matter Control API.

Adaptive

Able to react to digital business changes, providing a pathway enabling business insight, security, and connectivity across multiple environments reducing complexities while increasing and facilitating a business's digital transformation journey.

Intelligent

Use of artificial intelligence (AI) techniques simplifying and assisting a user experience while providing business insight and fleet wide management across Grey Matter connected resources.

Automation

Integration into any ecosystems and end-to-end automation through the lifecycle of the mesh app and service architecture.

Questions?

The Grey Matter Sidecar is a L7 reverse proxy based off of the popular Open-Source Envoy Proxy. Grey Matter's proxy enhances the base capabilities with custom filters, logic, and the ability for developers to write full-featured Envoy filters in Go.

Fabric Mesh

The primary use of the Grey Matter Sidecar is to act as the distributed network of proxies in the Grey Matter Fabric service mesh. In this use-case; each proxy starts out with very simple configuration, which is then modified by the control plane to suit the changing needs of the network. The documentation here is focused on the individual proxy itself; low-level configuration, filter specifications, etc.

How does event auditing work?

Individual Service

At the level of the individual service, event auditing works as follows:

1. One proxy collects all metrics that happen on the individual service.

2. At the Edge, they extract the PKI/cert.

3. The user that has accessed the service from outside Fabric is then decomposed based on one of the observable fields emitted by the Sidecar proxy.

Service-to-Service

At the service-to-service level, the sidecar tracks service-to-service calls within Fabric. This enables architecture inference and service dependency observation.

Observable Indexer

Grey Matter also has an observable indexer which can capture geolocation info and move it into Elasticsearch. Customizable event mappings are also available. These can be tailored per individual route so that a POST request may result in an EventAccess event in one route, while resulting in EventCreate on another.

Questions?

The Challenge

It's hard to improve performance and operations in a dynamic microservice environment.

The Solution

Dynamic Routing and Security

Grey Matter is a platform agnostic service mesh that simplifies network management. The mesh is made of --components that work in unison to optimize decentralized microservice performance on the network. Grey Matter's service policies and configurations enable dynamic routing and security based on service identity. These policies scale without IP-based rules or networking middleware.

Benefits of a Service Mesh

Grey Matter takes complexity from a single microservice and puts it into a "sidecar" proxy. This sidecar proxy works with its dedicated service to provide the following benefits:

• It gives its service behaviors the service needs to perform well in a microservice architecture, and

• It lets its dedicated service perform its business-specific tasks.

The following table summarizes the benefits a service mesh provides.

Features

Real-Time Performance Metrics

If you’re building microservices, you're anticipating the ability to scale, since a microservices architecture will look very different a year out. A new service introduces failure points, and microservices make it hard to find the root of failures without a mesh. A service mesh captures all communications as performance metrics. These metrics translate to more reliable service requests and a more secure way to scale.

Secure and Reliable Decision-Making

Grey Matter separates decision-making from data-gathering with its data and control planes to improve the performance of each of these important activities.

The data plane is a collection of sidecar proxies: one proxy for each service. The data plane manages traffic from one application to another and includes routing, forwarding, load balancing, even authentication and authorization.

The control plane connects data planes and serves as the policy and management layer of the service mesh. It collects telemetry data and makes decisions about configurations.

Hybrid Cloud, Traffic Management, and Observability

Grey Matter lets you set policies you can enforce across cloud instantiations. Its single abstraction layer hides details of the underlying cloud.

Service-to-service communication can be managed centrally, enabling advanced traffic management patterns such as service failover, path-based routing, and traffic shifting that can be applied across public and private clouds, platforms, and networks.

Centrally-managed service observability includes detailed metrics on all service-to-service communication such as connections, bytes transferred, retries, timeouts, open circuits, and request rates, response codes.

Secure Services Across Any Runtime Platform

Grey Matter offers secure communication between legacy and modern workloads. Sidecar proxies allow applications to be integrated without code changes and Layer 4 support provides nearly universal protocol compatibility.

Certificate-Based Service Identity and Encrypted Communications

Grey Matter uses TLS certificates to identify services and secure communications. Using TLS provides a strong guarantee of the identity of services communicating, and ensures all data in transit is encrypted. These certificates use the SPIFFE format for interoperability with other platforms. Grey Matter can be a certificate authority to simplify deployment, or integrate with external signing authorities like Vault. All traffic between services is encrypted and authenticated with mutual TLS.

Questions?

The Challenge

All Network Traffic Is Untrusted

First coined by Forrester Research, zero-trust architecture “abolishes the idea of a trusted network inside a defined corporate perimeter.” Put simply, zero-trust means “never trust, always verify.” Zero-trust assumes your systems are already compromised by cyber intrusion. Under zero-trust, the enterprise is mandated to create micro-segmentation around sensitive data, backed by deep visibility into how the enterprise uses data across its ecosystem in pursuit of customer satisfaction. This combination of micro-segmentation and awareness greatly enhances security across the enterprise.

As described by O’Reilly Media, the increasingly popular zero-trust approach is based on five key premises:

1. The network is always hostile.

2. External and internal threats always exist on the network.

3. Network locality isn’t enough to decide trust in a network.

Why is Network Perimeter Trust Insufficient?

Bad actors can breach networks in countless ways. Granting trust to a user who was somehow accessed only one layer of your network security creates both a false sense of security and introduces multiple security gaps.

Perimeter trust ignores policy and context change. Relying on IP address data to establish local trust is insufficient protection. For instance, this ignores risk based on user type and business role and request reason (geo-location/time)‌. In addition to security concerns, ignoring these controls also represents potential policy compliance failure.

Perimeter trust also ignores the possibility of compromised credentials. Per , 32% of breaches stemmed from phishing attacks, while 29% involved stolen credentials. By trusting the credentials of a single user, your network becomes susceptible to similar outside attack.

Finally, perimeter trust also ignores the existence of compromised devices already on the network. Compromised devices can be purposefully or accidentally introduced to corporate networks by countless means. research indicates that “one in 36 devices used in organizations present high risk.” Such devices may include malware and other malicious software. ‌

The Solution

Zero-trust Architecture

‌‌Zero Trust security provides defense in depth based on a handful of guiding practices. ‌

• Trusted user identities backed by strong identification, visibility & authentication,

• Enable secure access to all apps on the network,

• Enforce adaptive & risk-based policies at endpoints, and

Grey Matter ensures security and access control based on zero-trust design and implementation.

How does Zero Trust Improve Security?

Industry has signaled increased interest in zero-trust infrastructure for service-to-service mTLS connections, scheduled or on-demand key rotations, service cryptographic identifiers, observability (i.e. continuous monitoring, granular audit compliance), service level management, and policy management throughout the enterprise service fleet.

Grey Matter meets each of these requirements. Leveraging zero-trust within Grey Matter, development teams can quickly and flexibly deploy new capabilities and functions without specific instrumentation required for security or compliance. The platform enables zero-trust segmentation that enforce secure business operations while ensuring audit capture for every event in a normalized and repeatable way establishing a baseline for compliance reporting.

The concept of Zero Trust is centered on a belief that enterprises do not automatically trust systems or services inside or outside its perimeters, instead verify everything attempting to connect before granting access. Grey Matter is designed to operate using a zero-trust threat model to ensure each service and transaction running within a Grey Matter enabled hybrid mesh is appropriately protected.

Grey Matter elevates a multi-facet security model and unprecedented compliance insight into the service mesh and data layers, drastically reducing developer complexity burden. The platform enables Enterprise IT teams to continuously deploy to a common hybrid mesh while maintaining security enforcement and compliance reporting.

Steps to a Zero-Trust Enterprise

Zero Trust security requires the following six areas of control. Together, they provide a defense in depth approach to securing corporate resources no matter where they’re deployed and who needs access to them.

Establish Trust in User Identities

Strong Identification & Authentication

Verify the identity of all users with secure access solutions such as two-factor authentication (2FA) before granting access to corporate applications and resources

During authentication and authorization, verify the following before proceeding:

• Is this user legitimate?

• Was this user identified in a manner that is acceptable to the task being performed?

• Is their device healthy enough for the task they are performing?

from the moment of registration to each request for access is critical to improving security. These capabilities ensure that all users (privileged and not) and all resources are protected no matter where they’re deployed.

Gain Visibility into Devices & Activity

Gain visibility into every device used to access corporate applications, whether or not the device is corporate managed, without device management agents.

Enforce Adaptive & Risk-Based Policies

Endpoint Security

Legitimate users often incidentally expose their organizations to high levels of risk by accessing resources with compromised devices. These capabilities ensure that .

Protect every application by defining policies that limit access only to users and devices that meet your organization's risk tolerance levels. Define, with fine granularity, which users and which devices can access what applications under which circumstances.

Enable Secure Access to All Apps

Grant users secure access to all protected applications through a frictionless secure single sign-on interface accessible from anywhere without a VPN. Protect all applications - legacy, on-premises and cloud-based.

Network Security

Preventing lateral movement between segments is often the most effective way to minimize the impact of a breach. These capabilities ensure that breaches are contained with access terminated as soon as malicious behavior is detected or

Workload Security

Attacks come from those with valid credentials as well as from the outside. These capabilities ensure that and that vulnerabilities in applications and APIs are covered.

Transaction Security

Ensure Device and Data Transaction Trustworthiness

Certain transactions pose more risk than others. Grey Matter's capabilities ensure that high-risk transactions are verified by the user while recognizing anomalous behavior.

Grey Matter inspects all devices used to access corporate applications and resources at the time of access to determine their security posture and trustworthiness. Devices that do not meet the minimum security and trust requirements set by your organization are denied access to protected applications.

• Is this session still driven by the real user?

• Does the amount of trust in the user identity match the level of risk associated with this transaction?

• Has the request been verified?

Data Security

Whether its sensitive IP or user data covered by one of the many privacy regimes popping up around the globe, data security has become paramount for many organizations. These capabilities ensure that data is encrypted where it needs to be, and that .

Where to start?

Start your zero-trust journey with a strategic deployment of global, adaptive authentication. Use this capability as the policy administration and decision point for where all risk signals and policy decision meet.

Questions?

Prerequisites

1. git installed

2. helm v3

3. envsubst (a dependency of our helm charts)

4. eksctl or an already running Kubernetes cluster.

Steps

1. Install a Kubernetes Cluster

NOTE: if you already have a Kubernetes cluster up and running, move to step 2. Just verify you can connect to the cluster with a command like kubectl get nodes

For this deployment, we'll use to automatically provision a Kubernetes cluster for us. The eksctl will use our preconfigured AWS credentials to create master nodes and worker nodes to our specifications, and will leave us off with kubectl setup to manipulate the cluster.

The regions, node type/size, etc can all be tuned to your use case, the values given are simply examples.

Cluster provisioning usually takes between 10 and 15 minutes. When it is complete, you will see the follwing output:

When your cluster is ready, run the following to test that your kubectl configuration is correct:

2. Clone the Grey Matter Helm Charts Repo

Though Helm is not the only way to install Grey Matter into Kubernetes, it does make some things very easy and reduces a large number of individual configurations to a few charts. For this step, we'll clone the public git repository that holds Grey Matter and cd into the resulting directory.

NOTE: this tutorial is using a release candidate, so only a specific branch is being pulled. The entire repository can be cloned if desired.

3. Setup Credentials

Before running this step, determine whether or not you wish to install . If so, determine whether or not you will use S3 for backing. If you do want to configure Grey Matter Data with S3, follow the guide. You will need the AWS credentials from here.

To set up credentials, we need to create a credentials.yaml file that holds some secret information like usernames and passwords. The helm-charts repository contains some convenience scripts to make this easier.

Run:

and follow the prompts. The email and password you are prompted for should match your credentials to access the Decipher Nexus at . If you have decided to install Grey Matter Data persisting to S3, indicate that when prompted, and provide the access credentials, region, and bucket name.

Note that if your credentials are not valid, you will see the following response:

4. Configurations

To see the default configurations, check the global.yaml file from the root directory of your cloned repo. In general for this tutorial, you should use the default options, but there are a couple of things to note.

• If you would like to install a Grey Matter Data that is external and reachable from the dashboard, set global.data.external.enabled to true.

  • If you are installing data and set up your , set global.data.external.uses3 to true.

You can set global.environment to eks instead of kubernetes for reference, but we will also override this value with a flag during the installation steps in .

5. Install Grey Matter component Charts

Grey Matter is made up of a handful of components, each handling different pieces of the overall platform. Please follow each installation step in order.

1. Add the charts to your local Helm repository, install the credentials file, and install the Spire server.

2. Watch the Spire server pod.

   Watch it until the READY status is 2/2, then proceed to the next step.

6. Accessing the dashboard

NOTE: for easy setup, access to this deployment was provisioned with quickstart SSL certificates. They can be found in the helm chart repository at ./certs. For access to the dashboard via the public access point, import the ./certs/quickstart.p12 file into your browser of choice - the password is password.

An will be created automatically when we specified the flag --set=global.environment=eks during installation. The ELB is accessible through the randomly created URL attached to the edge service:

You will need to use this value for EXTERNAL-IP in the .

Visit the url (e.g. https://a2832d300724811eaac960a7ca83e992-749721369.us-east-1.elb.amazonaws.com:10808/) in the browser to access the Intelligence 360 Application

7. Configure the CLI

If you intend to move onto into your installation, or otherwise modify/explore the Grey Matter configurations, you will need to .

For this installation, the configurations will be as follows. Fill in the value of the edge service's external IP from the for <EDGE-EXTERNAL-IP>, and the path to your helm-charts directory in <path/to/helm-charts>:

Run these in your terminal, and you should be able to use the CLI, greymatter list cluster.

You have now successfully installed Grey Matter!

Cleanup

If you're ready to shut down your cluster:

Delete the Grey Matter Installation

Delete The EKS Cluster

NOTE: this deletion actually takes longer than the output would indicate to terminate all resources. Attempting to create a new cluster with the same name will fail for some time until all resources are purged from AWS.

Get Started with Grey Matter

Prerequisites

To complete the following guides, you will need:

1. An AWS account with permissions to provision an EKS cluster and the set up.

2. A Grey Matter account with access to the . If you don't have an account contact us at .

Steps to Grey Matter

To get started with a core Grey Matter installation, step through our guides in the following order:

Overview

Check out the for reference information on Grey Matter.

Questions?

Service meshes, microservices, server-less, and containers are key elements of Mesh application and service architecture (MASA) implementations. MASA, APIs, and internal traffic patterns represent one of the most effective pathways to enterprise modernization, but this doesn’t come without challenges.

Industry has signaled increased interest in zero-trust infrastructure for service-to-service mTLS connections, scheduled or on-demand key rotations, service cryptographic identifiers, observability (continuous monitoring, granular audit compliance, etc.), service-level management, and policy management throughout the enterprise service fleet.

Security Models

Understanding how the roles of Authentication, Authorization, Claims, and Principals will play within your MASA is important (figure 1). Authentication and authorization are both significant in any security model, but follow different concepts and implementation patterns. Authentication establishes and confirms an identity. Authorization takes action based on the confirmed identity authenticated. Principals are asserted claims that provide entitlements granting access to systems, services, or data based on Role-based Access Control (RBAC), Attribute-based Access Control (ABAC) and Next Generation Access Control (NGAC) controls.

Authentication

Grey Matter's authentication scheme establishes identities for every transaction within the platform. There are two types of identities: users and services.

User Authentication methods:

• OpenID Connect (OIDC)

• mTLS x.509 certificates (Distinguished names represent who the user is)

Service-to-Service Authentication methods:

• mTLS x.509 certificates (SPIFFE identities are incorporated into the x.509 certificate)

While distinct, these identities are not mutually exclusive. One of the most common access patterns within Grey Matter is a service making a request to another service on behalf of a user. In this case, there are three identities (two services and a user), each of which must be verified in order for the transaction to succeed. As users or services authenticate with Grey Matter, principals are asserted and flow to upstream services. This ensures that upstream services are aware of the entity (user or service) making a request. Grey Matter supports user authentication and service-to-service authentication methodologies identified below.

User Authentication with an OIDC Provider

Grey Matter integrates with existing public OIDC providers (Google, Github, etc.) or private OIDC providers (e.g., Ory Hydra) to support user authentication. OIDC is an authentication protocol built on top of OAuth 2.0 that allows delegation of authentication responsibility to a trusted external identity provider. Many implementations of OIDC providers are available and support on premise, cloud or as a service via a host of underlying technologies (e.g., LDAP). This sequence diagram (figure 3) shows the OIDC flow within Grey Matter.

1. The client initiates a request to Grey Matter Edge.

2. Grey Matter Edge responds with a 302 HTTP status code used to perform URL redirection, along with a callback URL.

3. Based on the redirect URL, the client initiates a request to the specified OIDC provider.

x.509 Certificates

Grey Matter supports x.509 for both users and for service to service transactions.

1. A client (user or service) initiates a request to a server.

2. The server responds with its server certificate.

   2.1. The client verifies that the server’s certificate is valid based on its certificate information.

3. The server requests the client's certificate.

User Authentication with x.509 Certs

Enterprise IT organizations that have existing public key infrastructure (PKI) in place for user authentication can pass their certifications with requests made to the Grey Matter Edge.

Service Authentication with x.509 Certs

Service authentication, (service-to-service communication), is based solely upon x.509 certificates and mTLS. Grey Matter Fabric is installed with a certificate authority that issues and reissues short- lived x.509 certificates to each sidecar proxy for intermesh communication. Each certificate contains a SPIFFE identity that uniquely identifies the sidecar to which it is issued. No sidecar will accept a connection from any service that does not present a certificate issued by the certificate authority. Like user authentication, these service identities enable authorization.

Note: In cases where requests already contain a signed cookie the edge simply verifies the signature and expiry. If valid, the edge forwards the request. If not valid, the request is treated as unauthenticated.

Authorization

Authorization is the process by which identities (users or services) are granted permission to access resources within the mesh. For example, we may wish to restrict access to a specific resource to a limited set of users, services or data. As an added complication, it is often more desirable to grant or deny access for a resource to entire classes of identities (i.e., administrative users or trusted services). Grey Matter uses the authenticated identities and their attributes to support fine-grained access controls using the following methods:

• Authorization Filters

• Data Authorization via the Grey Matter Data Platform Service

It's important to note that Sidecar-to-Sidecar (service-to-service) authorization follows similar patterns of a user with the exception that sidecar identities typically do not include additional attributes; however, there is nothing precluding the addition of attributes for a sidecar identity.

Authorization filters

Upon choosing an authorization pattern, access control becomes a deployment concern, not a development concern. Allowing microservice developers to focus on business value since their services will not receive any unauthorized request. Authenticated identity and attributes are available to the service should they be required.

The Grey Matter Sidecar uses authorization filters to manage who is allowed to access which resources and how. Since all requests to the mesh are authenticated, filters can be dynamically configured at runtime with no additional requirements. Attribute based authorization is also implemented via Grey Matter Sidecar filters but requires that requests contain a signed JSON Web Token (JWT) containing the identity claims. The creation and population of these tokens is left to the enterprise.

List Authorization Filter

The Grey Matter Sidecar supports list-based authorization decisions within the ListAuth filter. This filter allows whitelisting and blacklisting of individual identities based upon the identities distinguished name (i.e, “cn=user, dc=example, dc=com” or “cn=web server, dc=example, dc=com”) or relative distinguished name (i.e., “dc=example, dc=com”). This filter applies to all requests for the proxied service or services.

Role-Based Access Control Filter

The Grey Matter Sidecar supports fine grained authorization decisions to authorize actions by identified clients using Role-Based Access Control (RBAC). This filter allows complex whitelisting and blacklisting of individual identities based upon the identities distinguished name. Matching of regular expressions is supported to add additional flexibility. Further, whereas the ListAuth filter applies to all requests, the Role Based Access Control Filter can be defined for any combination of service, route, or verb. This is useful to explicitly manage callers to a service running within the Grey Matter mesh platform and protect the mesh from unexpected or forbidden agents.

Supported HTTP verbs include:

• GET The GET method requests a representation of the specified resource. Requests using GET should only retrieve data.*

• POST The POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server.*

• PUT The PUT method replaces all current representations of the target resource with the request payload.*

Definitions as described by the Mozilla Developer Network (MDN).

In situations where the identity is not sufficient to make all authorization decisions, the Grey Matter Sidecar can enforce finer-grained control based upon identity attributes, provided the request contains a sign JWT.

Using the RBAC filter, rules can be created to authorize specific claims found within a JWT to perform specific actions. This requires an external service to generate a signed JWT for each request. Since the JWT is included as a header, if a JWT is passed, it will propagate to all sidecars in the request chain. With that said, if the request is completed—meaning the destination service has received it, processed it, and invokes another service in the Mesh—this is a new request and the calling service would be required to pass the JWT for further authorization purposes.

Custom Access Control Filter

The Grey Matter sidecar offers a custom filter interface, so customers have the ability to create business-specific logic around their security and regulation concerns if required. This makes the mesh fully adaptable to an enterprise’s needs, and provides a way to take advantage of existing IT investments.

Data Authorization

One of the unique facets of Grey Matter is that data security and sharing is addressed by Grey Matter Data. As enterprises shift from monoliths to microservices, data tends to be duplicated across the architecture. Grey Matter Data provides a service to address the secure sharing of this data without marshalling it into and out of processes. This feature is described in greater detail in the Data Segmentation portion of this document, but the pattern employed by Grey Matter Data can be used by any service to enforce complex security policies for any resource via the Grey Matter JWT Security Service or customer JWT service adhering to the Grey Matter Data interface.

Traffic Patterns

One key feature of the Grey Matter hybrid mesh is its ability to secure, manage, and govern the traffic patterns of running services.

East/West Traffic

East/West traffic within the Fabric Mesh should be done via mTLS. Grey Matter uses two methods enabling this: direct integration with existing CAs and, automatic setup via SPIFFE/SPIRE. For integration with existing CAs, each sidecar in the mesh is configured to use provided x. certificates. In the automatic setup, each sidecar uses a unique SPIFFE ID to authenticate with SPIRE servers. Unique short-lived x.509 certificates are then automatically created and rotated for each connection between sidecars.

North/South Traffic

North/South traffic patterns use the Grey Matter Edge to establish principals and pass them to called services. The Edge supports both OIDC and mTLS x.509 certificate authentication modes, however, the Fabric Mesh is not limited to a single Edge. Multiple nodes can be configured to expose both authentication modes wherever an access point is needed. Note that the Edge node does not have to be exposed to a publicly addressable URL. In many cases, an API mediation layer may be put in front of the Edge node. In all cases, the Edge node is responsible for verifying and ensuring that the proper principals are available for downstream services within the Fabric Mesh to consume.

Principals such as user identities are moved by the Edge node into a user_dn header which flows through the entire service-to-service request chain. Each following link in the request chain is performed via mTLS, with each unique service using automatically rotating x.509 certificates established via SPIFFE Identities and the SPIRE framework.

In some cases traffic needs to flow outside of the mesh. Common scenarios include mesh-to- mesh communications, proxying to serverless functions, and supporting legacy systems that can’t be moved directly into the mesh. In all these cases, proxies are setup within the mesh with the sole purpose of communicating outside. Principals are established at the Edge. Inter-mesh communication is still handled by mTLS, and requests are authenticated via the outside system by whatever method it accepts: RPC, HTTP, mTLS, or OIDC.

Traffic Splitting

Traffic splitting is another important pattern in stable environments. Traffic splitting allows configurable service requests to siphon off percentages of requests to another source. This allows services, apps, or entire meshes to experience small amounts of live traffic while keeping most users on the original source. The percentage of users on the original service is then decreased until the service is fully migrated.

Circuit Breaking

Circuit Breaking is a way for each sidecar to protect the thing it is proxying to, but it is not a way to have that proxy harden itself. Grey Matter provides circuit breakers at every point in the mesh.

The most common place for this to occur is at the edge, where a DDOS could overwhelm the edge nodes themselves. To solve this, we employ Rate Limiting, which can protect the edge node from accepting too many requests and opening too many file handles and crashing. With proper configuration, each sidecar ceases queueing new requests before they’re overwhelmed, allowing the service time to heal. This ensures capabilities can withstand malicious attacks and accidental recursive network calls without going down.

Network Segmentation

Enterprises prefer hybrid environments capable of leveraging unified on-premise and cloud resources. Traditional networking patterns use features such as VLANs to create perimeter-based firewalls, but this concept breaks down with modern mesh application service architecture (MASA) patterns. In MASA, services are designed to be ephemeral, dynamically generating different IP:PORT pairs each time a new instance spins up.

Securing this type of architecture requires network segmentation. Grey Matter isolates services and network fabric communications to specific runtime environments or infrastructure resources. Grey Matter Fabric supports segmentation to a very fine level of granularity. Each service launched onto Fabric comes online with no knowledge of or connections to any other point on the mesh. The desired mesh is then built up through configuration with the required network topology. Segmentation is enforced through routing rules, service discovery, and mTLS. Dynamic configuration can facilitate any permutation of intra-mesh communication required. In addition to segmentation of individual meshes, Grey Matter can also support multi-mesh operations. This allows the bridging of environments already physically or logically isolated from each other.

Micro-segmentation

Micro-segmentation is a method of creating secure isolation zones either on-premise or in the cloud in order to separate different workloads. Authentication plays a key role in micro-segmentation. Authentication is responsible for establishing network communications and flow through the mesh. Strong authentication models enable Grey Matter to perform micro-segmentation for users, services, and data throughout the mesh.

User-to-Service segmentation is controlled through user authorization signatures. These can be coupled with claim-based assertions. User identities and claims flow through mTLS-encrypted communication channels established by service-to-service micro-segmentation patterns. Complex security policies within each sidecar allow ABAC/RBAC down to the service, route, and HTTP verb permit. This enables a very high degree of isolation. ABAC/RBAC policies cannot be achieved without strong authentication methodologies establishing identities for both users and services.

Service-to-Service segmentation is controlled through mTLS certificates and SPIFFE identities. These can be coupled with claims-based assertions and ABAC/RBAC policies. Images in the following section illustrate how this is achieved.

Data Segmentation

Grey Matter’s data segmentation capability is a key differentiator. Data segmentation is the process of dividing data and grouping it with similar data based on set parameters. Grey Matter Data adds complex policy assertions to stored objects. These object policies govern which users or services may access the objects. Objects stored within Grey Matter Data are encrypted at rest and in transit. A JSON Web Token (JWT) is provided to gain access to an object stored in Data.

The token’s claims are dynamically mapped to the policies stored with the object. JWTs for both users and services can be created enabling end-to-end security using authentication principals.

The example above shows how data-segmentation is achieved through simple policy. However, the Grey Matter Data policy engine is designed to deal with complex rules designed to suit any scenario. The following scenario presents a more complex use case.

1. Sidecar A saves an object into Data and provides access privileges to Sidecar B’s SPIFFE identity. Sidecar A dynamically discovers Data via the Data Sidecar routing information.

2. Data Sidecar receives Sidecar A’s request and streams the object (with policy) into the Grey Matter Data node.

3. Sidecar B (through a means of event-based architecture patterns) is notified that Sidecar A just saved an object of interest into Data. Sidecar B calls into Data (through the Data Sidecar) to retrieve the object. Sidecar B’s SPIFFE identity is passed along with the request.

Since Grey Matter uses a unified principal model, data segmentation can be achieved for users as well. Grey Matter Data policies can be set to identify different access privileges for services and users on a single stored object, and can be customized around business needs. This paradigm provides a new model that combines network, information assurance, and protection concepts around zero-trust.

Grey Matter Data supports the ability to host multiple Data nodes available through different routing rules. When coupled with other segmentation features, enterprises are able to further isolate how information is stored, accessed, and controlled based on customer regulations and requirements.

For example, logs and observable traffic can be isolated based on zones. Data nodes with specific routing rules and policies are set to enforce the topology. Customer application data can be stored and accessed via different Data nodes (on-premise or in the cloud) and tightly controlled at the micro-segmentation layer or via data policies. These types of flows are depicted in the diagram below.

Conclusion

Grey Matter’s zero-trust threat model ensures security across every service in the hybrid mesh. Each transaction is authenticated and authorized through a combination of mTLS and SPIFFE authentication and SPIRE authorization providing multiple layers of zero-trust security. Grey Matter also supports fine-grained access control by combining authenticated identities with policy-enforced object authorization and enables East-West and North-South traffic pattern splitting and shadowing for in-depth monitoring and configuration. Finally, Grey Matter uses network and data segmentation to decompose operations to their most basic elements, to mitigate cyber intrusion impacts, and to optimize operations.

Questions?

This guide is a step by step walkthrough on deploying a new service into an existing Grey Matter deployment. This guide is for a SPIFFE/SPIRE enabled deployment. For a walkthrough on deploying a service into a non-SPIFFE/SPIRE Grey Matter setup, follow the Quickstart Launch Service to Kubernetes for a non-SPIFFE/SPIRE Deployment.

If you are looking for a quick service deployment and/or configuration guide, check out the Grey Matter templates.

Prerequisites

1. An existing Grey Matter deployment running on Kubernetes ()

2. kubectl or oc setup with access to the cluster

3. greymatter cli with access to the deployment

Overview

1. Launch pod with the service and sidecar

2. Create Fabric configuration for the sidecar to talk to the service

3. Create Fabric configuration for the Edge to talk to the sidecar

Steps

1. Launch pod

The service we'll launch is a simple Fibonacci service. It has one route: /fibonacci that calculates the fibonacci sequence of any integer supplied.

Note the spire-specific configurations to the deployment - the volume and volume mount spire-socket and the environment variable SPIRE_PATH. These are the additions that will need to be made to any deployment for a service you wish to add to the mesh with SPIFFE/SPIRE.

Save the above deployment file as deployment.yaml and launch the pod:

2. Local Routing

The next steps are to create objects in the Fabric API. These objects will create all the configuration for the Sidecar to handle requests on behalf of the deployed service.

This step creates and configures the Grey Matter objects necessary to allow the sidecar container in the deployment to route to the fibonacci container (the service itself). We will refer to this as "local" routing. The will configure the Edge proxy to route to the fibonacci sidecar, thus fully wiring the new service into the mesh for routing.

NOTE: This guide goes over deploying a new service and configuring it for ingress routing. To configure a service for both ingress and egress routing within the mesh, see the .

For each Grey Matter object created, create the local file and send them to the API with: greymatter create <object> < <file_name.json>.

Domain

The first object to create is a Grey Matter Domain the ingress domain for the fibonacci sidecar. This object does virtual host identification, but for this service we'll accept any hosts ("name": "*") that come in on the port 10080 (the port with name proxy-or value of Grey Matter Control environment variable GM_CONTROL_KUBERNETES_PORT_NAME-in the sidecar container). Note that force_https is set to true, requiring any incoming connection be https.

See the for more information.

Save this file as domain.json and apply it with

Listener

The next object is the ingress listener. This is the physical binding of the Sidecar to a host interface and port and is linked in the field domain_keys to a specific domain. The listener and domain configurations determine where the sidecar should listen for incoming connections on and what kind of connections it should accept.

The listener object is also the place to configure . See the for more information.

Note the secret field. This field is required for service to service communication in a SPIFFE/SPIRE setup. This secret tells the sidecar to fetch its SVID (with ID spiffe://quickstart.greymatter.io/fibonacci) from Envoy and present it to incoming connections. It also will set a with match subject alternative names specifies to only accept incoming requests with SAN spiffe://quickstart.greymatter.io/edge. See the for specifics. The listener secret configuration will be important for the .

Save this file as listener.json and apply it with

Proxy

The proxy object links a sidecar deployment to it's Grey Matter objects. The name field must match the label on the deployment (in this case greymatter.io/control) that Grey Matter Control is looking for in it's environment variable GM_CONTROL_KUBERNETES_CLUSTER_LABEL. It takes a list of domain_keys and listener_keys to link to the deployment with cluster label matching name.

See the for more information.

Save this file as proxy.json and apply it with

Local Cluster

The next object to create is a local cluster. The cluster is in charge of the egress connection from a sidecar to whatever service is located at its configured instances, and can set things like circuit breakers, health checks, and load balancing policies.

This "local" cluster will tell the sidecar where to find the fibonacci container to send requests. From the deployment above, we configured the fibonacci container at port 8080. Since the sidecar and fibonacci containers are running in the same pod, they can communicate over localhost.

See the for more information.

Save this file as fibonacci-local-cluster.json and apply it with

Local Shared Rules

The shared rules object is used to match to . They can do some features of routes like setting retry_policies and appending response data, but they also can perform traffic splitting between clusters for operations like blue/green deployments.

This local shared rules object will be used to link the in the next step to the that we just created.

See the for more information.

Save this file as fibonacci-local-rules.json and apply it with

Local Route

match against requests by things like URI path, headers, cookies, or metadata and map to shared_rules. Since this service only needs to forward everything it receives to the local microservice, the setup is fairly simple.

This "local" route will link the (fibonacci-domain) to the (fibonacci-local-rules) that we just created. We know that the fibonacci-local-rules object is used to link routes to the fibonacci-cluster, thus with this route object applied, the fibonacci sidecar will be configured to accept requests and route to the fibonacci service.

See the for more information.

The path indicates that any request coming into the sidecar with path / should be routed to the fibonacci service. We will see in the next step when configuring that all requests from the Edge proxy to the fibonacci service will come in at this path.

Save this file as fibonacci-local-route.json and apply it with

The sidecar will now be configured to properly accept requests and route to the fibonacci service. The next step will configure the Edge proxy to route to the sidecar.

3. Edge Routing

Now that the Sidecar-to-Service routing has been configured, we will set up the Edge-to-Sidecar routing because we want this service to be available to external users.

The process will take similar steps to what was done before, but we only need to create a cluster, a shared_rules object pointing at that cluster, and two routes.

Edge to Fibonacci Cluster

This will handle traffic from the Edge to the Fibonacci Sidecar. The Edge has existing domain (with domain key edge), listener, and proxy much like the ones we just created for the fibonacci service. The first step to configure the Edge to fibonacci service is to create a cluster to tell it where to find the fibonacci sidecar.

NOTE that there are several differences between this cluster and the created above:

1. The instances field is left as an empty string, whereas the fibonacci-local-cluster instances were configured. This is because Grey Matter Control will discover the fibonacci deployment and the instances array will be automatically populated from this service discovery: the instances will go up and down whenever the service scales or changes. To do this, (in the same way as described in creating the object above) the name field must match the cluster label on the deployment.

2. This cluster has a secret set on it, and

Save this file as edge-to-fibonacci-cluster.json and apply it with

Edge to Fibonacci Shared Rules

The edge to fibonacci will be used to link the to the edge-to-fibonacci-cluster .

Save this file as edge-to-fibonacci-rules.json and apply it with

Edge to Fibonacci Routes

There are two that need to be created to send requests through the Edge proxy to the fibonacci service. In the same way that the was connected to the fibonacci-domain, these routes will be connected to the edge domain, and will configure how the edge sidecar sends routes.

They are nearly identical, but one provides for the case that the path on the request to send to the fibonacci service ends in a slash, and one for the case that it doesn't.

This route looks for the path on a request into the edge proxy /services/fibonacci with no trailing slash, replaces that path with /, and sends the request to the cluster (edge-to-fibonacci-cluster) being pointed to by the edge-to-fibonacci-rules.

This route looks for the path on a request into the edge proxy /services/fibonacci/ this time with a trailing slash, replaces that path with /, and sends the request to the same cluster.

Save these files as edge-to-fibonacci-route.json and edge-to-fibonacci-route-slash.json and apply them with

Once these routes are applied, the service is fully configured in the mesh! You should be able to access the service at https://{your-gm-ingress-url}:{your-gm-ingress-port}/services/fibonacci/ with response Alive. To send a request for a specific fibonacci number, https:///{your-gm-ingress-url}:{your-gm-ingress-port}/services/fibonacci/fibonacci/<number>.

If you don't know your gm ingress url and you followed the guide, run

and copy the EXTERNAL-IP and port (by default the port will be 10808).

4. Catalog

The last step in deploying a service is to add the expected service entry to the Grey Matter Catalog service. This will interface with the control plane, and provide information to the Intelligence 360 Application for display.

Save this file as fibonacci-catalog.json and using the quickstart certificates from the helm chart repository at ./certs, make the following post request to the catalog service:

You'll see the following response if the addition was successful.

And the service will display in the Intelligence 360 Application.

Data

Grey Matter Data is a microservice for the versioned and encrypted storage of media blobs and assets. It is a high-performance time-series object storage. It contains an immutable sequence of events which collectively describes a file system at a given moment in time. If the first event is a creation of an object and the second is its deletion, then the object will not appear in listings as of the current time, but it will appear as of any time after the creation and before the deletion.

Questions?

Because the Sidecar admin interface can contain sensitive information and allows users to manipulate the sidecar, access is not routed through the mesh. In a Grey Matter deployment, access is locked down and only accessible by admin users who have direct access to the pod.

This guide is a step by step walkthrough on how to access a Sidecar's admin interface. This interface is used to both inspect and operate on the running server; doing tasks like inspecting stats or performing a hard shut-down. In this walkthrough, we'll perform the following common tasks:

• Examine the live stats

Prerequisites

1. An existing Grey Matter deployment running on Kubernetes ()

2. kubectl or oc setup with access to the cluster

NOTE, the operations on the admin server do not depend on the platform in use. Since this guide is focused on Kubernetes, we'll use kubectl for access to the server. On other platforms, tools like ssh sessions can be used instead.

Steps

For all examples shown here, the Sidecar's admin server has been started on port 8001. This is the default for Grey Matter deployments.

1. Establish a Session

The first step is to establish a session inside the pod we need to examine. For this walkthrough, lets look at the edge service. To do this, we'll need the POD ID for any edge nodes in the deployment:

We can see we have one edge pod with an ID of edge-7d7bf848b9-xjs5l. We'll now use the kubectl tool to start a shell inside that pod. After running the command below, you'll be left off with a shell inside the container.

Now that we're in the pod, we can run commands against the admin interface. Here we'll use the curl to send commands since it's widely available and installed by default on Grey Matter docker images.

2. General Admin Commands

To see the full list of endpoints available, send a GET request to the /help endpoint. Full descriptions of each can be found in the .

In the next 3 sections, we'll show sample usage of the /stats and /config_dump endpoints.

2. Stats

The /stats endpoint is used to output all of the collected statistics. The first ~30 lines are shown below, but the full output is much larger and is dependent on the exact configuration of the sidecar being examined. For each new filter, listener, or cluster that is configured, new stats will be output. The stats will also be different depending on each connection type, e.g. or .

To query just a subset of the stats, use the filter query parameter:

or

3. Configuration Dump

Next we'll dump out the entire configuration of the Sidecar. The output of this command is routinely many thousands of lines long, but is very useful to inspect the exact state of a sidecar at any given moment. This config is mostly useful for verifying the exact behavior of the sidecar against what was intended.

5. Exit

Type exit in the terminal to return to drop the connection to the pod and return to your local shell.

The Grey Matter Command Line Interface (CLI) is a configuration tool for Grey Matter. Leveraging the , the CLI mainly performs dynamic configuration of the Fabric mesh.

This is an example of deploying a new service into the mesh and configuring it for ingress and egress actions using SPIFFE/SPIRE identities.

Prerequisites

1. An existing Grey Matter deployment following the guide.

Simple Deployment Model

In the simple deployment model, there is a single point of ingress between the Client and the Grey Matter Edge. The Edge routes traffic to appropriate Grey Matter Sidecars.

Prerequisites

1. An existing Grey Matter deployment running on Kubernetes _()

Envoy allows configuration of how many requests / second a service can field. This is useful in preventing DDOS attacks and otherwise making sure your server's resources aren't easily overrun.

Unlike which are configured on each cluster, rate limiting is configured across multiple listeners or clusters. Overall, circuit breakers are good for avoiding cascading failures of a bad downstream host. However, when:

[A] large number of hosts are forwarding to a small number of hosts and the average request latency is low (e.g., connections/requests to a database server). If the target hosts become backed up, the downstream hosts will overwhelm the upstream cluster. In this scenario it is extremely difficult to configure a tight enough circuit breaking limit on each downstream host such that the system will operate normally during typical request patterns but still prevent cascading failure when the system starts to fail. Global rate limiting is a good solution for this case.

This is because rate limiting sets a global number of requests / second across a service, which is independent of the number of instances configured for a cluster. This guide shows how to configure rate limiting on the edge node in a Grey Matter deployment to an

This guide will help you set up a secure, zero-trust environment in Grey Matter to achieve the following:

• Establish trust in user identities

• Enforce adaptive and risk-based policies

Prerequisites

1. An existing Grey Matter deployment.

Learn how to configure the Grey Matter Sidecar in Fabric to achieve the following:

• Dynamic configuration for adaptive proxying

• Service discovery through the Grey Matter Control API

There are four main filters that can be used to construct an Open ID Connect (OIDC) filter chain:

• • This filter checks the existence of a request attribute (header, cookie, etc.) and will copy/move it to the next filter in the chain.