Audits and Observables

Overview of how Grey Matter handles audits and observables.

Grey Matter Fabric helps you visualize and analyze audit data. As long as you deploy the Grey Matter Sidecar with a service, the Sidecar will send metrics and audit data to Fabric.

Key Definition

Audits are a security-relevant event within Grey Matter. An audit event, or simply event, can be any of the following:

• Change to the security state of the system

• Attempted or actual violation of the system access control or accountability security policies

• Both

An audit event report includes the following information:

• Name of the event

• Success or failure of the event

• Additional event-specific information that is related to security auditing

How Do Audits Work in Grey Matter?

1. The Grey Matter Sidecar emits audit data to a Kafka topic for easy observability.

2. If Fabric is set up with an Edge, it pulls audit data from the PKI certificate, the IP address of the originating request, etc.

3. This audit data--also called events or observables--allows for detailed event auditing of ingress and egress traffic, and process resource use.

Learn About the Grey Matter Sidecar

Learn more about the process and capabilities of the Grey Matter Sidecar here.

Configure an Observables Filter

Learn how to set up an observables filter here.

Configure Audits

Visualize Observables

Learn how to use Grey Matter to visualize observables here.

Visualize Audits

How Does Grey Matter Index Audit Events?

Grey Matter does not index audit events directly into Elasticsearch. Instead, Grey Matter contains a Kafka consumer that reads Kafka observables. This consumer transforms and indexes them to use with Elasticsearch.

Use Kibana to Visualize Observables

Kibana is an open source Elasticsearch plugin that takes observables from Grey Matter and visualizes them in a graphical dashboard.

Kibana simplifies the creation of visualizations to explore, search, view, and interact with audit data stored in Elasticsearch indices. Kibana helps you analyze and visualize individual events and trends such as:

• Total requests

• Number of requests by individual users

• Geographic locations of requests made in Fabric

• What individual users are doing

• Timing of user requests

• What user are looking at

• userDNs (Authenticated user names)

• Geographic location of IP addresses

• Requests per hour by user

• Response codes

• Paths

• Service vs. userDN

• Services

• Response bodies

• User agents

Enable Audits to Be Ingested into Elasticsearch with Kibana

To enable audits to be ingested into Elasticsearch with Kibana, follow these steps:

1. Configure audits: this guide helps you gather observables.
2. Set up the Audit Proxy Observable Consumer (APOC) code: this guide helps visualize observables.

What about Splunk?

While the Grey Matter Sidecar does not support direct emission of events into Splunk, you can create or modify a consumer to provide that capability. Learn more.

Sample Observable Information

The following observable information was captured from a user accessing an event through a Sidecar operating within Grey Matter Fabric:

{
 "_index": "audit",
 "_type": "_doc",
 "_id": "FvUJ2GsBQetsYfWuW1Ab",
 "_score": 1,
 "_source": {
   "eventId": "00f4b3e4-a279-11e9-b433-0a580a82025d",
   "eventChain": [
     "00f4b3e4-a279-11e9-b433-0a580a82025d"
   ],
   "schemaVersion": "1.0",
   "originatorToken": [
     "cn=minos.kepheus, dc=hellas, dc=com",
     "CN=*.greymatter.svc.cluster.local,OU=Engineering,O=Decipher Technology Studios,=Alexandria,=Virginia,C=US",
     "CN=*.greymatter.svc.cluster.local,OU=Engineering,O=Decipher Technology Studios,=Alexandria,=Virginia,C=US"
   ],
   "eventType": "fibonacci",
   "timestamp": 1562697620,
   "xForwardedForIp": "15.188.27.135,10.129.2.140",
   "systemIp": "10.130.2.93",
   "action": "GET",
   "payload": {
     "isSuccessful": true,
     "request": {
       "endpoint": "/fibonacci/18",
       "headers": {
         ":authority": "demo-oauth.production.deciphernow.com",
         ":method": "GET",
         ":path": "/fibonacci/18",
         "accept-encoding": "gzip",
         "content-length": "0",
         "cookie": "OauthExpires=1562757619; OauthSignature=0OgHLzHBxSUdNk557aKWeYW9jrg; OauthUserDN=cn%3Dminos.kepheus%2C+dc%3Dhellas%2C+dc%3Dcom",
         "external_sys_dn": "CN=*.greymatter.svc.cluster.local,OU=Engineering,O=Decipher Technology Studios,=Alexandria,=Virginia,C=US",
         "forwarded": "for=15.188.27.135;host=demo-oauth.production.deciphernow.com;proto=https;proto-version=",
         "ssl_client_s_dn": "CN=*.greymatter.svc.cluster.local,OU=Engineering,O=Decipher Technology Studios,=Alexandria,=Virginia,C=US",
         "user-agent": "Go-http-client/1.1",
         "user_dn": "cn=minos.kepheus, dc=hellas, dc=com",
         "x-envoy-original-path": "/services/fibonacci/1.0.0/fibonacci/18",
         "x-forwarded-for": "15.188.27.135,10.129.2.140",
         "x-forwarded-host": "demo-oauth.production.deciphernow.com",
         "x-forwarded-port": "443",
         "x-forwarded-proto": "https",
         "x-real-ip": "10.129.2.140",
         "x-request-id": "234aff6f-e376-41d5-89b8-1aa6dd0bbf4f"
       }
     },
     "response": {
       "code": 200,
       "headers": {
         ":status": "200",
         "content-length": "5",
         "content-type": "text/plain; charset=utf-8",
         "date": "Tue, 09 Jul 2019 18:40:20 GMT",
         "x-envoy-upstream-service-time": "0"
       },
       "body": "2584\n"
     }
   },
   "event_mapping": {
     "type": "EventAccess",
     "action": "ACCESS"
   },
   "time_audited": "20190709T184020.249380",
   "geo_ip": {
     "accuracy_radius": 1000,
     "latitude": 48.8607,
     "longitude": 2.3281,
     "time_zone": "Europe/Paris"
   },
   "location": {
     "lat": 48.8607,
     "lon": 2.3281
   }
 },
 "fields": {
   "payload.response.headers.date": [
     "2019-07-09T18:40:20.000Z"
   ],
   "time_audited": [
     "2019-07-09T18:40:20.249Z"
   ]
 }
}

Questions?

Create an account at Grey Matter Support to reach our team.