# Structure

Grey Matter Data tracks information through a collection of **Events** that represent a Kafka messaging queue. Each Event is associated to a folder or file in the system and is used to capture all historical changes to it. New Events form a timeline of actions describing state modifications to a folder or file (e.g. create, update, delete).

The primary key for each Event in the system is identified by two timestamps:

1. An Object ID (`oid`) which is a nanosecond timestamp assigned at creation of the folder or file.
2. A `tstamp` which is recorded for each new Event belonging to the folder or file.

Events may also be used to define relationships between folders and files in two ways:

1. An Event's `parentoid` field defines folder-to-folder and file-to-folder relationships. For example, if the system receives an update `action` (`U`) that changes the `parentoid` for a given file's `oid`, the file will be "moved" from one folder to another.
2. An Event's `derived` field specifies a folder or file came from another. For example, a thumbnail of an image or a PDF copy of a Word document could point back to its original by assigning the original's `oid` to its `derived` field.

## Event

| Field           | Data Type                 | Description                                                                                                                                                                                                                                            |
| --------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `action`        | string                    | Describes a modification to an object's state                                                                                                                                                                                                          |
| `blobalgorithm` | string                    | The blob algorithm. If not specified, we store in S3 with SSECustomerKey (?)                                                                                                                                                                           |
| `checkedtstamp` | string                    | The `tstamp` of the previous version of the `oid` compared against (?)                                                                                                                                                                                 |
| `custom`        | JSON (all string values)  | Custom metadata fields                                                                                                                                                                                                                                 |
| `defaultfile`   | string                    | If the object is a folder, a [Stream](https://github.com/DecipherNow/gm-gitbook-sync/tree/fdbad23611cffcdb0b6cb5d9fadb49bf146ffcaa/usage/platform-services/data/internals/retrieval.md#stream) request will return this file; defaults to `index.html` |
| `derived`       | [Derived](#derived)       | See above                                                                                                                                                                                                                                              |
| `description`   | string                    | Describes the contents of the object                                                                                                                                                                                                                   |
| `encrypted`     | (?)                       | Allow for fields to be encrypted on a case-by-case basis                                                                                                                                                                                               |
| `expiration`    | string                    | The `tstamp` as of which this object and its Events are no longer accessible                                                                                                                                                                           |
| `isfile`        | string                    | Specifies that this object is a file                                                                                                                                                                                                                   |
| `mimetype`      | string                    | Same as Content-Type                                                                                                                                                                                                                                   |
| `name`          | string                    | The object's folder name or filename without pathing                                                                                                                                                                                                   |
| `objectpolicy`  | string                    | Lisp-syntax rules that permit various actions to a given owner                                                                                                                                                                                         |
| `oid`           | string                    | Numeric identifier for the object                                                                                                                                                                                                                      |
| `parentoid`     | string                    | Numeric identifier for the parent folder of the object                                                                                                                                                                                                 |
| `purgetstamp`   | string                    | If set, purges an Event with the matching `oid` and `tstamp`                                                                                                                                                                                           |
| `references`    | \[]Reference (?)          | An array of updates when uploading files into folders that don't yet exist.  Indices are negative relative values (?)                                                                                                                                  |
| `rname`         | string                    | Used to define where this object is stored on disk (via ${FILE\_BUCKET}/${FILE\_PARTITION}/${rname})                                                                                                                                                   |
| `schema`        | string                    | Version of the schema from which this object came                                                                                                                                                                                                      |
| `security`      | [Security](#security)     | A security label to assign to the object                                                                                                                                                                                                               |
| `sha256plain`   | string                    | sha256 of the plaintext; can be used in the client to calculate a minimum number of files to send for update                                                                                                                                           |
| `size`          | string                    | Same as Content-Length                                                                                                                                                                                                                                 |
| `tstamp`        | string                    | Timestamp unique to this Event                                                                                                                                                                                                                         |
| `userpolicy`    | [UserPolicy](#userpolicy) | The JWT claims used when creating this Event                                                                                                                                                                                                           |

### Derived

| Field    | Data Type | Description                                     |
| -------- | --------- | ----------------------------------------------- |
| `oid`    | string    | Numeric identifier for the object               |
| `tstamp` | string    | Numeric identifier for the Event                |
| `type`   | string    | Description of the derivation (e.g. doc-to-pdf) |

### Security

| Field        | Data Type | Description                                           |
| ------------ | --------- | ----------------------------------------------------- |
| `label`      | string    | Title of the security label (e.g. `DECIPHER//GMDATA`) |
| `foreground` | string    | The text color (e.g. `#FFFFFF`)                       |
| `background` | string    | The background color (e.g. `#00FF00`)                 |

### UserPolicy

| Field    | Data Type               | Required | Description                                                                                                                                                                                  |
| -------- | ----------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `label`  | string                  | yes      | Title of the UserPolicy                                                                                                                                                                      |
| `exp`    | number                  | yes      | Expiry in seconds                                                                                                                                                                            |
| `sub`    | string                  |          | Subject                                                                                                                                                                                      |
| `iss`    | string                  |          | Issuer                                                                                                                                                                                       |
| `aud`    | string                  |          | Audience                                                                                                                                                                                     |
| `values` | JSON (array of strings) |          | [Hashtable for evaluating JWT tokens](https://github.com/DecipherNow/gm-gitbook-sync/tree/fdbad23611cffcdb0b6cb5d9fadb49bf146ffcaa/usage/platform-services/data/internals/authentication.md) |

## Sample Events

For API usage, refer to the [Data Modification](https://github.com/DecipherNow/gm-gitbook-sync/tree/fdbad23611cffcdb0b6cb5d9fadb49bf146ffcaa/usage/platform-services/data/internals/modification.md) section. These sample Events are only meant to be instructive.

### Create

The parameters below are the bare minimum that must be specified for the create action to be successful.

When the action specified is `C` (create / upload), the system will backfill the `oid`.

```javascript
{
  "action": "C",
  "name": "New Folder",
  "isfile": false,
  "parentoid": 1,
  "userpolicy": {
    "label": "asRoot"
  },
  "objectpolicy": "(if (contains email rob@foo.com)(yield-all)(yield R X))",
  "userpolicy": {
    "label": "asRob",
    "values": {
      "email": ["rob@foo.com"]
    }
  }
}
```

### Delete

When the action specified is `D` (delete), the system will backfill most of the parameters. It is only necessary to specify `oid`, `action`, `parentoid`, and `objectpolicy`.

```javascript
{
  "action": "D",
  "oid": "42",
  "parentoid": 1,
  "objectpolicy": "(if (contains email rob@foo.com)(yield-all)(yield R X))"
}
```

### Update

**Important:** Anytime the action specified is `U` (update), all parameters except `tstamp` must be specified, mimicking the previous Event. In the following examples, assume all other parameters are specified.

#### Renaming a file

```javascript
{
  ...
  "action": "U",
  "oid": "42",
  "name": "new-filename.txt"
}
```

#### Moving a file

```javascript
{
  ...
  "action": "U",
  "oid": "42",
  "parentoid": 2
}
```
