# Structure Grey Matter Data tracks information through a collection of **Events** that represent a Kafka messaging queue. Each Event is associated to a folder or file in the system and is used to capture all historical changes to it. New Events form a timeline of actions describing state modifications to a folder or file (e.g. create, update, delete). The primary key for each Event in the system is identified by two timestamps: 1. An Object ID (`oid`) which is a nanosecond timestamp assigned at creation of the folder or file. 2. A `tstamp` which is recorded for each new Event belonging to the folder or file. Events may also be used to define relationships between folders and files in two ways: 1. An Event's `parentoid` field defines folder-to-folder and file-to-folder relationships. For example, if the system receives an update `action` (`U`) that changes the `parentoid` for a given file's `oid`, the file will be "moved" from one folder to another. 2. An Event's `derived` field specifies a folder or file came from another. For example, a thumbnail of an image or a PDF copy of a Word document could point back to its original by assigning the original's `oid` to its `derived` field. ## Event | Field | Data Type | Description | | --------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------- | | `action` | string | Describes a modification to an object's state | | `blobalgorithm` | string | The blob algorithm. If not specified, we store in S3 with SSECustomerKey (?) | | `checkedtstamp` | string | The `tstamp` of the previous version of the `oid` compared against (?) | | `custom` | JSON (all string values) | Custom metadata fields | | `defaultfile` | string | If the object is a folder, a `Stream` request will return this file; defaults to `index.html` | | `derived` | [Derived](#derived) | See above | | `description` | string | Describes the contents of the object | | `encrypted` | (?) | Allow for fields to be encrypted on a case-by-case basis | | `expiration` | string | The `tstamp` as of which this object and its Events are no longer accessible | | `isfile` | string | Specifies that this object is a file | | `mimetype` | string | Same as Content-Type | | `name` | string | The object's folder name or filename without pathing | | `objectpolicy` | string | Lisp-syntax rules that permit various actions to a given owner | | `oid` | string | Numeric identifier for the object | | `parentoid` | string | Numeric identifier for the parent folder of the object | | `purgetstamp` | string | If set, purges an Event with the matching `oid` and `tstamp` | | `references` | \[]Reference (?) | An array of updates when uploading files into folders that don't yet exist. Indices are negative relative values (?) | | `rname` | string | Used to define where this object is stored on disk (via ${FILE\_BUCKET}/${FILE\_PARTITION}/${rname}) | | `schema` | string | Version of the schema from which this object came | | `security` | [Security](#security) | A security label to assign to the object | | `sha256plain` | string | sha256 of the plaintext; can be used in the client to calculate a minimum number of files to send for update | | `size` | string | Same as Content-Length | | `tstamp` | string | Timestamp unique to this Event | | `userpolicy` | [UserPolicy](#userpolicy) | The JWT claims used when creating this Event | ### Derived | Field | Data Type | Description | | -------- | --------- | ----------------------------------------------- | | `oid` | string | Numeric identifier for the object | | `tstamp` | string | Numeric identifier for the Event | | `type` | string | Description of the derivation (e.g. doc-to-pdf) | ### Security | Field | Data Type | Description | | ------------ | --------- | ----------------------------------------------------- | | `label` | string | Title of the security label (e.g. `DECIPHER//GMDATA`) | | `foreground` | string | The text color (e.g. `#FFFFFF`) | | `background` | string | The background color (e.g. `#00FF00`) | ### UserPolicy | Field | Data Type | Required | Description | | -------- | ----------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `label` | string | yes | Title of the UserPolicy | | `exp` | number | yes | Expiry in seconds | | `sub` | string | | Subject | | `iss` | string | | Issuer | | `aud` | string | | Audience | | `values` | JSON (array of strings) | | [Hashtable for evaluating JWT tokens](https://greymatter.gitbook.io/grey-matter-documentation/1.7-beta/usage/platform-services/data/internals/auth) | ## Sample Events ### Create The parameters below are the bare minimum that must be specified for the create action to be successful. When the action specified is `C` (create / upload), the system will backfill the `oid`. ```javascript { "action": "C", "name": "New Folder", "isfile": false, "parentoid": 1, "userpolicy": { "label": "asRoot" }, "objectpolicy": "(if (contains email rob@foo.com)(yield-all)(yield R X))", "userpolicy": { "label": "asRob", "values": { "email": ["rob@foo.com"] } } } ``` ### Delete When the action specified is `D` (delete), the system will backfill most of the parameters. It is only necessary to specify `oid`, `action`, `parentoid`, and `objectpolicy`. ```javascript { "action": "D", "oid": "42", "parentoid": 1, "objectpolicy": "(if (contains email rob@foo.com)(yield-all)(yield R X))" } ``` ### Update **Important:** Anytime the action specified is `U` (update), all parameters except `tstamp` must be specified, mimicking the previous Event. In the following examples, assume all other parameters are specified. #### Renaming a file ```javascript { ... "action": "U", "oid": "42", "name": "new-filename.txt" } ``` #### Moving a file ```javascript { ... "action": "U", "oid": "42", "parentoid": 2 } ```