# Data Grey Matter Data is a platform service for storing encrypted and security labeled file attachments. Its main functionalities are: * write files (in batch) with permissions (in the form of logic on groups) and security labels * read files back (in the context of access privilege, encoded into JWT claims) Gm-data can exist as a singleton but also has optional dependencies, depending on the use case: * database (MongoDB) for storing metadata, including tree structure and file attributes * blob storage (AWS S3, or disk) for storing ciphertext blobs for files * permissions (JWT), where USER\_DN is translated into a JSON Web Token (JWT) token for its signed privilege claims * message queue (Kafka), for where database records are duplicated for replication purposes; and for an audit log when files are accessed It makes it easier to understand how it is setup by looking at the smallest possible standalone docker-compose of a running Grey Matter Data instance: ```yaml version: '2' services: gmdata-standalone: networks: mesh: aliases: - gmdata-standalone image: docker.greymatter.io/development/gm-data:latest ports: - "8181:8181" depends_on: - mongo-ephemeral - jwt-standalone environment: - SLEEP_BEFORE_START=5 - CLIENT_JWT_ENDPOINT_ADDRESS=jwt-standalone - CLIENT_JWT_ENDPOINT_PORT=8080 - CLIENT_JWT_ENDPOINT_PREFIX=/ - CLIENT_PREFIX= - CLIENT_USE_TLS=false - CLIENT_HOST=localhost - CLIENT_PORT=8181 - GMDATA_NAMESPACE=world - GMDATA_NAMESPACE_USERFIELD=email - JWT_API_KEY=Zm9vCg== - JWT_PUB=LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlHYk1CQUdCeXFHU000OUFnRUdCU3VCQkFBakE0R0dBQVFCSUVrNGpHL1FEMkZwNURxcmMrTzFPVy9CaG1BLwpKcmgyRFRaRWpybEZONnJYbTA0Vms0bUluNENZSmJ0VDdIQjc2cVJIeE9DNTFORVk0eFZHb1RUUVZta0Fnc3ljCllVdEdqZ3pKQUdTZExsSXZKSmtabWkrSjZBbWVtNng5UFZkajcxc2hHSnhNdjM4SDFTa2RRS29EZmllS3dZbFIKNENPUWFxdXdJVEpPYWd2R1VUQT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg== - TEST_JWT_PRIV=LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1JSGNBZ0VCQkVJQUludTVqbEdaU2Z6M2dMOWczOVNoeE5UaWtUQ0lvUWtFVVUyckdnV1ZJTm9TN3RlM2d1Uk8KK2VKb3FDZTd0Z3pKL1RZcGVxU055elE2UGVTS2JmZC91K2FnQndZRks0RUVBQ09oZ1lrRGdZWUFCQUVnU1RpTQpiOUFQWVdua09xdHo0N1U1YjhHR1lEOG11SFlOTmtTT3VVVTNxdGViVGhXVGlZaWZnSmdsdTFQc2NIdnFwRWZFCjRMblUwUmpqRlVhaE5OQldhUUNDekp4aFMwYU9ETWtBWkowdVVpOGttUm1hTDRub0NaNmJySDA5VjJQdld5RVkKbkV5L2Z3ZlZLUjFBcWdOK0o0ckJpVkhnSTVCcXE3QWhNazVxQzhaUk1BPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo= - MASTERKEY=fark - FILE_BUCKET=decipherers - FILE_PARTITION=gmdatax - USES3=false - MONGOHOST=mongo-ephemeral - MONGODB=chili volumes: - ./privacy.html:/static/privacy.html mongo-ephemeral: networks: - mesh image: mongo jwt-standalone: networks: - mesh image: nexus.greymatter.io/gm-jwt-security:1.1.1 environment: - JWT_PUB=LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlHYk1CQUdCeXFHU000OUFnRUdCU3VCQkFBakE0R0dBQVFCSUVrNGpHL1FEMkZwNURxcmMrTzFPVy9CaG1BLwpKcmgyRFRaRWpybEZONnJYbTA0Vms0bUluNENZSmJ0VDdIQjc2cVJIeE9DNTFORVk0eFZHb1RUUVZta0Fnc3ljCllVdEdqZ3pKQUdTZExsSXZKSmtabWkrSjZBbWVtNng5UFZkajcxc2hHSnhNdjM4SDFTa2RRS29EZmllS3dZbFIKNENPUWFxdXdJVEpPYWd2R1VUQT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg== - PRIVATE_KEY=LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1JSGNBZ0VCQkVJQUludTVqbEdaU2Z6M2dMOWczOVNoeE5UaWtUQ0lvUWtFVVUyckdnV1ZJTm9TN3RlM2d1Uk8KK2VKb3FDZTd0Z3pKL1RZcGVxU055elE2UGVTS2JmZC91K2FnQndZRks0RUVBQ09oZ1lrRGdZWUFCQUVnU1RpTQpiOUFQWVdua09xdHo0N1U1YjhHR1lEOG11SFlOTmtTT3VVVTNxdGViVGhXVGlZaWZnSmdsdTFQc2NIdnFwRWZFCjRMblUwUmpqRlVhaE5OQldhUUNDekp4aFMwYU9ETWtBWkowdVVpOGttUm1hTDRub0NaNmJySDA5VjJQdld5RVkKbkV5L2Z3ZlZLUjFBcWdOK0o0ckJpVkhnSTVCcXE3QWhNazVxQzhaUk1BPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo= - TOKEN_EXP_TIME=30000 - REDIS_HOST= - ENABLE_TLS=false - ZEROLOG_LEVEL=debug - JWT_API_KEY=Zm9vCg== ports: - "8480:8080" volumes: - ./users.json:/gm-jwt-security/etc/users.json networks: mesh: {} ``` In this setup: * gmdata-standalone points directly to a JWT server, jwt-standalone, so that the USER\_DN header can look up a JWT token. Alternatively, a userpolicy cookie can be set with this JWT in the browser; so that no lookup is required. * CLIENT\_JWT\_ENDPOINT vars are setup to talk plaintext to jwt-standalone, to turn a USER\_DN into JWT claims. JWT\_API\_KEY is a shared password that allows the USER\_DN to be turned into a JWT; a privileged operation only allowed behind the mesh. * gmdata-standalone talks to mongo-ephemeral, which is a simplified MongoDB that is not persisting to disk. * gmdata-standalone does not use S3 for storage, and is not mounting persistence for its blobs. * Setup the signing keypair JWT\_PUB/TEST\_JWT\_PRIV so that Grey Mater Data can check the signatures on JWT and we can also use this container to generate JWTs to set on userpolicy cookie if we want to. * CLIENT env vars are just here to facilitate connecting /static/gmdatatool.linux binary to Grey Matter Data for administrative tasks such as creating a JWT. * GMDATA\_NAMESPACE sets how the system is configured, which you can validate with `GET /config`. We want to have home directories fit the pattern `/world/${email}`, so that users can create their own home directory to write files into; without involving an administrator to set them up. * The jwt-standalone server is configured to issue JWTs that expire in 3000 seconds and will stop caching them in half that time (1500 seconds) to ensure that tokens always have enough time to be actually used when we are given a token. * Redis is disabled so jwt-standalone needs no Redis for persistence * JWT\_API\_KEY is a shared password with gmdata-standalone to allow use of privileged /policies endpoint to convert a USER\_DN into a JWT. * The jwt-standalone has a user database in users.json file. The entries in this database are the JWT claims that get signed (with privileges removed from entries by default). When a call to gmdata-standalone has a USER\_DN set, it looks into this data structure in users.json to match up its `label` field with `USER_DN`, to create a JWT with this label and values field. {% hint style="info" %} The `privilege` field is removed by default, as this is used in a sudo mechanism to allow for privilege escalation when needed. {% endhint %} > See [usage](https://greymatter.gitbook.io/grey-matter-documentation/1.3/usage/platform-services/data#common-get-requests) for how to form a request to upload and download files. It is critically important that you understand the policy language before trying to upload files; because there is a permission system that requires that either the UI or the User can express the desired permissions. ## Environment Variables ### Environment and Deployment Grey Matter Data creates a binary called `gmdatax.linux`, that is configured entirely by environment variables (to avoid a requirement to mount files). This binary however is packaged with some other files. * `./runforever` - A shell script that keeps `./gmdatax.linux` in a re-start loop to handle non-intentional crashes of the binary. This allows us to catch things like array out of bounds, nil pointer de-reference, or catastrophic resource exhaustion such as out of file handles. It is these latter cases that drive the decision to allow the binary to die. * `./gmdatax.linux` - The actual binary that reads in environment variables * `./VERSION` - The version of this service * `./static/` - A bundle of runtime API user documentation, and test user interface. This directory is literally served out of the service from the `/static/` endpoint. * `./certs/` - A directory that the binary can write certificates into on startup. The certificates originate from environment variables passed in as single-line base64 encoding full `pem` files. * `./logs/` - A place to write logs (in non-default cases), and may be mounted over to keep the root partition from running out of space. {% hint style="info" %} Grey Matter Data will make every possible attempt to look at your configuration and immediately crash with a detailed explanation of what to actually do about it. This includes looking up hostnames in DNS to verify that they exist. Always look in the log files for the service if something does not seem right on startup. It cannot, however, detect inconsistency issues at a higher level, such as one service offering a cert that is then trusted by a service that will try to connect to it. That would require analyzing a larger set of environment variables that are destined for multiple services. {% endhint %} #### Basic Environment Variables * `MASTERKEY` is mandatory. This is the key that is used to encrypt data. * `JWT_PUB` is the single-line base64 encode of the signing key that the Grey Matter Data server trusts to sign JWT tokens. This is a mandatory parameter. It is not an X509 certificate. It is an actual Elliptic Curve key that is suitable for `ES512` in the JWT standard. * `FILE_BUCKET` is mandatory (aka: `AWS_S3_BUCKET`). This is where we write Grey Matter Data ciphertext out to AWS. * `FILE_PARTITION` is mandatory (aka: `AWS_S3_PARTITION`). This should be set to a value that is unique to a set of replicated Fabric clusters. It is literally a subdirectory in `FILE_BUCKET`. This exists so that we don't need to create lots of buckets constantly, yet can still distinguish which bucket data belongs to which installation. * `AWS_REGION` is required if `USES3=true`. * `AWS_S3_ENDPOINT` is only required in government setups that need to point to a different hostname for S3. * `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` may be set to give AWS credentials in the case where IAM roles are not used for the EC2 instance. `AWS_SECRET_ACCESS_KEY` is a secret, obviously. {% hint style="info" %} When you disable S3 usage via `USES3=false`, the local bucket and partition are still used. The directory `./buckets/${FILE_BUCKET}/${FILE_PARTITION}` should exist and be writable by the Grey Matter Data process `./gmdatax.linux`. {% endhint %} The `JWT_PUB` is the public part of an elliptic curve key. The private part of it is `PRIVATE_KEY` for the JWT server. The parameters for use with the JWT libraries are rather specific, due to the curve name `secp521r1`. This is how we generate our keypairs, which is done specifically for `gm-jwt-security` to get a private key for signing (file `jwtES512.key`), and then the public key derived from that (`jwtES512.key.pub`) and set for Grey Matter Data as `JWT_PUB`. ``` openssl ecparam -genkey -name secp521r1 -noout -out jwtES512.key openssl ec -in jwtES512.key -pubout -out jwtES512.key.pub ``` #### References Between services When Grey Matter Data needs to make reference to another service, these are the relevant environment variables: * `CLIENT_PREFIX` is the URL that the gateway is mapping the Grey Matter Data service to. This is done so that we can send back links that resolve properly in HTML files. We do this because we cannot hardcode even our own service name, and also cannot correctly give a relative path e.g. `/services/gmdatax/latest`. * `CLIENT_JWT_PREFIX` is the URL that the gateway is mapping our peer service gm-jwt-security to. This is done so that we can send back links that resolve properly in HTML files e.g. `/services/jwt-server/1.0` or `/services/jwt-server-gov/1.0`. We have explicit dependencies on these things: * A JWT token issuer that has a proper sidecar and is reachable through the edge. * A Mongo database that is not mounted into the Fabric framework; so is not reached via a sidecar or through the edge. * A Kafka deployment that is not mounted into the Fabric framework; so is not reached via a sidecar or through the edge. #### TLS Services that use TLS will end up creating a large number of environment variables. We follow a principle of passing in pem files as a single line of base64 of the original pem file. That means that we create such files as environment variables on the host that is preparing the deployment. Here is an example of setting up the trust for our Mongo dependency: ``` MONGO_TRUST=`cat server.trust.pem | base64 -w 0` ``` When TLS connections to peer services are involved, this pattern in name suffixes arises: * `ADDRESS` (or `HOST`) - IP or hostname of the peer * `PORT` - port for the peer * `USE_TLS` - use TLS * `CERT` - A base64 single-line encode of the pem cert (which also happens to be multi-line base64). * `KEY` - base64 single-line encode of the pem key (which is a multi-line base64). This is also a secret. * `TRUST` - This is similar to `CERT`. It may encode a concatenated list of pem files for certs. * `CN` - The ServerName expected. This is usually the same as the CN in the remote cert, but may also be an SNI name that matches a wildcard in the CN. If this is not set, then we will contact the server to try to grab the CN out of the remote certificate. With that being said, these variables are grouped together. * MONGO related connect info * `MONGOHOST` - Slightly violates our pattern. This can be a list of host:port pairs, like `mongodata:27017,mongodata:27017`. This is because in a clustered setting, connections are not made to individual machines, but to entire clusters. The `PORT` part is already taken care of. * `MONGODB` - Is not strictly part of TLS, but we need to know the database that we are connecting to. * `MONGO_USE_TLS` - Says whether to use the TLS variables to make a TLS connection. * `MONGO_CERT` - Is the client PKI cert that we identify ourselves with. * `MONGO_KEY` - Is the key that goes with `MONGO_CERT`. * `MONGO_TRUST` - Is the trust file to connect to Mongo servers. * `MONGO_CN` - Is the SNI name for the Mongo cert with the manually set ServerName expected. If this is not set, then we will contact the server to try to grab the CN out of the remote certificate. * `MONGO_INITDB_ROOT_USERNAME` - Is the username we will use (not necessarily related to the root username however). * `MONGO_INITDB_ROOT_PASSWORD` - The password for `MONGO_INITDB_ROOT_USERNAME`. This is a secret of course. * TLS info for the Grey Matter Data service. This generally only happens when the sidecar egress is mTLS. * `GMDATA_USE_TLS` - Says whether to use TLS. This will need to be coordinated with how our sidecar is setup. Our sidecar EGRESS will need to be a client of this TLS connection. * `GMDATA_CERT` - The identity cert of the Grey Matter Data service that will be presented to sidecar. * `GMDATA_KEY` - The key that goes with `GMDATA_CERT`. * `GMDATA_TRUST` - The sidecar will need to present a cert that is signed by something in this TRUST. * Client JWT environment variables are relevant to Grey Matter Data looking up `userpolicyid` (a random key to find a JWT) to get a `userpolicy` (an actual JWT token). This is only needed in cases where we have a JWT server indirectly via `userpolicyid`. * `CLIENT_JWT_ENDPOINT_ADDRESS` - is the hostname of the JWT server * `CLIENT_JWT_ENDPOINT_PORT` - is the port of the JWT server * `CLIENT_JWT_ENDPOINT_USE_TLS` * `CLIENT_JWT_ENDPOINT_CERT` * `CLIENT_JWT_ENDPOINT_KEY` * `CLIENT_JWT_ENDPOINT_CN` - expected SNI name * `CLIENT_JWT_ENDPOINT_TRUST` * `CLIENT_JWT_ENDPOINT_PREFIX` - If we connect directly to the service or to the sidecar, then this is just left as an empty string. But if we go through the edge, which is an unlikely case, this ends up needing to be set to the same value as `CLIENT_JWT_PREFIX`. * `JWT_API_KEY` - is a base64 password that the JWT server will require to accept connections to resolve access codes for JWT tokens (`userpolicyid`) to actual JWT tokens (`userpolicy`). > Note that for the JWT server, we are trying to form a connection URL like: ``` # proto is either HTTP or HTTPS depending on CLIENT_JWT_ENDPOINT_USE_TLS # cert setup is the normal pattern: # CLIENT_JWT_ENDPOINT_CERT # CLIENT_JWT_ENDPOINT_KEY # CLIENT_JWT_ENDPOINT_TRUST GET ${proto}://${CLIENT_JWT_ENDPOINT_ADDRESS}:${CLIENT_JWT_ENDPOINT_PORT}${CLIENT_JWT_ENDPOINT_PREFIX}/policies ``` Internally, Grey Matter Data sees a `userpolicyid` header and connects to that URL to try to get a `userpolicy` object, which may be too large to have fit into an HTTP header. Notice that the inclusion of `CLIENT_JWT_ENDPOINT_PREFIX` exists only to go through the edge instead of the sidecar. In the normal case `CLIENT_JWT_ENDPOINT_PREFIX` is an empty string because we want to talk to the sidecar. Examples: * Talk to our own local sidecar in plaintext to reach JWT (preferred): * `CLIENT_JWT_ENDPOINT_PREFIX=/services/jwt-server/latest` * `CLIENT_JWT_ENDPOINT_ADDRESS=gmdata-proxy` * `CLIENT_JWT_ENDPOINT_PORT=8080` * Talk to a JWT sidecar directly (not preferred): * `CLIENT_JWT_ENDPOINT_PREFIX=` * `CLIENT_JWT_ENDPOINT_ADDRESS=jwt-server-proxy` * `CLIENT_JWT_ENDPOINT_PORT=8080` {% hint style="info" %} `CLIENT_JWT_ENDPOINT_USE_TLS` may require connecting to a sidecar-issued cert, that may not exist at the time Grey Matter Data launches. So, note that using `GMDATA_USE_TLS` in the mesh may be complicated by this fact. {% endhint %} #### Miscellaneous parameters * `DONT_PANIC` - Is an advanced parameter that says to only WARN, but do not CRASH when inconsistent environment variables are detected. If you run with this setting, you run the risk of creating a setup that we cannot support. Sometimes you need to temporarily ignore known problems. So, this should be disabled as soon as possible, if it is ever used. * `LESS_CHATTY_INFO` - By default, we like less chatty logs. If you want a lot more logging information that includes the start and end of sessions in which there were no problems, then you can set this to `false`. * `GMDATAX_SESSION_MAX` - Is an admission control value. This imposes a limit on the number of outstanding requests Grey Matter Data will allow to be concurrently serviced. It is literally a maximum population at which Grey Matter Data just issues `503` to tell the client to get out of line and come back later. It exists because if we run out of filehandles, the server will become unstable and crash in an irregular manner. If this server runs out of filehandles, then `GMDATAX_SESSION_MAX` should be lowered to a value that causes us to stop running out of filehandles. It may need to be raised if we get `503` errors that actually originate from Grey Matter Data itself. Our proxy may also issue `503` in the case of admission control, which complicated determining which one ran out. It is more likely that Envoy will run out of filehandles before Grey Matter Data will, because the front-end is dealing with a lot of services concurrently. * `GMDATA_NAMESPACE` - Typical value is `world`. In order to avoid having to create root access tokens to get the system bootstrapped. We allow for the creation of a self-service directory. If this value is `/world` then the home directory can be created here, on the condition that the directory is named after the field mentioned in `GMDATA_NAMESPACE_USERFIELD`, which is typically `email`. For example: `/world` is created empty on initialization of Grey Matter Data. User uses `static/ui` to create directory `/world/rob.johnson@email.com`, which is only allowed because he came in with a JWT token matching `{values: {email: ["rob.johnson@email.com"]}}`. * `GMDATA_NAMESPACE_USERFIELD` - Typical value is `email`. If an environment variable you are looking for was not mentioned here, it's likely something that is not needed to change in a normal setup. ## All Environment Variables > Since these are all env vars, these are all technically strings. But the Type is a hint about what format is required, when the description, default, or example doesn't make it clear. This table is *ADVANCED*, and it is an automatically generated listing of every environment variable that is referenced inside of the code. If anything in here is used frequently, it needs to be listed in the previous section. | Name | Default | Description | Example | Type | | | ------------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------ | | DISABLE\_LOOKUPS | false | don't DNS check env vars representing hosts | true | boolean | | | DONT\_PANIC | false | disable panic when environment looks mis-configured | true | boolean | | | LESS\_CHATTY\_INFO | true | chatty info logs will write something to the log when a transaction begins, when there are no problems | false | boolean | | | CLIENT\_JWT\_PREFIX | /services/gm-jwt-security/1.0 | endpoint prefix for primary JWT service to resolve pointers to JWT tokens | /services/gm-jwt-security-gov/1.0 | | | | CLIENT\_JWT\_ENDPOINT\_ADDRESS | | IP of JWT server in the network | | a hostname | | | CLIENT\_JWT\_ENDPOINT\_PORT | | port of JWT server in the network | 8443 | an unsigned int | | | CLIENT\_JWT\_ENDPOINT\_CERT | | JWT server client cert | base64 line pem written to certs/jwt.cert.pem | mounted path or one-line base64 | | | CLIENT\_JWT\_ENDPOINT\_KEY | | JWT server client key | base64 line pem written to certs/jwt.key.pem | mounted path or one-line base64 | | | CLIENT\_JWT\_ENDPOINT\_TRUST | | JWT server trust | base64 line pem written to certs/jwt.trust.pem | mounted path or one-line base64 | | | CLIENT\_JWT\_ENDPOINT\_PREFIX | | prefix to reach the CLIENT\_JWT\_PREFIX when proxied | localhost | path | | | CLIENT\_JWT\_ENDPOINT\_USE\_TLS | false | use TLS to connect to JWT endpoint | true | boolean | | | CLIENT\_JWT\_ENDPOINT\_CN | | the server name expected for this cert | | hostname matching server cert cn | | | GMDATA\_FABRIC\_CLUSTER | default | the name of this fabric cluster | us-east | | | | ZEROLOG\_LEVEL | WARN | logging level: INFO, DEBUG, WARN, ERR | INFO | | | | MASTERKEY | | Master key for the encrypted content | som3r9doMg1bberish | master key for the data | | | AWS\_REGION | | Bucket location | us-east-1 | some non-whitespace token | | | AWS\_S3\_BUCKET | | Bucket name, overridden by FILE\_BUCKET | | AWS\_S3\_BUCKET= must match a token without whitespace or special chars | | | AWS\_S3\_PARTITION | | Subdirectory within the S3 bucket, overridden by FILE\_PARTITION | username | | | | FILE\_BUCKET | | Bucket name | | FILE\_BUCKET= must match a token without whitespace or special chars | | | FILE\_PARTITION | | Subdirectory within the file bucket | username | | | | AWS\_S3\_ENDPOINT | | Bucket host override | s3.region.aws.com | a hostname | | | AWS\_REKOGNITION\_ENDPOINT | | Bucket host override | rek.region.aws.com | a hostname | | | AWS\_ACCESS\_KEY\_ID | | Set if not using IAM roles for the machine | AKAI... | iam roles used | | | AWS\_SECRET\_ACCESS\_KEY | | Set if not using IAM roles for the machine | AEFE... | iam roles used | | | USES3 | true | Use S3 | false | S3 bucket setup | | | S3\_TASKS | 512 | Max number of concurrent S3 tasks | 64 | an unsigned int | | | KAFKA\_PEERS | | Kafka nodes to talk to directly. A comma-delimited list of host:port pairs | localhost:9092 | a comma-delimited list of host:port | | | KAFKA\_TOPIC\_UPDATE | | Kafka topic for update events | gmdu | some non-whitespace token | | | KAFKA\_TOPIC\_READ | | Kafka topic for read events | gmdr | some non-whitespace token | | | KAFKA\_TOPIC\_ERROR | | Kafka topic for errors | gmde | some non-whitespace token | | | KAFKA\_CONSUMER\_GROUP | test1 | Kafka consumer group id | imageconverters | some non-whitespace token | | | KAFKA\_CERT | | id cert | single line base64 of pem | KAFKA\_CERT is expecting a single-line base64 encoded string | | | KAFKA\_KEY | | id key | single line base64 of pem | KAFKA\_KEY is expecting a single-line base64 encoded string | | | KAFKA\_TRUST | | id trust | single line base64 of pem | KAFKA\_TRUST is expecting a single-line base64 encoded string | | | KAFKA\_USE\_TLS | false | use TLS for kafka directly | true | boolean | | | KAFKA\_CN | false | cn for kafka | true | boolean | | | TEST\_JWT\_PRIV | | TEST ONLY! a base64 encoded single line of the private key for internal signing during tests | | base64 encoded line | | | JWT\_PUB | | the single-line base64 encode of the public key of JWT tokens we accept | export JWT\_PUB=\`cat jwtRS256.key.pub \\ | base64 -w 0\` | JWT\_PUB is expecting a single-line base64 encoded string | | JWT\_PUB\_1 | | the single-line base64 encode of the public key of JWT tokens we accept | export JWT\_PUB=\`cat jwtRS256.key.pub \\ | base64 -w 0\` | JWT\_PUB\_1 is expecting a single-line base64 encoded string | | JWT\_PUB\_2 | | the single-line base64 encode of the public key of JWT tokens we accept | export JWT\_PUB=\`cat jwtRS256.key.pub \\ | base64 -w 0\` | JWT\_PUB\_2 is expecting a single-line base64 encoded string | | JWT\_PUB\_3 | | the single-line base64 encode of the public key of JWT tokens we accept | export JWT\_PUB=\`cat jwtRS256.key.pub \\ | base64 -w 0\` | JWT\_PUB\_3 is expecting a single-line base64 encoded string | | JWT\_PUB\_4 | | the single-line base64 encode of the public key of JWT tokens we accept | export JWT\_PUB=\`cat jwtRS256.key.pub \\ | base64 -w 0\` | JWT\_PUB\_4 is expecting a single-line base64 encoded string | | JWT\_NOT\_BEFORE\_SKEW\_SECONDS | 86400 | seconds that not-before is in the past, to handle mutual clock skews | 60 | an unsigned int | | | MONGOHOST\_MASTER | | Mongo host ip:port that we replicate with | m1:27017,m2:27017 | a comma-delimited list of host:port | | | MONGODB\_MASTER | | Mongo database we replicate with | gmdatadev | some non-whitespace token | | | MONGOHOST | | Mongo host ip:port | m1:27017,m2:27017 | a comma-delimited list of host:port | | | MONGODB | gmdatax | Mongo database | gmdatadev | some non-whitespace token | | | MONGO\_CERT | | Mongo TLS cert base64 | cat ./certs/server.cert.pem \| base64 -w 0 | MONGO\_CERT is expecting a single-line base64 encoded string | | | MONGO\_KEY | | Mongo TLS cert key base64 | cat ./certs/server.key.pem \| base64 -w 0 | MONGO\_KEY is expecting a single-line base64 encoded string | | | MONGO\_TRUST | | Mongo TLS trust base64 | cat ./certs/server.trust.pem \| base64 -w 0 | MONGO\_TRUST is expecting a single-line base64 encoded string | | | MONGO\_CN | | Mongo SNI name | | | | | MONGO\_SOURCE | | Mongo login source | $external | | | | MONGO\_MECHANISM | | Mongo login mechanism | MONGODB-X509 | | | | MONGO\_USE\_TLS | false | Mongo use TLS | true | | | | MONGO\_INITDB\_ROOT\_USERNAME | | MongoDB user id | mongoadmin | | | | MONGO\_INITDB\_ROOT\_PASSWORD | | MongoDB password | S0m3Pass | | | | TEST\_LOAD\_ITERATIONS | | number of iterations for load test | 10000 | an unsigned int | | | GMDATA\_NAMESPACE | | A Directory in the root that lets you create content as yourself | | | | | GMDATA\_NAMESPACE\_USERFIELD | | The field that is that matches up with the directory you can create | | | | | GMDATA\_NAMESPACE\_TEMPLATE | (if (contains %s "%s") (yield-all) (yield R X)) | The default template to create a user implicitly | | | | | DELETE\_EXPIRED | false | Actually remove expired entries periodically to comply with privacy laws | | DELETE\_EXPIRED= should be true or false | | | DELETE\_EXPIRED\_POLL\_SECONDS | 600 | Number of seconds to poll for expired data | 3600 | an unsigned int | | | NOTIFICATION\_CACHE\_SIZE | 1000 | Number of items to cache when watching notifications on an oid | 100 | an unsigned int | | | MIMETYPES\_OVERRIDE | | Supply an alternate mime.types | ./mime.types | | | | LISTING\_DEBUG | false | Turn on debug for listing package | true | | | | BIND\_ADDRESS | 0.0.0.0 | bind address for port | 127.0.0.1 | a hostname | | | BIND\_PORT | 8181 | bind port | 9123 | an unsigned int | | | PRETTY\_PRINT | true | pretty print returning json by default. set this to false in production, as it makes json larger. | false | | | | HTTP\_TRANSPORT\_CANCEL\_HOURS | 4 | Hours before HTTP call is cancelled | 24 | an unsigned int | | | USE\_PPROF\_CPU | true | CPU profiling in pprof | false | | | | USE\_PPROF\_MEM | true | mem profiling in pprof | false | | | | HTTP\_CACHE\_SECONDS | 10 | HTTP default cache in seconds | 60 | an unsigned int | | | TRACE\_LOG | | write a trace to file name | /logs/trace.out | | | | REKOGNITION\_FACE\_INDEX | | Set a face index for AWS Rekognition | hackathon | | | | LOG\_OPEN\_FILE\_HANDLES | true | log open file handles to look for leaks | false | | | | GMDATAX\_CATCH\_PANIC | false | catch panics rather than restarting gmdatax | true | | | | GMDATAX\_SESSION\_MAX | 4096 | max HTTP sessions in progress | 10000 | an unsigned int | | | JWT\_API\_KEY | | JWT API key | a password | JWT\_API\_KEY is expecting a single-line base64 encoded string | | | NAMED\_BANNER | true | include name in banner | false | | | | GMDATA\_CERT | | id cert | single line base64 of pem | GMDATA\_CERT is expecting a single-line base64 encoded string | | | GMDATA\_KEY | | id key | single line base64 of pem | GMDATA\_KEY is expecting a single-line base64 encoded string | | | GMDATA\_TRUST | | id trust | single line base64 of pem | GMDATA\_TRUST is expecting a single-line base64 encoded string | | | GMDATA\_USE\_TLS | false | use TLS for Grey Matter Data directly | true | | | | GMDATA\_REQUIRE\_CLIENT\_CERT | true | demand a client cert | false | | | | GMDATA\_AUTHENTICATION\_HEADER | USER\_DN | a header that is TRUSTED to contain an authenticated user id. disable with value '-'. | - | | | | POLICY\_CACHE\_LIFETIME | 60 | amount of time an object lives in objectpolicy cache | 30 | an unsigned int | |