Authentication and Authorization
Policies
This service does not contact an authorization service (or authentication service for that matter). This service should stay that way. In order to have a secure service, encryption of data is not nearly enough. There needs to be a workable system for calculating access to objects. For the authentication part, we do not want a username/password for authentication purposes (though we might want a password for end-to-end encryption purposes):
Treat incoming users as a set of digitally signed attributes only. They are not usernames that we need to go look up elsewhere.
This allows us to operate without having to further defer to more backend microservices for authorization.
I HAVE NO backend user service that I can defer to at the moment.
There is a standard that does this, and it is possible to use a minimum subset of it so that it is easy to secure and implement.
JWT (json web token) is as simple of a specification as it can be. There is an RFC mess over top of it that brings in hazards, and we can not implement that.
The basic spec contains a base64 header that hints on the signing algorithm (and we must CHECK that it is the only thing that we expect).
The basic spec allows expiration dates to be encoded.
The basic spec allows for arbitrary content in the payload.
All of the header and payload are json chunks, so that it is not hard to parse or document.
Example Of Generic Authentication
Some signing service, that is not us, generates a DSA signing key ❗:
From that DSA signing key, it can generate and publish a DSA public key:
If we trust this service to sign attributes, then we include this public key in our trust store of public keys. We currently only support one public key for this purpose. If that server generates a digitally signed statement. This statement is designed to be passed in http headers, used in OAuth tokens. The user goes to an Authorization service that we have to login (with an X509 certificate or a password. It can SUGGEST to the site what needs to be signed. That service should reject any assertions that the user makes about himself that cannot be verified as true (according to its internal database), and the server is allowed to inject new attributes such as expiration.
So, the server responded to the user request and gave it this token (❗).
Our app ONLY sees these tokens on incoming requests, that were set on http headers (cookies, authentication). The first part of it is actually not encrypted. But this bearer token should NOT be leaked to anybody but the service that needs this token. It decodes in plaintext as this:
Header
Claims
The header says how to interpret the tail-end, which contains the digital signature. It is critical that we only allow alg that matches our actual trust cert, which is ES512. We should also only be honoring claims that include expiration timestamps, to limit the time that a leaked token can be used.
UserPolicy Specific
If a user comes in with a statement such as the one above, then we can accept it as a true statement. There needs to be some specific structure in these in order to be useful. Presume that we design the Claims payload to specifically look like this:
Some things about this document:
It has an expiration date, after which we should not honor the signature
The values are specific to an application family that want to automatically process flexible policies.
It has a regular structure to it.
map[string][]stringis the regular structure for the values.This regular structure helps a domain-specific language to be written into objectPolicy for evaluation later
ObjectPolicy Specific
When an object is created in the system, a policy is attached to it. The policy is authored in JSON (and literally stored as BSON). It is actually a function. The output of this function is a Permission. The inputs are the claims of the input token. Each of the sample objectPolicies are attached to an object on each of its updates.
These statements are rendered as LISP syntax for legibility. The
originalobjectpolicyfield can be set to this LISP statement directly, and the server will set theobjectpolicyfield to the equivalent, but much more verbose, json value ofobjectpolicy. The fieldoriginalobjectpolicymight also be set to something that is not our LISP language, such as an proprietary ACL language that gm-data does not understand directly.
This is how you would allow anonymous access (did not present us with a UserPolicy at all, then give them R and X access (ie: read is for properties. execute is for streams and directory listings.):
Note that this LISP syntax statement compiled to the json that is actually set in the objectpolicy field is.
In general, the json field "f" is the first argument in a parenthesis list (
headof the list in LISP terminology), and "a" is the list of remaining arguments (tailof the list in LISP terminology). The actual string values are represented with "v". This means that the transform between LISP and actualobjectpolicyis straightforward, and goes in both directions. Double-quotes can be used in values to include spaces and parenthesis in the actual value.
If we need audited-but-public access, then we can demand an email, just so that it shows up in audit logs:
This is a file that is owned by a particular email address (full access, including P for purge):
If this file is owned by a user and shared for read-access to a group:
You have to be a non-dual citizen to use this resource. (ie: US citizen required, but must not also be a citizen somewhere else):
Everything in this system is an event such as a Create, Update, or a Delete. When updating or deleting an object, it will search for the latest version of the object to evaluate the userPolicy against. Similarly, for a Create, it will look up the latest version of the parent directory and check the permissions against that. These evaluators take UserPolicy in as input and return permissions as output. Each of these evaluators is attached to each version of a resource that is access controlled.
As a general pattern, expect that access control whether a file is known to exist, and ownership determines who has edit access on a file, while read access may be granted otherwise. That means that this pattern should be common:
Example of "US Adult citizens can read files and metadata", that is "jointly editable by two specific email addresses or anybody in a specific admin group". Anyone not meeting the access threshold will even know that the file exists, which would require R or X.
So, taking COMPLICATED_ACCESS to be:
And OWNERSHIP to be:
We then would have a full originalobjectpolicy of:
Which when compiled to its equivalent json, becomes the requirements field of objectpolicy.
Combining The Concepts
UserPolicy values:
Is allowed access because we told our email.
And this object has full ownership by any bearer with an email equal to jane.doe@gmail.com:
The ObjectPolicy Language
The current policy language has a minimum number of functions to demonstrate the capability. It will probably needs some higher-level functions to compress common policy expressions.
(if $cond $trueBranch $falseBranch) : boolBased on boolean truth of$condit will execute either$trueBranchor$falseBranchand not evaluate the other branch.(and $arg+) : boolAnd against any number of arguments (usually two).(or $args+) : boolOr against any number of arguments (usually two).(not $args+) : boolNot against one argument.(contains $field $arg+) : boolWe presume that$fieldis a[]string. It returns true if any arg matches. (note: we should probably have contains-and and contains-or, just to be less surprising)(has $op $field $arg+) : boolWe presume that$fieldis a[]string.$opcan be specified as eithereq,notso that we can express more than equality that contains can do. Multiple args will do. ie:(has not employer snapchat zynga). (note: we probably wanthas-orandhas-andto make it less ambiguous.)(tells $field) : boolWe demand that this field is told to us in UserPolicy values. This supports saying that we want an attribute to exist for auditing purposes. Example:(tells email)(yield $args) : (bool,permssions-side-effect)If we encounter this in the tree, then we write each argument to the output list.(allow-all) : (bool,permissions-side-effect)Is identical toyield-all(allow-read) : (bool,permissions-side-effect)Is identical toyield R Xtrue : boolIs used in cases where we unconditionally yield permissionsfalse : boolIs used in cases where we document that this branch of the expression fails.
When a true branch is taken, the whole expression reduces to the true branch only, until a leaf is reached. The leaf must be a macro that eventually yields possible permissions.
Possible Permissions
Note that we NEVER mutate data in this design. We insert events that sort on (objectId, tstamp) so that we maintain full history. This is what allows this permission system to have a uniform structure to it.
"R" - Reading metadata about resources such as file/dir, but not necessarily being able to retrieve files or list directories. Without "R" we are not even told that the file exists. It wont be in the listings.
"X" - Execute. This is not the same as Unix filesystem "execute". What we mean is listing directories (same as in Unix), and for opening the file stream (not same as Unix).
"C", "U", "D" - For the mutation operations we need to look up the latest version of what we are changing before we allow the change to be appended.
When performing one of these on a dir, we check for "C" on the latest version of the parent dir.
When performing update on a dir or file, we check for "U" against the latest version that has this objectId that we are updating.
"C" - Create. This is used on directories. When inserting a file, we check that we have C on the latest version of the parent directory. Creates in the parent will be allowed by UserPolicy that yields C.
"U" - Update. This is used on everything. Updates on this objectId will be allowed for UserPolicy that yields U.
"P" - Purge. This DOES alter the database. Records are automatically hidden when overridden by new versions, and when they expire. But they are physically in the database. For compliance situations, such as GDPR, we must be able to purge objects as well. This means periodically complying by physically removing items that are either expired, or that a user has demanded be removed.
The reason we never mutate data is that we need to be able to recreate a replica of the database from the Update logs alone. Purge is kind of an exception to mutating data. But it can be safely included as long as we:
Never recreate the same objectId more than once (in the update audit logs)
Hold a purge request and only execute it after we actually find what we are asked to purge. This way we don't lose purge requests on replication.
Also, internally, we take advantage of the immutability of data so that we can run queries on snapshots in time. As we are processing as of the current time Now(), more records are coming in with future timestamps on them.
ObjectPolicy Authorization
The OAuth2 standard is used in a way that handles the authentication part of what is necessary to make access control in an application work. However, the steps of authentication and authorization need not be mashed together into one step. Doing so is actually a bad idea. There are several reasons for this:
Delegated Authorization (Authorizations cannot realistically be centralized into one large system)
Application-specific authorizations (Some authorizations only make sense in the context of a particular application)
Anonymity requirements (Instead of having a GDPR rule to try to legislate privacy requirements, design systems to separate Authentication from Authorization in a technically enforceable manner)
Extremely diverse authorization models that can be plugged in by collaboration between users and issuers (Build a system that does not need to change every time a new requirement is imagined for an authorization system)
We want to lock down resources by role, but audit users in those roles. The way that tokens are issued should rotate people through required roles, without us having to keep updating object permissions themselves.
The Authorization part of an application is usually very specific to an enterprise. A resource server, such as gm-data (that locks down objects with policies), cannot anticipate all of the strange requirements that an organization will have for security. So, it's best to actually keep Authentication, Authorization, and ResourceServing all in separate applications. Authentication may be extremely centralized, like Google. Authorization is specific to enterprises. And data stores that do ResourceServing are actually reasonably generic, but contain lots of enterprise specific data in them.
OAuth2 Authentication
A basic goal of OAuth2 is to reliably assert the identity of a user, with some context about this assertion. If a user has been authenticated, that generally has nothing to do with authorization (even though there is often some mixing of the two concepts). Examples of Authentication (only):
A user is the owner of
jane.doe@gmail.com, as asserted by GoogleA user owns a PKI cert issued by
deciphernow.comenterprise CA, with a name ofjane.doeThe identity assertion is valid for the next 24hours.
It is a bad idea to go any farther than just asserting identity here if we ever need to support privacy requirements; even if it is just preventing one user from learning too much about the privileges of another user. This is because we assume that this server is going to log everything that we tell it. We do not need to tell this server what we are going to do with our identity yet. We just need to be able to prove this identity to an Authorization step later, which may be quite a bit more invasive.
There can be many trusted identity providers. Presume for the sake of an example, that these entities will sign JWT tokens that assert claims about a user's identity and validity time for the signature:
In this case, the json is expected to have a field exp and sub and to be signed by an entity that Authorization recognizes. If we do not sign for reasonable finite expiration times, then the authorization servers will remove us from their list of trusted authentication signers. Note that this authentication has no idea what kind of roles we can assume (admin, operations, dev, etc).
Authorization
Integration into gm-data is done by keeping authorization servers separate from the resource servers being protected.
In a manner that is very similar to OAuth2 authentication, we can take an authentication (that could be done by any means that we trust) in exchange for an authorization. These authorizations may be very application-specific attributes. There can be many authorizations per authentication. Presume that many governments and large enterprises are willing to sign age and known citizenship attributes about an identity.
Given...
It looks in its database, and is willing to sign this intentionally vague attribute assertion:
All things that it signs have definite requirements. The exp field must be present so that these tokens have a finite lifetime. The length of token that we allow them sign is sufficiently small for security purposes, or else the ResourceServer will remove USA from the list of trusted Authorization signers.
We assume that this system is going to log everything that it can about us, regardless of what laws exist (GDPR, etc). So, it will log this task that it did for us, to sign off on some attributes. If we sent in any parameters with this request to either shorten the validity of the token, or to limit the attributes in the token, it will probably log those too. The user may have asked that the email address is not signed into the token. But in any case, the authorization server happens to know who the user is, and what attributes it could possibly sign for that user.
This authorization provider could trust many authentication providers to sign statements.
Policy
When a user arrives at a resource, the OAuth2 authentication token doesn't necessarily need to be presented. An authorization token may be sufficient for the task at hand. Instead, the authorization token, which is quite similar to an authentication token, can be presented. An authorization token may identify the user at a level that is blurred out to an appropriate level of detail.
We call this authorization the userPolicy. The values are a set of attributes that we can believe are true. We design the attributes to match up with objectPolicy that we attach to objects to secure them.
An objectPolicy is a function that takes in a userPolicy and returns a list of permission. The possible items returned could be:
C - create (in a directory)
R - read (read about a file)
U - update
D - delete (left in the trash can)
X - execute (to pull its actual content)
P - purge (remove from the trash can so that it cannot be found in history or brought back)
When the user tries to access an object, presume that it's /thesite/images/a9134.jpg, there was an objectPolicy attached to the object that looks like this:
In order to know of this file's existence at all, you must be an adult with a Netherlands citizenship
You must have an email of rutger.mueller@arts.nl to get all permissions on it
Otherwise, you can read (R for "read") its metadata and stream its contents back (X for "execute")
Also, any security label that we want to show on this file (like a movie rating), can just be encrypted with the object. So that this label is only shown if R permission was granted:
Notice a few things about this:
If the resource server that we send this request to logs everything, in an attempt to violate GDPR laws, it may not have a mechanism to de-anonymize us. The
userPolicydoesn't have our email address. It would need to collude with authorization provider "USA" to do that. This significantly simplifies compliance by giving us a mechanism to actually have compliance.If we are in a world where there is no expectation of privacy at all (regulated industries and governments), then we can just sign the identity of the user into the authorization, so that the user is definitively logged and identified as an exact individual. This is an instance of mixing Authentication and Authorization, and it is done as a requirement of the task at hand, rather than being forced upon the system by design.
The
objectPolicyis literally a function taking auserPolicyas input, to return apermission.
We have a sequence of proof starting from a Google login, and ending with a permission.
Furthermore, when the actual fields in policies are sensitive, then it is possible to have Authorization not only issue the userPolicy tokens, but to also author the objectPolicy chunks. Because of this, it can obfuscate all of the terms in the issued tokens, because it only needs to evaluate with the correct answer, and not necessarily be legible. An example pair:
This is a userPolicy created by USA:
This is an objectPolicy also created by USA when the author was creating the object:
Evaluation of this incurs no extra runtime cost, because it's comparing values between the generated objects; which could have been obtained through a MAC, or just a random identifier mapped to a dictionary lookup.
The security label can also be decrypted if access was granted to see the security label with the R permission. The security label is encrypted strongly, but because the objectPolicy label needs to be evaluable, it counts more as obfuscation that doesn't leak names, but allows for correlations to be made in some circumstances.
We may want a feature to not show
objectPolicyback to the user in some situations. Only the server really needs to evaluateobjectPolicy. But the user will want a mechanism to figure out why permissions come back the way that they do. An alternative is to not even obfuscate the values, but not let the server return them to users unless there is a need for them in the client.
Roles
The combinations of fields that can be represented in an objectPolicy can readily represent roles with complex rules. Roles can allow the staff of an organization to be rotated without having to re-classify content. Take as an example, a user that has been put into a tech lead role on a project. That means that authorizations (in the form of JWT tokens) will sign off on these roles.
This says that if the token pertains to a US citizen that does development on gm-data, that we allow the person to read the file. But the person must be a team-lead on gm-data to modify the file. If not a US citizen developer on gm-data, then the user will not know that the file exists at all. The label is taken to be the user identity, and is written to an audit log. This way, we lock down data by role, but we audit by identity.
The point of a role orientation is to avoid encoding user names into the permissions of the data. There is a privacy aspect to this, in that we would like to avoid enumerating clearances of all persons in the data unnecessarily. The more important aspect is that this handles organizational churn. We do not need to update objects to let developers and team leads move on. This sets us up so that we do not need privileged administrators to come in and turn over ownership to specific identities when users leave the system.
Contextual Roles
A similar use for roles is to allow for JWT tokens to be issued in context. A task in progress can be a context for having a JWT token issued. Since the authorizations are application-specific, they can accept parameters that describe the task in progress. These JWT servers are relatively simple servers to write, and can be written per organization to do things such as wrap an existing LDAP or ActiveDirectory server. The JWT servers facilitate keeping gm-data away from organization-specific schemes, so that gm-data does not need authentication or authorization plugins to inter-operate with enterprises.
A patient only known as "patient.0000" enters the office of "Dr Edward Jones MD". The doctor greets the patient and logs in, with that user present. The call to the JWT server can be parameterized in some manner that is enterprise-specific. Presume that when we authenticate, we must plug in which patents are with us doing a visit:
When the JWT token is issued, if patient.0000 is one of our patients, then the JWT server is willing to write us the token. This action is audited by the JWT server itself.
So, now with this signed authorization token, the doctor can fetch a directory listing, where there is a surgery video surgery-2019-04-05-patient.0000.mp4. The video is locked down with an objectPolicy like this:
This states that: "Dr Edward Jones MD" is the owner/editor of this file. But as long as "patient.0000" is present, dermatologists can bring it up for viewing. The idea is that since this file may indicate costly conditions such as cancer, we do not want people associated with the insurance company getting these files.
Bootstrapping Security
An important aspect of this design is to prevent privilege escalations from being a normal way of just getting work done. When gm-data is started, it has a completely read-only root directory, with objectPolicy: (yield R X). So, how does anybody write into it? By default, there are special functions in the objectPolicy language for getting started without administrative intervention. We can create a home directory /home that has this objectPolicy:
What this means is that by default, we have a read-only directory, via (yield R X) where you can only read metadata (R - read) or stream files back (X - execute). But the first check has something unusual in it. It will give create permissions as well, via (yield C R X), but only under very specific conditions. The JWT email field must be equal to the proposed Name of the object, and the object must be a directory (or equivalently, not a file). This condition in isolation:
This allows users to do self-service. If a JWT token is issued with an email in it, then the users can create a directory in this directory, so long as it is named after the email. Due to this, administrators do not need to get involved in creating home directories.
Reducing or Escalating Privileges
This same principle can be applied to asking for reducing (or escalating) privileges. If getting authorized, there is always a need for some context beyond prior Authentication. When going to the JWT server to get an Authorization token, parameters can be passed in to manipulate how the JWT token will be issued. The JWT server has its own database of the most privileged tokens it can possibly write, and users may require them to be less than fully privileged, for the context they are in. These are common situations:
privilege: root- asudomechanism may or may not be supported. If so, escalations should be audited and only done when demanded in the context of the job. This prevents situations where there are users that just have too much access to everything, because of rare situations where they need to do almost anything you can think of.privilege: readonly- the opposite ofsudo, tosudown to only have readonly privileges. This way, time-limited read-only versions of a user privilege can be supported.reduce token to only include a subset of possible roles, projects, etc. As an example, a JWT server might be able to assert your age. You may want a JWT that only asserts that you are an adult, with no other information written into the token. It is up to the JWT server whether it is willing to sign such a thing. But that opens up the possibility of anonymous and pseudo-anonymous mass consumer content, such as movie serving. ex:
(if (contains age adult)(yield R X)), or(if (contains membership platinum)(yield X)).
OAuth2 and Similar Authentication Sequences
For this system, we take measures to simplify the system:
We are completely agnostic as to how Authentication actually happened.
If we are guaranteed to be behind a reverse proxy that can control our headers, then we can look in the cookies and headers to normalize the authentication name, and put that into a well-known header such as USER_DN.
When we are sent to an Authorization server, the USER_DN is accepted, and used to look up attributes. There are multiple such Authorization servers (which we call JWT servers internally).
An app can choose which JWT server to get authorization from, and accept multiple JWT servers.
The only requirement is that the
userPolicythat the resource server expects, and theobjectPolicybeing written onto objects are done consistently.An Enterprise can write a JWT server to deal with its internal ActiveDirectory notion of permissions.
Any details about organizational structure, such as who is allowed to impersonate as who stays out of the resource server, and goes into the Authorization servers. For example: granting users sudo access, or allowing superiors to escalate to include the privileges of a subordinate in a principled and logged manner. This is important for dealing with situations where employees leave the company.
Technical
In order to avoid forcing systems to go back to the signer to have signatures verified, asymmetric signatures must be used. The Authorization server signs JWT tokens:
We only issue or recognize ES512 tokens. We cannot allow HMAC without forcing clients to be re-verified against the original issuer. We want to support use cases where we cannot, or do not want to go back to issuers to validate tokens.
The Authorization token in addition to a ES512 format, is specific about the fields that it requires:
"exp" must specify an acceptable expiration date
"values" must specify a mapping from string to array of string values.
In order to support large-token scenarios (where cookies cannot fit into http headers), Authorization does something similar to OAuth authentication codes. In this situation, the JWT caches the cookie where the user can find it, and only gives the user a cryptographically random number to look the cookie up later. There is also a security benefit to not having the user handle the cookies.
The cookies are HttpOnly, set to Secure, and are limited to an actual Path that they can be sent to.
Beware of reverse proxies interacting with same-origin policy, and defeating CORS protections!
The flow for Authorization is similar to OAuth2. But it is designed to work right next to an existing OAuth2 authentication that we trust. Then we have apps serving up restricted data trust Authorization tokens.
USER_DN
The basic idea of splitting Authorization from Authentication can be applied to cases where we hide how a previous step was done. The Authorization step does not care how Authentication was done. When we are guaranteed to be behind a proxy that enforces a header, then we can take a header as "proof" that this step was done.
If we rely on USER_DN being set as the last step of the Authentication process, then the Authorization process can just read the value of USER_DN. This means that there does not need to be a cookie set in the browser for either Authentication or Authorization.
USER_DN can be instead set as the result of a client X509 certificate.
It might have been set as a result of an OAuth2 login. It does not matter how, because USER_DN is set.
When we have API client code, they can use client X509 certificates to do their work, and there are no cookies involved.
Bulk Consumers vs Identified Content Creators
Combining the use of JWT tokens that have personally identifying information, to name owners, can use the USER_DN mechanism. Each of these users will generally need to be looked up in a system such as an LDAP to associate USER_DN with JWT claims.
But consumers may come in under JWT tokens that are not personally identifying. A user may get a URL issued from somewhere, that has a JWT in it.
This would respond by setting the JWT token in setuserpolicy to be a cookie userpolicy in the browser. The user would be looking at the UI, and the JWT claims would be set. They would be something like:
In this example, this might be the JWT claims of a purchase of 30 day access to Konami web-based games. The point of in-person issuance is to verify that the user meets the age and parental consent arguments.
The main use cases:
Users may purchase anonymous JWT tokens for time-limited access to a site; possibly in-person. This is similar to buying phone cards and iOS Redeem codes at gas stations and convenience stores. (This happens in the world of cell phone apps because for game charge-ups, kids do not have credit cards. They buy redeem codes in cash.)
Redeem codes could be generated for users based on actually ensuring that they meet the requirements to have such tokens. An example would be for games that must obey age restrictions, such as PG-13. The token may only be issuable after physical age verification, or verification of parental consent.
The number of anonymous content consumers could easily dwarf the number of content-creators. These users do not necessarily have a USER_DN, or any means whatsoever for being identified, yet may be using the data services in a way that audits and blocks unauthorized accesses to the required level.
Last updated
Was this helpful?
