# Lifecycle

An object is represented by a sequence of _Events_ that mutate it. The object is associated with a state machine. ![objects.png](https://2512473607-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LsNFVozLgvw3NDMzxBg%2Fsync%2Fe431725e21f1166484e323386cc5833cc4812351.png?generation=1597773092958594\&alt=media) ![objectstate.png](https://2512473607-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LsNFVozLgvw3NDMzxBg%2Fsync%2Fe60da9c6b8feac8e844a1da3791018117eebc032.png?generation=1597773085140401\&alt=media)

## Concurrency

In gm-data, an append-only design is used. Records are never mutated. An `oid` will have a lifetime determined by the `(oid, tstamp)` pairs on every event that gets written. Due to concurrency and parallelism (in which "parallel" means that the order in which items is accomplished does not matter), it is possible for records to get inserted in an order that is not exactly the same as `tstamp` order; particularly in cases involving replication.

Even in the simple cases, such as gm-catalog, where a service that uses gm-data wants to write, we have a problem to deal with. This issue is not any different from problems that arise from any shared filesystem, such as a shared NFS mount. Some definitions first:

* Define mutation to mean that there is an update that creates new information while also _destroying_ old information.
* An "update" that only appends new records without modifying old records is not the same thing as an outright mutation.
* Conflicting records can be stored in such a way that all databases at least agree on the final result when there is a semantic conflict.  This does not imply that any database got the result that it wanted.
* A classic _data race_ that _destroys_ the old value is a serious problem, because this can cause a pair of databases to converge on different states.  When two databases end up with different states, with no way to converge them to one consistent state, we call that `database corruption`.

As an example, if on startup, `gm-catalog` has two instances that both decide to write the same file, they are _racing_ to perform that file write. It may be that one file does the create (action `C`), and the other one does update on that (action `U`). It could also be that they both managed to do a create (`C`), creating two directories called `/world` with different `oids`. Even with a locking mechanism, this could still have happened during a network partition; because locks require consensus, which is generally impossible during a network partition (ie: use cases where datacenters are disconnected from the world, and need to keep collecting data).

* This race happens when we need to create the home directory into which to start writing, such as `/world`.  But, because we do _not_ throw away the records that lose in a conflict, it is actually not a real problem.
* When there are no conflicts, `checkedtstamp` will always point to the `tstamp` of the previous record in history.  If multiple records have the same `checkedtstamp`, then we know that two concurrent edits were created.  They are like git branches that need a merge.  This can be rendered in an user interface, so that a human can attend to a problem that may require manual conflict resolution.
  * The `checkedtstamp` for a create refers to the `parentoid` at the given `tstamp`, because action `C` means "permission to create, given by the parent".
  * The `checkedtstamp` for an update refers to the `oid` at the value `tstamp`.
  * Before you update a file, it is presumed that you have a `tstamp` of the current version you are looking at for that `oid`.  After the upload, if `checkedtstamp` is not the `tstamp` you had on the event just before the upload, then some other client must have modified the same file.  The conflict may or may not need a new update to resolve it.  But what is important is that _the conflicting versions_ are all available for inspection, to create a resolved update.

![concurrency.png](https://2512473607-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LsNFVozLgvw3NDMzxBg%2Fsync%2F4f4c5364799d74a172e6d9814c40f155b3bd9bcb.png?generation=1597773085162726\&alt=media)

> There are other strategies that can be employed. A service, such as gm-catalog, can elect a leader (ie: take a census, and lowest identifier wins) to be the only one _trying_ to perform the write to gm-data. A general lock service might be taken out by a microservice for the duration of a write attempt (ie: 10 sec lease on exclusive update access to a file). gm-data might have a mechanism for leasing out a lock on updates to a file, though such a mechanism would only work within a strongly consistent cluster.

Another form of concurrency that is very real with documents is that a user may download a document and spend hours looking at it and editing it. When it is time to re-upload the document, fetching latest properties can tell us that our edits are not based on the current version.

> Here it is possible to have applications define locks that periodically renew, to tell user interfaces to refrain from updates until the lock is removed or expired. This kind of functionality probably belongs in a different service, because this is about performing transactions across microservices in general.

We may want to tell the user to compare the two versions and upload a merged version of the document. When the upload is finally performed, if the last known `tstamp` is not equal to `checkedtstamp`, then we know that there was a concurrent update. We can't stop this with locks, because we may replicate in a conflicting update that happened in our past. What is important is that given the `/history` of an oid, we can show conflicts to the user when they happen, and we always have _all_ versions required to produce a merged version. Users who have spent many hours _creating_ modified versions of docs do not want their updates to be rejected outright. Instead, they need to be able to use the conflicting versions to produce a de-conflicted version.

In general, automated merging is done by having data structures that merge automatically. But when uploading arbitrary files, such as Word/PowerPoint/PDF, it is not possible to completely automate file merges to keep the user out of it.
