Storage

kslate keeps everything in Postgres. The gateway holds no state of its own, so two gateways serving the same record see the same rows. This page lists the tables, says what each column holds, and shows the two UPDATE statements that make a write safe.

You do not need this page to use kslate. Read it if you operate the database, write a migration, or want to know what an archived record contains.

Nothing but the gateway and the worker writes these tables. Read them directly and you bypass scope, so an auditor sees fields their role may not read. Use the HTTP API for that instead.

Tables

Table	Holds
`classes`	One row per class. A cache of the class the CRD or the YAML file defines
`records`	One row per record. Carries `record_version` and the expiry
`fields`	The current value of every `versioned`, `soft` and `task` field
`entries`	Every entry of every `append` field. Never updated
`history`	Every write, applied or rejected
`checkpoints`	Every saved checkpoint. The newest per role wins
`archive`	A snapshot per record, for classes that set `onExpire: archive`
`scope_reads`	What a caller read, so the gateway can work out an omitted `based_on`

classes

Column	Type	Notes
`name`	`text`	Primary key. Class names are unique across the cluster
`spec`	`jsonb`	The class as loaded
`source`	`text`	Which policy source loaded it
`updated_at`	`timestamptz`

This row is a cache. The class itself lives in a SharedContext resource or a YAML file.

Nothing deletes the row when the class goes away. So “in this table but not in the live policy source” is what CLASS_GONE means, and the cached spec is what the gateway reads to answer who may still read the records. See Honest limits.

records

Column	Type	Notes
`class`, `id`	`text`	Primary key together
`record_version`	`bigint`	Rises on every applied write to any field
`created_at`	`timestamptz`
`last_write_at`	`timestamptz`
`expires_at`	`timestamptz`	Null when the class sets no TTL. Fixed at creation

fields

The current state of a field that is not append.

Column	Type	Notes
`class`, `id`, `field`	`text`	Primary key together
`value`	`jsonb`	Null for a `task` field, which has no value
`version`	`bigint`
`author`	`text`	The resolved identity. Never from the request body
`evidence`	`text`
`updated_at`	`timestamptz`
`owner`	`text`	`task` fields. Null when nobody holds it
`lease_expires_at`	`timestamptz`	`task` fields
`based_on`	`jsonb`	The premise of this value
`record_version`	`bigint`	The record version at which this value was applied
`identity_strength`	`text`	Null means unknown, not weak
`invocation`	`jsonb`	Writer-asserted. The gateway never reads a key out of it

The row has a foreign key to records, so a field cannot outlive its record.

entries

One row per entry of an append field. Rows are inserted, never updated.

Column	Type	Notes
`seq`	`bigserial`	Primary key. Orders entries
`class`, `id`, `field`	`text`
`entry`	`jsonb`
`author`	`text`
`evidence`	`text`	Not null. Evidence is required on every entry
`based_on`	`jsonb`
`created_at`	`timestamptz`
`record_version`	`bigint`
`identity_strength`	`text`
`invocation`	`jsonb`

The number of rows for a field is that field’s version. An append field at version 4 has four entries.

history

One row per write the gateway considered, whether or not it applied.

Column	Type	Notes
`seq`	`bigserial`	Primary key
`class`, `id`, `field`	`text`
`op`	`text`	See below
`version`	`bigint`	The version the write produced. Null when it was rejected
`record_version`	`bigint`
`value`	`jsonb`	What the caller tried to write
`author`	`text`
`evidence`	`text`
`expected_version`	`bigint`	What the caller said it read
`based_on`	`jsonb`
`inferred`	`boolean`	True when the gateway worked the premise out rather than the caller declaring it
`outcome`	`text`	See below
`created_at`	`timestamptz`
`identity_strength`	`text`
`invocation`	`jsonb`

op is one of:

`put`	`append`	`claim`	`release`	`takeover`

outcome is one of:


`applied`	the write landed
`stale`	the target field had moved
`stale_premise`	a field named in `based_on` had moved
`held`	another agent held the task
`scope_denied`	the role may not write that field
`evidence_required`	the field requires evidence and none was given
`identity_mismatch`	the body named an author
`weak_identity`	the credential was below the field’s floor

Both lists are enforced by a check constraint, so a new outcome needs a migration. Which rejections reach this table at all is covered in blame.

checkpoints

Column	Type	Notes
`seq`	`bigserial`	Primary key
`class`, `id`, `role`	`text`
`author`	`text`
`document`	`jsonb`	At most 64 KiB
`record_version`	`bigint`	The record version when it was saved
`created_at`	`timestamptz`

A load takes the highest seq for the role. Earlier checkpoints stay.

Column	Type	Notes
`snapshot`	`jsonb`	The whole record
`class`, `id`	`text`
`archived_at`	`timestamptz`

scope_reads

What one caller read, so the gateway can fill in a based_on the caller did not send. See based_on.

Column	Type	Notes
`scope`, `identity`, `class`, `id`, `field`	`text`	Primary key together
`version`	`bigint`	The version that caller saw
`read_at`	`timestamptz`

The scope comes from a header the host sends. The gateway does not invent one. This lives in Postgres rather than in gateway memory because a read served by one replica has to be visible to a write served by another.

How a write stays safe

Every write is one transaction. It locks the record’s row first:

SELECT record_version FROM records WHERE class=$1 AND id=$2 FOR UPDATE

That lock is what makes record_version rise one at a time under concurrent writers. The premise checks read fields and entries inside the same transaction, so a premise cannot pass against a value that changed a moment later.

A version check is one conditional UPDATE:

UPDATE fields SET … WHERE class=$1 AND id=$2 AND field=$3 AND version=$expected

It updates one row or none. None means the field moved, and the caller gets STALE.

A claim is the same shape with a different condition:

UPDATE fields SET owner=$me, lease_expires_at=… WHERE … AND (owner IS NULL OR owner=$me OR lease_expires_at < now())

The three conditions are the three cases where a claim succeeds. The lease is free, it is already yours, or it has run out. Anything else updates no row, and the caller gets HELD.

No caller ever waits on a lock, holds one across requests, or sees one. A request gets an answer or a rejection. The only advisory lock kslate takes is during migrations, so that replicas starting together apply them one at a time.

Why the current value keeps its own based_on

fields and entries each carry based_on, and history carries it for every write. The copy on the current value is not redundant. It is what get and query return, so one query tells an agent which records are behind without joining history for each one.

How long rows live

Rows	Kept until
A record, its fields, entries, checkpoints and history	The record’s TTL. Then deleted, or archived if the class says so
A record in a class with no TTL	For as long as the class has no TTL
History rows belonging to no record	A week. The worker reaps them
`scope_reads`	An hour. The worker reaps them
An archive snapshot	Nothing deletes it

History goes with its record. That is not the same as kept forever, so a class whose history you want to keep sets onExpire: archive.

A rejected write that never created a record leaves a history row with no record to belong to. A STALE on a record that does not exist is one example. Those are the rows the week-old reaper removes.

The gateway works out an omitted based_on from reads in the last 15 minutes. The worker keeps scope_reads for an hour, so the window is always shorter than the retention.

Migrations

Numbered files compiled into the binary and applied when the gateway or the worker starts. Upgrade describes the ordering, the advisory lock and the checksum.

There is one, 0001_schema.sql, and it creates everything on this page. kslate has not been released, so no database predates it and there is nothing to upgrade from. The number is kept because migrations are applied in filename order and a second one has to sort after this one.

Never change the SQL of an applied migration — a change in what it does is a new migration. The checksum covers the whole file, comments included, so even a reworded comment makes a database that already applied it refuse to start. Before 1.0 the answer is to recreate the database.

Status and compatibility