Skip to content
Docs
Build an agent
The audit trail

The audit trail

See what happened read one field’s story. This page is the rest: what the record keeps, what it refuses to keep, and how to check your agents behaved.

Refused writes are in it

The row that matters most is the one that did not land.

v2   14:20:51  remediation          apikey:remediator/worker-2 takeover APPLIED
v2   14:20:48  planned_action       apikey:diagnoser       put    APPLIED
      evidence: eviction caused a second outage on node-4
--   14:20:48  remediation          apikey:remediator/worker-2 claim  HELD
v1   14:20:48  remediation          apikey:remediator/worker-1 claim  APPLIED
v1   14:20:48  planned_action       apikey:diagnoser       put    APPLIED
      evidence: kubelet NotReady 12m

A -- in the version column is an attempt that was refused. Claims, renewals, releases and takeovers are rows too, so the record says who held the task and when, not only who wrote values.

What never reaches it

Only failures the gateway got far enough to decide against the record leave a row: stale, stale_premise, held, scope_denied, weak_identity, evidence_required and identity_mismatch.

Rejected earlier, so absent entirely: a bad credential, an identity with no role, an unknown field, the wrong operation for a field’s mode, a malformed body, a failed checkpoint gate. History is not a request log.

Checking your agents, not just your record

kslate verify reads the same history and reports which guarantees actually fired and which roles behaved oddly:

KSLATE_APIKEY=rem-token bin/kslate verify --server http://127.0.0.1:8080 incidents/inc-111
incidents/inc-111
guarantees fired:
  HELD                 1
findings: 4
  WROTE_WITHOUT_PREMISES       apikey:diagnoser  planned_action  seq 10
      wrote planned_action with no based_on while remediation was at v1 — the decision's premises are unknown
  ROLE_WROTE_NOTHING           communicator
      no identity of this role has written to the record
  ROLE_WROTE_NOTHING           detector
      no identity of this role has written to the record
  ROLE_WROTE_NOTHING           dev
      no identity of this role has written to the record

WROTE_WITHOUT_PREMISES is the useful one: an agent changed the plan while a remediation was claimed, and declared nothing about what it had read. The write was legal and kslate accepted it — but afterwards nobody can say what that decision rested on. That is the gap based_on closes, and verify is how you find agents that are not using it.

ROLE_WROTE_NOTHING is usually just a role this record never needed. verify exits non-zero whenever it has findings, so a script calling it should expect that.

What it cannot tell you

Whether the evidence was true. kslate checks evidence is present, never that it is accurate. An agent citing a log line that does not exist is recorded as having cited it.

Whether the decision followed from the premise. based_on proves what was current when the write was made, not that the reasoning was sound.

What happened outside the record. If something changed in your cluster and no agent wrote it down, the record does not know.

The audit trail answers who wrote this, when, citing what, relying on which versions, and what was refused. It does not answer whether anyone was right.

Next

  • Design a class decides how much of this you get: which fields require evidence, which require a strong credential, and which identities may write at all.
  • blame and history for the endpoint, the row shape and the retention rules.
Surviving a crashDesign a class