Audit Storage
Authoritative journal, encrypted evidence, checkpoints, and projections
Audit Storage and Integrity
KYA-OS separates preservation, evidence, transparency, and search. Combining them in one mutable JSON table makes independent verification and safe retention impossible.
Storage roles
| Store | Authority | Required properties |
|---|---|---|
| Authoritative journal | Accepted ledger history | Exact canonical bytes, signed envelope, parsed event, atomic compare-and-append, unique logical event identity |
| Evidence store | Sensitive payloads | Ciphertext before append, digest and size verification, tenant-scoped key, retention class, legal hold, disposal evidence |
| Checkpoint store | Merkle commitments | Idempotent checkpoint identity, previous checkpoint link, tree size, root, signature |
| Observation and anchor store | External views | Observer continuity and supporting WORM, RFC 3161, or SCITT receipts |
| Projection store | Search and UI | Rebuildable rows, projection version, source high-water, reconciliation status |
| Producer outbox | Delivery durability | FIFO source order, byte-stable retry, backoff, alarm recovery, dead-letter visibility |
Canonical bytes are the evidence
JSONB is useful for queries, but it is not the preservation format. Preserve the exact canonical entry bytes and signed receipt alongside parsed JSON. Projections can be deleted and rebuilt without changing ledger authority.
Atomic append
For each authenticated submission, the recorder:
- validates schema, producer authority, tenant authority, binding, and epoch;
- verifies encrypted evidence bytes against every referenced digest and size;
- resolves an identical retry or rejects divergent reuse of the event identity;
- reads the epoch head and builds the candidate entry;
- signs the canonical entry and receipt;
- compares and appends atomically, retrying bounded head conflicts;
- returns the complete signed entry only after durable commit.
One hundred concurrent appends must still produce a contiguous sequence and one predecessor per entry. No acknowledged append may disappear after process or database restart.
Evidence ordering
Evidence is encrypted by the producer or a trusted boundary before submission. The recorder stores and verifies ciphertext before committing an entry that references it. If evidence persistence fails, append fails. If append later fails, the recorder may dispose of uncommitted ciphertext only after confirming that no racing idempotent commit references it.
An EvidenceRef commits:
- object identity;
- ciphertext SHA-256 digest;
- optional plaintext commitment;
- media type and decimal-string byte size;
- A256GCM key identifier, nonce, and associated-data digest.
Retention and disposal
Retention policy applies independently to journal entries, encrypted evidence, and disposable projections. Legal hold overrides ordinary disposal. When evidence is destroyed, emit evidence.disposed with the target reference and retain the ledger commitment so later verifiers can distinguish deliberate disposal from unexplained absence.
Recovery
Backups and replicas must preserve ledger ordering, canonical bytes, signatures, key policy, checkpoint state, evidence metadata, legal holds, and producer high-water state. Regularly rehearse:
- restore into an isolated environment;
- replay and verify every chain and checkpoint;
- rebuild projections from the journal;
- reconcile outbox and source high-water state;
- prove that epoch and key transitions remain verifiable.
Never repair a damaged ledger by editing committed rows. Seal the affected epoch, document the incident, and start a successor epoch with an explicit transition.