`@octabits-io/evidence`

When an agent prepares and a person decides, someone later asks for the proof: which actions were applied, who decided them, what the reviewer changed, what each value was read from, which model saw what, and whether the record was touched since. Writing that by hand is the work an audit costs. This package generates it from the two records the platform already keeps — the agent ledger and the disclosure log — as one bundle per period or per run:

every applied action — the ledger entry and its hash, who acted and on whose behalf, reviewed / autopilot / manual, what it changed, whether the reviewer edited each value, the passages each value was read from (derivedFrom.citations), and whether it was reverted since;
the model calls behind each action, joined on the workflow id — where each went, the content class, sent / failed / denied, a digest of the payload and never the payload — and the calls no applied action accounts for;
the edit rate per field over every accepted value an agent proposed, the number that says whether the preparation is worth trusting;
stated confidence against review — when the producer gave a derivedFrom.confidence, how often reviewers kept values stated at each level, so a model’s claim to be calibrated is checked on your own data;
disclosure totals per destination, class and outcome, with how many calls a detector raised;
what the detectors did, where the gate had one: calls raised (by the class they were declared as), clear, skipped, failed, and the finding types summed — never what was found;
the ledger chain verified — every link and hash recomputed — against a head you anchored outside the database;
findings, most severe first: a broken chain, an unanchored one, an action applied without review, a run with no model call in the log, values with no citation, a field reviewers keep rewriting, a confidence that runs ahead of what reviewers kept, a call that declared no class, a detection that failed.

It is JSON under a published schema (schema/evidence-bundle.schema.json), the record you file, and a Markdown cover sheet, its reading for a person and a document-management system. The root is zod and three sibling roots; no driver, no model runtime, no node:*.

npm i @octabits-io/evidence @octabits-io/agent-ledger @octabits-io/disclosure @octabits-io/proposal zod

From the stores

import { collectEvidence, formatEvidenceMarkdown } from '@octabits-io/evidence';

const bundle = await collectEvidence({
  ledger,                 // your AgentLedgerStore, bound to the scope
  disclosureLog,          // your DisclosureLogStore — bound to the scope, or read with scopeKey below
  anchor: lastAnchor,     // a head you kept outside the database; omit and the bundle says "unanchored"
  since: new Date('2026-07-01'), until: new Date('2026-10-01'),
  scopeKey: orgId, generatedBy: userId,
});
if (!bundle.ok) throw new Error(bundle.error.message);   // evidence_source_failed, with the store named

await files.put(`evidence-${quarter}.json`, JSON.stringify(bundle.value));
await files.put(`evidence-${quarter}.md`, formatEvidenceMarkdown(bundle.value));

workflowId instead of a period covers one run — the evidence for a single decision. collectEvidence reads the whole ledger chain once (a chain verifies only from its beginning) and pages the disclosure log for the period.

From what you already read

buildEvidence is the same, pure and synchronous, over data: the chain oldest first, its verifyLedgerChain verdict, the anchor, and the log’s rows. Use it when the rows come from somewhere else — an export, a replica, a test.

import { buildEvidence } from '@octabits-io/evidence';
import { verifyLedgerChain } from '@octabits-io/agent-ledger';

const bundle = buildEvidence({ entries, chain: await verifyLedgerChain(entries, { anchor }), anchor, disclosures, since, until });

A model gateway: the disclosure log alone

A gateway in front of a chat — LiteLLM behind the disclosure guardrail, say — has no agent that applies anything, so it has no ledger. Leave ledger out (or entries and chain, for buildEvidence): the bundle covers the disclosure log alone and says ledgerCovered: false, finds nothing about a chain it never read, and the cover sheet reads as Evidence of model calls — the calls by destination, class and outcome, what the detectors found, and, where the log names one, the calls per actor (actors).

const bundle = await collectEvidence({ disclosureLog, since, until, generatedBy: 'monitor' });

The actor is whatever the log was given: a key or a team by default in the guardrail, a person only where the host may record one.

What the host does for it

Store the resolved operations in the ledger. An apply records resolveDecision(proposal, decision) as its operations: that is where the fields, the edited flags and the citations come from. Values and labels may be transformed before the ledger hashes them — sealed into references so an erasure can reach them — since the bundle reads neither (a label is used only while it is a string). Operations in another shape are reported as unreadable_operations, and the action appears without them.
Record what was turned down. A decision names what it accepts, so the ledger alone never sees a rejection. Store { ...decision, rejected: rejectedOperations(proposal, decision) } as the entry’s decision — the rejected operations by shape (field, label, stated confidence), never their values — and the bundle counts each one: rejected and rejectRate per field, a high_rejection_rate finding above rejectRateThreshold (1/3) over minProposedForRejectRate (3) values, and a miss in the calibration. A decision that turns everything down is still an entry: no operations, only the rejected list. quality.withoutRejections counts the reviewed actions that do not say.
Give every model call its run. The disclosure log’s correlationId is the workflow id the ledger entry carries — disclosureLog.sink({ correlationId: String(workflowId) }) on the guard. A run with no call in the log is a finding.
Name the agents whose model is not yours. An outside agent — an MCP client proposing through the app’s tools — runs its model on its own side; nothing it does passes the gate. Pass outsideAgent: (principal) => principal.id.startsWith('mcp:') (or however the host names them) and its actions are an outside_agent_action (info) instead of no_model_calls_recorded. An agent it does not name is still expected to have its calls in the log.
Cite. A value an agent proposed without derivedFrom.citations is counted as uncited.
Anchor. Keep ledger.head() somewhere the database’s owner cannot rewrite and pass it back. Without it the chain can still be verified, but a rewrite of the whole chain would not be seen, and the bundle says so.

Stated confidence against review

Some models report a probability with each answer, and some vendors sell it as calibrated. A producer that has one puts it on the operation as derivedFrom.confidence (0 to 1); bundle.calibration holds it against what reviewers did:

bundle.calibration
// { scored: 120, unscored: 14, meanConfidence: 0.86, keptRate: 0.71, calibrationError: 0.15,
//   bins: [{ from: 0.8, to: 1, values: 96, meanConfidence: 0.91, keptRate: 0.74 }, …] }

Values are grouped into five bins by stated confidence; each bin gives the mean stated and the share kept unchanged, and calibrationError is the weighted mean gap. Only reviewed actions count — an autopilot value has no reviewer to disagree. A value is an update or a create: a delete or a reorder proposes none, and is counted nowhere. A rejected value counts as not kept where the entry records it (see above); where it does not, the miss never reached the ledger, so for those actions keptRate is an upper bound — a gap found here is real, and a model that looks calibrated may still be worse. A value that was accepted and reverted later still counts as kept, which is one more reason the rate is a ceiling. A bin whose stated confidence runs more than calibrationTolerance (0.15) ahead of its kept rate, over at least minValuesPerBin (10) values, is an overconfident_values finding. Until that number is small on your data, do not order, route or skip review on a confidence.

Decisions feed the same calibration. A host that applies a decision step’s outcome (octaflow’s defineDecisionStep) writes one update per question with the decider’s confidence in derivedFrom.confidence and edited: true where a person chose otherwise. Under a threshold policy the reviewed decisions are the ones the decider was unsure of, so the low bins fill first; the questions above the threshold in a decision a person settled still count, because the person saw them too.

Applied without review

What took effect with no person checking it first — a decision above its threshold, a grant that allowed it — is counted per agent in bundle.autopilot, with how much of it a person reverted since:

bundle.autopilot
// { actions: 214, reverted: 9, revertRate: 0.042,
//   agents: [{ agent: 'ai:message-triage', actions: 214, reverted: 9, revertRate: 0.042 }] }

A revert is the only sign that an autopilot action was wrong, so the rate is a lower bound. An agent whose revert rate passes autopilotRevertThreshold (0.1) over at least minAutopilotActions (5) is an autopilot_reverted finding: raise its threshold, or send more of its work to review. The field is optional in the schema, so a bundle written before it existed still reads.

Findings

Code	Severity	Means
`chain_broken`	high	a ledger entry was changed, removed or inserted after it was written, or the chain does not reach the anchor
`chain_unanchored`	medium	verified without an anchor
`unreadable_operations`	medium	an entry’s operations are not proposal operations
`autopilot_action`	medium	applied by an agent without review, under a grant that allowed it
`autopilot_reverted`	medium	more than a tenth of what one agent applied without review was reverted since (at least five actions) — `autopilotRevertThreshold`, `minAutopilotActions`
`no_model_calls_recorded`	medium	an agent’s run behind an action left no call in the log for the period — not raised for an agent `outsideAgent` names
`high_edit_rate`	medium	reviewers rewrote more than a third of a field’s accepted values (at least three) — `editRateThreshold`, `minAcceptedForRate`
`overconfident_values`	medium	values stated at one confidence level were kept unchanged markedly less often than stated — `calibrationTolerance`, `minValuesPerBin`
`class_not_declared`	medium	a model call declared no content class and was treated as confidential
`detection_failed`	medium	the detector failed on a call, which was treated as confidential: check it is reachable and answers
`outside_agent_action`	info	proposed by an agent whose model runs outside the app (`outsideAgent`, an MCP client say): its model calls are not in the log, and none is expected
`uncited_agent_values`	info	values an agent proposed without a citation
`disclosure_failed`	info	a call failed after the gate let it through; the payload may have left
`disclosure_denied`	info	calls the gate denied; nothing left
`detection_raised`	info	calls whose content a detector found above the declared class; the gate decided on the higher one

Exports

Path	What
`.`	`collectEvidence`, `buildEvidence`, `formatEvidenceMarkdown`, `evidenceField`, `evidenceBundleSchema` and the part schemas, `EVIDENCE_FORMAT`, `EVIDENCE_VERSION`, `EVIDENCE_FINDING_CODES`, `EVIDENCE_SEVERITIES`, and their types
`./schema/evidence-bundle.schema.json`	the bundle as JSON Schema (draft 2020-12)

What it does not do

Decide. It reports; it never blocks, reverts or approves anything.
Prove the disclosure log. The ledger is hash-chained and anchored; the disclosure log is not. The bundle reports what the log holds.
Map to a regulation. No article numbers, no Annex IV template: the bundle is the facts a DPO, an ISO or an auditor maps onto their own framework.
Keep payloads. Neither source stores prompts or outputs, so neither does the bundle.

Stability

0.x: minors may break and say so; patches never do. The bundle carries format and version; a change to its shape bumps version.