---
title: "@octabits-io/evidence"
summary: "The audit evidence for AI actions, generated rather than written: from the agent ledger and the disclosure log, one bundle per period or per run — every applied action with who decided it, what it changed, the passages it was read from and the model calls behind it, the edit rate per field, stated confidence against what reviewers kept, the disclosure totals, the ledger chain verified against an anchor, and the findings an auditor should read first — as JSON under a published schema and as a cover sheet in Markdown."
install: "pnpm add @octabits-io/evidence"
availability: "Available from November 2026"
version: "0.7.0"
license: "MIT"
related:
  - name: "@octabits-io/agent-ledger"
    url: "https://octabits.io/platform/agent-ledger.md"
  - name: "@octabits-io/disclosure"
    url: "https://octabits.io/platform/disclosure.md"
  - name: "@octabits-io/proposal"
    url: "https://octabits.io/platform/proposal.md"
  - name: "@octabits-io/result"
    url: "https://octabits.io/platform/result.md"
canonical_url: "https://octabits.io/platform/evidence"
---

# `@octabits-io/evidence`

When an agent prepares and a person decides, someone later asks for the proof:
which actions were applied, who decided them, what the reviewer changed, what
each value was read from, which model saw what, and whether the record was
touched since. Writing that by hand is the work an audit costs. This package
generates it from the two records the platform already keeps — the
[agent ledger](https://octabits.io/platform/agent-ledger.md) and the [disclosure log](https://octabits.io/platform/disclosure.md) — as
one bundle per period or per run:

- **every applied action** — the ledger entry and its hash, who acted and on
  whose behalf, `reviewed` / `autopilot` / `manual`, what it changed, whether
  the reviewer edited each value, the passages each value was read from
  (`derivedFrom.citations`), and whether it was reverted since;
- **the model calls behind each action**, joined on the workflow id — where each
  went, the content class, sent / failed / denied, a digest of the payload and
  never the payload — and the calls no applied action accounts for;
- **the edit rate per field** over every accepted value an agent proposed, the
  number that says whether the preparation is worth trusting;
- **stated confidence against review** — when the producer gave a
  `derivedFrom.confidence`, how often reviewers kept values stated at each
  level, so a model's claim to be calibrated is checked on your own data;
- **disclosure totals** per destination, class and outcome, with how many
  calls a detector raised;
- **what the detectors did**, where the gate had one: calls raised (by the
  class they were declared as), clear, skipped, failed, and the finding types
  summed — never what was found;
- **the ledger chain verified** — every link and hash recomputed — against a
  head you anchored outside the database;
- **findings**, most severe first: a broken chain, an unanchored one, an action
  applied without review, a run with no model call in the log, values with no
  citation, a field reviewers keep rewriting, a confidence that runs ahead of
  what reviewers kept, a call that declared no class, a detection that failed.

It is JSON under a published schema (`schema/evidence-bundle.schema.json`), the
record you file, and a Markdown cover sheet, its reading for a person and a
document-management system. The root is `zod` and three sibling roots; no
driver, no model runtime, no `node:*`.

```bash
npm i @octabits-io/evidence @octabits-io/agent-ledger @octabits-io/disclosure @octabits-io/proposal zod
```

## From the stores

```ts
import { collectEvidence, formatEvidenceMarkdown } from '@octabits-io/evidence';

const bundle = await collectEvidence({
  ledger,                 // your AgentLedgerStore, bound to the scope
  disclosureLog,          // your DisclosureLogStore — bound to the scope, or read with scopeKey below
  anchor: lastAnchor,     // a head you kept outside the database; omit and the bundle says "unanchored"
  since: new Date('2026-07-01'), until: new Date('2026-10-01'),
  scopeKey: orgId, generatedBy: userId,
});
if (!bundle.ok) throw new Error(bundle.error.message);   // evidence_source_failed, with the store named

await files.put(`evidence-${quarter}.json`, JSON.stringify(bundle.value));
await files.put(`evidence-${quarter}.md`, formatEvidenceMarkdown(bundle.value));
```

`workflowId` instead of a period covers one run — the evidence for a single
decision. `collectEvidence` reads the whole ledger chain once (a chain
verifies only from its beginning) and pages the disclosure log for the period.

## From what you already read

`buildEvidence` is the same, pure and synchronous, over data: the chain oldest
first, its `verifyLedgerChain` verdict, the anchor, and the log's rows. Use it
when the rows come from somewhere else — an export, a replica, a test.

```ts
import { buildEvidence } from '@octabits-io/evidence';
import { verifyLedgerChain } from '@octabits-io/agent-ledger';

const bundle = buildEvidence({ entries, chain: await verifyLedgerChain(entries, { anchor }), anchor, disclosures, since, until });
```

## A model gateway: the disclosure log alone

A gateway in front of a chat — LiteLLM behind the disclosure guardrail, say —
has no agent that applies anything, so it has no ledger. Leave `ledger` out
(or `entries` and `chain`, for `buildEvidence`): the bundle covers the
disclosure log alone and says `ledgerCovered: false`, finds nothing about a
chain it never read, and the cover sheet reads as *Evidence of model calls* —
the calls by destination, class and outcome, what the detectors found, and,
where the log names one, the calls per actor (`actors`).

```ts
const bundle = await collectEvidence({ disclosureLog, since, until, generatedBy: 'monitor' });
```

The actor is whatever the log was given: a key or a team by default in the
guardrail, a person only where the host may record one.

## What the host does for it

- **Store the resolved operations in the ledger.** An apply records
  `resolveDecision(proposal, decision)` as its `operations`: that is where the
  fields, the `edited` flags and the citations come from. Values and labels may
  be transformed before the ledger hashes them — sealed into references so an
  erasure can reach them — since the bundle reads neither (a label is used only
  while it is a string). Operations in another shape are reported as
  `unreadable_operations`, and the action appears without them.
- **Record what was turned down.** A decision names what it accepts, so the
  ledger alone never sees a rejection. Store `{ ...decision, rejected:
  rejectedOperations(proposal, decision) }` as the entry's `decision` — the
  rejected operations by shape (field, label, stated confidence), never their
  values — and the bundle counts each one: `rejected` and `rejectRate` per
  field, a `high_rejection_rate` finding above `rejectRateThreshold` (1/3) over
  `minProposedForRejectRate` (3) values, and a miss in the calibration. A
  decision that turns everything down is still an entry: no operations, only
  the rejected list. `quality.withoutRejections` counts the reviewed actions
  that do not say.
- **Give every model call its run.** The disclosure log's `correlationId` is the
  workflow id the ledger entry carries — `disclosureLog.sink({ correlationId:
  String(workflowId) })` on the guard. A run with no call in the log is a
  finding.
- **Name the agents whose model is not yours.** An outside agent — an MCP
  client proposing through the app's tools — runs its model on its own side;
  nothing it does passes the gate. Pass `outsideAgent: (principal) =>
  principal.id.startsWith('mcp:')` (or however the host names them) and its
  actions are an `outside_agent_action` (info) instead of
  `no_model_calls_recorded`. An agent it does not name is still expected to
  have its calls in the log.
- **Cite.** A value an agent proposed without `derivedFrom.citations` is counted
  as uncited.
- **Anchor.** Keep `ledger.head()` somewhere the database's owner cannot rewrite
  and pass it back. Without it the chain can still be verified, but a rewrite of
  the whole chain would not be seen, and the bundle says so.

## Stated confidence against review

Some models report a probability with each answer, and some vendors sell it
as calibrated. A producer that has one puts it on the operation as
`derivedFrom.confidence` (0 to 1); `bundle.calibration` holds it against what
reviewers did:

```ts
bundle.calibration
// { scored: 120, unscored: 14, meanConfidence: 0.86, keptRate: 0.71, calibrationError: 0.15,
//   bins: [{ from: 0.8, to: 1, values: 96, meanConfidence: 0.91, keptRate: 0.74 }, …] }
```

Values are grouped into five bins by stated confidence; each bin gives the
mean stated and the share kept unchanged, and `calibrationError` is the
weighted mean gap. Only **reviewed** actions count — an autopilot value has no
reviewer to disagree. A value is an `update` or a `create`: a `delete` or a
`reorder` proposes none, and is counted nowhere. A rejected value counts as not
kept where the entry records it (see above); where it does not, the miss never
reached the ledger, so for those actions `keptRate` is an upper bound — a gap
found here is real, and a model that looks calibrated may still be worse. A
value that was accepted and reverted later still counts as kept, which is one
more reason the rate is a ceiling. A bin whose stated confidence runs
more than `calibrationTolerance` (0.15) ahead of its kept rate, over at least
`minValuesPerBin` (10) values, is an `overconfident_values` finding. Until that
number is small on your data, do not order, route or skip review on a
confidence.

**Decisions** feed the same calibration. A host that applies a decision step's
outcome (octaflow's `defineDecisionStep`) writes one `update` per question
with the decider's confidence in `derivedFrom.confidence` and `edited: true`
where a person chose otherwise. Under a threshold policy the reviewed
decisions are the ones the decider was unsure of, so the low bins fill first;
the questions above the threshold in a decision a person settled still count,
because the person saw them too.

## Applied without review

What took effect with no person checking it first — a decision above its
threshold, a grant that allowed it — is counted per agent in
`bundle.autopilot`, with how much of it a person reverted since:

```ts
bundle.autopilot
// { actions: 214, reverted: 9, revertRate: 0.042,
//   agents: [{ agent: 'ai:message-triage', actions: 214, reverted: 9, revertRate: 0.042 }] }
```

A revert is the only sign that an autopilot action was wrong, so the rate is
a lower bound. An agent whose revert rate passes `autopilotRevertThreshold`
(0.1) over at least `minAutopilotActions` (5) is an `autopilot_reverted`
finding: raise its threshold, or send more of its work to review. The field is
optional in the schema, so a bundle written before it existed still reads.

## Findings

| Code | Severity | Means |
|---|---|---|
| `chain_broken` | high | a ledger entry was changed, removed or inserted after it was written, or the chain does not reach the anchor |
| `chain_unanchored` | medium | verified without an anchor |
| `unreadable_operations` | medium | an entry's operations are not proposal operations |
| `autopilot_action` | medium | applied by an agent without review, under a grant that allowed it |
| `autopilot_reverted` | medium | more than a tenth of what one agent applied without review was reverted since (at least five actions) — `autopilotRevertThreshold`, `minAutopilotActions` |
| `no_model_calls_recorded` | medium | an agent's run behind an action left no call in the log for the period — not raised for an agent `outsideAgent` names |
| `high_edit_rate` | medium | reviewers rewrote more than a third of a field's accepted values (at least three) — `editRateThreshold`, `minAcceptedForRate` |
| `overconfident_values` | medium | values stated at one confidence level were kept unchanged markedly less often than stated — `calibrationTolerance`, `minValuesPerBin` |
| `class_not_declared` | medium | a model call declared no content class and was treated as confidential |
| `detection_failed` | medium | the detector failed on a call, which was treated as confidential: check it is reachable and answers |
| `outside_agent_action` | info | proposed by an agent whose model runs outside the app (`outsideAgent`, an MCP client say): its model calls are not in the log, and none is expected |
| `uncited_agent_values` | info | values an agent proposed without a citation |
| `disclosure_failed` | info | a call failed after the gate let it through; the payload may have left |
| `disclosure_denied` | info | calls the gate denied; nothing left |
| `detection_raised` | info | calls whose content a detector found above the declared class; the gate decided on the higher one |

## Exports

| Path | What |
|---|---|
| `.` | `collectEvidence`, `buildEvidence`, `formatEvidenceMarkdown`, `evidenceField`, `evidenceBundleSchema` and the part schemas, `EVIDENCE_FORMAT`, `EVIDENCE_VERSION`, `EVIDENCE_FINDING_CODES`, `EVIDENCE_SEVERITIES`, and their types |
| `./schema/evidence-bundle.schema.json` | the bundle as JSON Schema (draft 2020-12) |

## What it does not do

- **Decide.** It reports; it never blocks, reverts or approves anything.
- **Prove the disclosure log.** The ledger is hash-chained and anchored; the
  disclosure log is not. The bundle reports what the log holds.
- **Map to a regulation.** No article numbers, no Annex IV template: the bundle
  is the facts a DPO, an ISO or an auditor maps onto their own framework.
- **Keep payloads.** Neither source stores prompts or outputs, so neither does
  the bundle.

## Stability

0.x: minors may break and say so; patches never do. The bundle carries
`format` and `version`; a change to its shape bumps `version`.