--- title: "@octabits-io/disclosure" summary: "A disclosure gate before every model call: the call site declares what the content is — public, internal or confidential — and each model destination what it may receive." install: "pnpm add @octabits-io/disclosure" availability: "Available from November 2026" version: "0.8.1" license: "MIT" related: - name: "@octabits-io/result" url: "https://octabits.io/platform/result.md" canonical_url: "https://octabits.io/platform/disclosure" --- # `@octabits-io/disclosure` Pseudonymisation protects personal data. It does nothing for the content that actually hurts when it reaches a model vendor: which system runs in which zone, which finding is still open, what a customer's contract says. This package puts a **disclosure gate** in front of every model call. The call site declares what the content is, each model destination declares where it runs, and one fixed rule decides: | Class | Content | May reach | |---|---|---| | `public` | published material, standards text, generic questions | any destination | | `internal` | your records, but nothing that names what must stay in | a vendor **only redacted**, inside the gate; a self-hosted model as it is | | `confidential` | your systems, zones, findings, anything a call declares nothing for | a **self-hosted** model only | A call that declares no class is `confidential`, so a forgotten declaration fails closed. No model decides the class of the content it is about to receive. A **detector** may take a second look and raise a call's class — never lower it — for the place a declaration cannot cover: free text a person typed, declared `public` because the feature is, carrying an IBAN or a sick note. Every call — sent, failed or denied — is recorded as the **SHA-256 digest** of what was sent and never the payload itself, or the audit trail would be the leak. And a check fails your lint when a model is reachable without the gate. The root needs `zod` and nothing else, so a settings page can show the same policy the gate enforces, and it carries `guardModelCall`, the gate around any function that sends a payload to a model. `./ai-sdk` binds it to the Vercel AI SDK (`ai` and `@ai-sdk/provider` optional peers); `./postgres` and `./drizzle` keep the records (`drizzle-orm` an optional peer for the column set). No `node:*`: Node, Bun, Deno, edge runtimes. ```bash npm i @octabits-io/disclosure zod # add ai for ./ai-sdk ``` ## The gate on an AI SDK model ```ts import { guardLanguageModel, isDisclosureDeniedError } from '@octabits-io/disclosure/ai-sdk'; import { generateText } from 'ai'; // Where you build models, and only there. const assessmentModel = guardLanguageModel(vendorModel, { destination: { id: 'mistral-eu', region: 'eu' }, class: 'internal', redact: (text) => pseudonymise(text), // runs inside the gate, on every text the call carries audit: { sink: (record) => disclosureLog.append(record) }, }); const { text } = await generateText({ model: assessmentModel, prompt }); // the call site does not change try { await generateText({ model: guardLanguageModel(vendorModel, { destination, audit }), prompt }); } catch (error) { if (isDisclosureDeniedError(error)) { // error.reason === 'confidential_leaves_the_house' — no class declared, nothing was sent } } ``` `guardEmbeddingModel` is the same gate for `embed` and `embedMany`: the values are the payload, redacted one by one when they must be. What every call goes through: 1. **Decide.** `evaluateDisclosure` from the declared class and the destination. A denial is recorded and thrown as `DisclosureDeniedError` (`key: 'disclosure_denied'`, `reason`, `class`, `destinationId`) before the model is called. 2. **Redact** when the decision says so — `internal` content for a destination outside the house. Your `redact` runs over the system prompt, every message text, tool inputs and tool results. Without a `redact` those calls are denied (`redaction_required`); a file or raw bytes cannot be redacted as text and are denied too (`unredactable_content`). With a detector that counts personal data, a redaction that replaced fewer occurrences than it counted is denied (`redaction_incomplete`): what a detector saw and no redactor recognised would leave in clear. 3. **Digest, call, record.** The payload as sent is digested, the model is called, and the record goes to your sink with `outcome: 'sent'` or `'failed'` (a thrown call may still have left the house). A sink that throws goes to `audit.onSinkError` and never changes the call. Wrap outermost — around the refusal and usage wrappers you already have, and around octaflow's circuit breaker — so the gate sees the call first: `guardLanguageModel(withCircuitBreaker(createRefusalAwareModel(createInstrumentedModel(model, usage))), options)`. The breaker inside the gate means a denial never reaches it and is never counted as a vendor fault; a breaker outside the gate would still not trip on a denial, but its notice would be the gate's to give. ## The gate on any other model call Not every model is an AI SDK model: a vendor's own client, a classifier or a reranker behind `fetch`, a typed-decision API that returns a choice and a probability rather than text. `guardModelCall` is the same gate around the function that sends the payload: ```ts import { guardModelCall } from '@octabits-io/disclosure'; // Where you build models, and only there. const decide = guardModelCall((request: DecisionRequest) => vendor.decide(request), { model: { provider: 'vendor', modelId: 'decision-1' }, // what the record names destination: { id: 'vendor-eu', region: 'eu' }, class: 'internal', redact: (text) => pseudonymise(text), structuralKeys: ['schema'], // your question and its choices: not redacted, still digested audit: { sink: log.sink({ scopeKey: orgId, correlationId: runId }) }, }); const answer = await decide(request); // Result if (!answer.ok) { // answer.error.reason === 'redaction_required' | 'confidential_leaves_the_house' | …; send was never called } ``` The decision, the redaction, the digest and the record are the ones above. The payload is the one argument: `send` receives it as it may leave (redacted when the decision says so) and must send nothing else — whatever it adds from a closure is outside the digest and outside the redaction. The redactor runs over every string in the payload except under `structuralKeys`, which your code writes (a request's schema, its enum of choices, ids); a key named there that carries content leaves unredacted, so name only what your code builds. Bytes or a file in content that must be redacted are denied, in the AI SDK's shape or a vendor's (`media_type`, `mime_type` or `mimeType` beside `data` or `url`), and so is a file inlined as a base64 `data:` URL. The payload is plain data and bytes: an object whose content is not in its own keys (a `Blob`, `FormData`, a `Map`, a stream) or one that refers to itself has no digest, so the call throws a `TypeError` before anything is decided, recorded or sent. A denial is a value, recorded, and `send` is never called; a `send` that throws is recorded as `failed` and rethrown. `operation` on the record defaults to `'generate'`. ## Destinations ```ts { id: 'local-llm', region: 'self-hosted' } // inside the house: receives every class { id: 'mistral-eu', region: 'eu' } // a vendor: public, and internal redacted { id: 'openai-us', region: 'third-country' } // the same rule; the region is for the record { id: 'shared-gpu', region: 'self-hosted', maxClass: 'internal' } // a lower ceiling than the region allows ``` `maxClass` can only narrow. A vendor destination claiming `confidential` is refused when the guard is built, as is an unknown region or class — those are programming errors and throw. ## Routing by class A host with more than one model — a gateway's aliases, a vendor and a model in the house — names the class at the call site and lets the routes pick the model: ```ts import { routeLanguageModels } from '@octabits-io/disclosure/ai-sdk'; const models = routeLanguageModels({ routes: { public: { model: gateway.languageModel('approval-public'), destination: { id: 'approval-public', region: 'eu' } }, confidential: { model: gateway.languageModel('approval-confidential'), destination: { id: 'approval-confidential', region: 'self-hosted' } }, }, redact, audit: { sink }, }); await generateText({ model: models.languageModel('internal'), prompt }); ``` A call takes the route of its own class. A class without one takes the nearest route of a more sensitive class — above, `internal` goes to the confidential route, inside the house. With nothing above it, it takes the nearest route below, and the gate there refuses it and records why. The routes are checked when they are built: a route whose destination would refuse its own class — a confidential route outside the house, an internal one outside it with no `redact` — throws. `resolveDisclosureRoute` and `checkDisclosureRoutes` are the same rules as plain functions at the root, for a settings page. ## A class per field A host that sends structured data knows what each field is, and can say it once, on its Zod schema, instead of at every call: ```ts import { disclosureClassOf, disclosureFieldClasses } from '@octabits-io/disclosure'; const Ticket = z.object({ product: z.string().meta({ disclosure: 'public' }), customer: z.object({ name: z.string(), email: z.string() }).meta({ disclosure: 'internal' }), notes: z.string().meta({ disclosure: 'internal' }).optional(), diagnosis: z.string().meta({ disclosure: 'confidential' }).optional(), }); const cls = disclosureClassOf(Ticket, ticket); // the most sensitive field it carries await generateText({ model: models.languageModel(cls), prompt: render(ticket) }); ``` - The record's class is the most sensitive class among the fields it carries. A field that is `undefined` or `null` carries nothing and counts for nothing; a record with no field carrying content is `public`. - A field without a class is `confidential`, as a call without one is: a field added to the schema and forgotten in the annotation fails closed. So is a key the schema does not name (a loose object). - A class on an object or an array covers everything below it; without one, the walk goes into its fields and items. Wrappers — `optional`, `nullable`, `default` — are looked through. - `disclosureFieldClasses(schema, value)` lists every field that counted, with its path (`customer.email`, `lines[0].price`), its class and whether it was declared: what a settings page or a test shows to explain a record's class. - The field classes decide the declared class; they do not reach into the gate. A detector on the guard still reads the whole call — the gate cannot tell which text of a rendered prompt came from which field. ## Detection: raise a class, never lower it The declared class holds for code paths. It does not hold for text people type: a chat or a form declared `internal`, and an admin password pasted into it. A detector reads the texts a call carries and returns the class they need at least; the gate decides on the more sensitive of the declared class and the detector's raise. ```ts import { combineDetectors, createPatternDetector } from '@octabits-io/disclosure'; const model = guardLanguageModel(base, { destination: { id: 'mistral-eu', region: 'eu' }, class: 'internal', redact, detect: combineDetectors(createPatternDetector(), modelDetector), // a detector over a self-hosted model is the host's audit: { sink }, }); ``` - **It can only make a call stricter.** A detector that misses something leaves the call as declared — as without a detector. One that sees too much costs a redaction or a denial, never a leak. A detector cannot answer `public`; an answer that is not `internal` or `confidential` is a failure. - **It fails closed.** A detector that throws, does not answer within `detectTimeoutMs` (15 s by default; it is aborted through the signal it was given) or answers something unreadable makes the call `confidential`. `onDetectError` hears why. - **It runs only where a raise can change the decision.** Content already `confidential`, a self-hosted destination that takes every class, a call already denied: the detector does not run, and the record says `skipped`. On a vendor route it runs on every call, so its latency is the price of sending content out of the house. - **It reads what the redactor reads, before the redactor:** the system prompt, messages, tool inputs and results, embedding values — not the structural keys, not files or bytes. - **It must run in the house.** It sees the payload before the gate decides where it may go: a detector on a vendor's endpoint has already sent what it was meant to judge. - **It is counted, never quoted.** It returns finding types and counts; the record keeps those, and a finding type that looks like found text (anything beyond letters, digits and `_ . : -`, or longer than 64 characters) fails the detection instead of reaching the log. `createPatternDetector()` finds structured identifiers by their shape and their check digits — IBAN (mod 97), the German tax ID and health insurance number, phone numbers, e-mail addresses — with no dependency, in microseconds, and raises to `internal`. It finds no names, no health data, no passwords: that takes a model. `combineDetectors(...detectors)` runs several over the same texts at once and takes the highest raise; one that fails fails the whole. A detector is any object with an `id`, an optional `version` and `detect(texts, { signal })` — the record names both, so a result can be traced to the detector and the prompt that produced it. A result may say how many occurrences of personal data it found (`identifiers`; the pattern, model and Presidio detectors do); combined detectors report the highest count, not the sum, and where the gate redacts it denies a call whose redaction replaced fewer. ### A model you run as the detector Names, health data, a password, a vulnerability: that takes a language model, and `./ai-sdk` turns one into a detector. It must run in the house — building it with a destination that is not `self-hosted` throws. ```ts import { combineDetectors, createPatternDetector } from '@octabits-io/disclosure'; import { detectorFromModel, guardLanguageModel } from '@octabits-io/disclosure/ai-sdk'; const detect = combineDetectors( createPatternDetector(), detectorFromModel(local.languageModel('qwen3.6-35b-a3b'), { destination: { id: 'detector', region: 'self-hosted' }, providerOptions: { local: { chat_template_kwargs: { enable_thinking: false } } }, // a thinking model, turned down }), ); const model = guardLanguageModel(vendorModel, { destination, class: 'internal', redact, detect, audit: { sink } }); ``` - The model is asked, with `detectorPrompt()`, for personal identifiers and for sensitive content — GDPR Art. 9/10 data, credentials, network facts, vulnerabilities, prices, salaries, procurement — as kinds and counts, never quotes. A sensitive finding raises to `confidential`, an identifier to `internal`. - The record names the detector as the model (`/`, or `id`) and its version as the prompt (`DETECTOR_PROMPT_VERSION`): a result depends on both. - An answer that cannot be read — prose, a cut-off list, a reasoning model's answer without its final channel — throws, and the gate makes the call confidential. Reasoning left in an answer (``, the harmony analysis channel) is skipped. A finding type outside the kinds the prompt names is recorded as `identifier` or `sensitive`, so nothing the model wrote reaches the record. - Pass the model unguarded: its call stays in the house and must not run through the gate it serves. A call longer than `maxChars` (24 000) is read in parts. - `detectorPrompt`, `detectorInput` and `parseDetectorAnswer` are at the root, for a host whose model runtime is not the AI SDK. **Which model is a measurement, not a default.** On the German set in `evals/pii-detection-de` (72 prompts, one laptop), Qwen3.6 35B-A3B leaves 1 prompt too low at 0.65 s, Mistral Small 3.2 24B 3 at 4.8 s; of ten models measured there with a quoting prompt, the weakest left 47. Size and maker predict little. Measure the model you would run, on your own prompts, before you trust it: the repository has the test set and `run_detector.mjs`, which runs exactly this prompt. ### GLiNER behind Presidio's analyzer API A model misses names now and then — a lower-case `müller`, a bare `Ronny` — and GLiNER, a small span model, misses what a model finds. Together they left no German text of the test sets too low; on long business text, GLiNER also raised every harmless one (below). GLiNER has no HTTP interface of its own; Presidio's analyzer can host it, and `infra/single-host` in the repository ships a smaller service that speaks the same API — GLiNER in Rust on ONNX Runtime, with no Python in the image, answering span for span what Presidio hosting GLiNER answers on every text of the eval. `./presidio` is a detector and a redactor over that REST API (`fetch`, no dependency), whichever of the two answers it: ```ts import { createPresidioDetector } from '@octabits-io/disclosure/presidio'; const detect = combineDetectors( createPatternDetector(), detectorFromModel(localModel, { destination: inside }), createPresidioDetector({ url: 'http://presidio-analyzer:3000', destination: inside, version: 'presidio-gliner-1' }), ); ``` - Each text of a call is one `/analyze` request (`language` default `de`). An entity type in `sensitiveEntities` raises to `confidential`, any other to `internal`; the record keeps the entity types with counts. - The analyzer is configured, not this package. The configuration it was measured with is `evals/pii-detection-de/presidio-gliner.yaml` in the repository: no NLP engine, GLiNER as the only recognizer, told the same categories as the model, each label mapped to an entity type — `PRESIDIO_SENSITIVE_ENTITIES` are its sensitive ones. Presidio's own pattern recognizers add little beside a model and `createPatternDetector`, and more false alarms. - What it costs: on short prompts GLiNER flags about one harmless German prompt in five, where the models alone flag none. On long business text — a pasted e-mail thread, a policy, release notes — it reads ordinary nouns as its sensitive labels ("Lieferung" as procurement, public rates as prices, at scores as high as its true findings) and raised every harmless long text measured to `confidential`. Friction, not a leak, but where vendor routes should carry pasted documents it closes them; detect with the patterns and a model, and use GLiNER to redact. Some 60 ms per short prompt, 0.5 s per long text, on a laptop CPU. - Building it with a destination that is not `self-hosted` throws, and so does a URL that is not `http(s)`. A failed request or an answer that is not a list of results throws, and the gate makes the call confidential. A detector bounds even a model you would not otherwise trust: it can only fail to raise — the call then goes out as declared, as with no detector — or raise too far, which costs friction. It cannot make a call more permissive, and served with no route out it has nowhere to send what it reads. ## Inside a LiteLLM proxy Where the prompts come from no code of yours — people in a chat client, tools pointed at a company model proxy — the gate sits in the proxy. `./litellm` is a fetch handler for LiteLLM's Generic Guardrail API: LiteLLM posts each request's texts to it before sending them on, and every request is decided like a guarded model call. ```ts import { createLiteLlmGuardrailHandler } from '@octabits-io/disclosure/litellm'; const guardrail = createLiteLlmGuardrailHandler({ destinations: [{ id: 'mistral-eu', region: 'eu' }, { id: 'local', region: 'self-hosted' }], models: { 'chat-eu': 'mistral-eu', 'chat-local': 'local' }, // what LiteLLM's model_list routes each alias to classOf: ({ teamAlias }) => (teamAlias === 'marketing' ? 'internal' : undefined), // undeclared → confidential redact, detect, token: process.env.GUARDRAIL_TOKEN, audit: { sinkFor: (info) => log.sink({ correlationId: info.callId }) }, }); // Serve it at /beta/litellm_basic_guardrail_api — e.g. toNodeHandler(guardrail) from @octabits-io/server/node. ``` ```yaml # LiteLLM config.yaml — one guardrail, on every request guardrails: - guardrail_name: disclosure litellm_params: guardrail: generic_guardrail_api mode: pre_call default_on: true api_base: http://disclosure-guardrail:3000 api_key: os.environ/GUARDRAIL_TOKEN # fail_on_error: true and unreachable_fallback: fail_closed are the defaults — keep them ``` - **The answer:** denied → `BLOCKED` with the reason, which the client gets as an error; anything redacted → `GUARDRAIL_INTERVENED` with the texts as they may leave, which LiteLLM sends in place of the originals; otherwise `NONE`. - **The class comes from who asks**: `classOf` gets the virtual key, its team, user and organisation as LiteLLM reports them. A request `classOf` declares nothing for is `confidential` — blocked on every route outside the house, passed to a self-hosted model. - **The destination comes from the alias.** LiteLLM names the model alias in each request; `models` says where the proxy sends each one, because the guardrail cannot see the routing. An alias `models` does not name is blocked. What the proxy does after the guardrail answered — a fallback — the guardrail never sees: hold the two files together with `checkLiteLlmConfig`, below. (Guardrails per model are a LiteLLM Enterprise feature; with them, each guardrail entry can name its destination in `additional_provider_specific_params.destination`, and `models` is left out.) - **What LiteLLM cannot take back rewritten** — images, tool calls — is denied when the decision says redact. A detector reads the texts. - **It fails closed.** A wrong token (`x-api-key`, as LiteLLM sends its `api_key`), a body that is not a guardrail request, a `post_call` entry — a model's answer is not a disclosure — and a redactor that throws are answered with an error status, which LiteLLM, failing closed, turns into a blocked request. - **Call the detector's model directly**, not through the proxy: through it, the detector's own call comes back to the guardrail. - **Check what LiteLLM keeps.** Its own logs and spend tracking can store the original prompt; the disclosure record never does, but it cannot make the proxy's logs forget. - The request shape is LiteLLM's and not a documented contract; the handler is held against LiteLLM 1.103.0 in a Testcontainers suite, on a Postgres with keys and teams, as in production. `liteLlmClassOf` builds `classOf` from data — keys by alias, teams by alias or id, organisations by id; the most specific entry wins, and a request none of them names is undeclared: ```ts classOf: liteLlmClassOf({ teams: { marketing: 'internal', docs: 'public' }, keys: { 'press-bot': 'public' } }), ``` ### Pseudonyms the answer is read back with A bare `[PERSON]` for every name makes two people one, and the answer comes back with placeholders in it. With `pseudonymize: true` the redactors write through a table per call instead: each value its own numbered placeholder — `[PERSON_1]`, `[PERSON_2]`, `[EMAIL_1]`, the same one wherever it occurs — and a second LiteLLM entry sends each answer back, which the handler restores from that call's table before LiteLLM hands it to the client. ```yaml guardrails: - guardrail_name: disclosure # decides, as above litellm_params: { guardrail: generic_guardrail_api, mode: pre_call, default_on: true, api_base: http://disclosure-guardrail:3000, api_key: os.environ/GUARDRAIL_TOKEN } - guardrail_name: disclosure-restore # reads the answer back litellm_params: { guardrail: generic_guardrail_api, mode: post_call, default_on: true, api_base: http://disclosure-guardrail:3000, api_key: os.environ/GUARDRAIL_TOKEN } ``` - **Numbers hold across a conversation without being stored.** They are given in the order values first occur, and a chat resends its whole history in the same order every turn: the person who was `[PERSON_1]` stays `[PERSON_1]`. - **The table never leaves the house** and is kept in memory, by `litellm_call_id`, until the answer comes back (`ttlMs`, default ten minutes; `maxCalls`, default 10 000). Behind several guardrail replicas, route a call's request and answer to the same one. - **A streamed answer keeps its placeholders.** LiteLLM runs a `post_call` guardrail on a stream after the client has it — proven against 1.103 by the integration suite. Turn streaming off for the models that leave the house (Open WebUI: *Stream Chat Response* off on the model, which wins over the chat's and the account's). - **It fails toward placeholders, never toward a leak:** an expired table, a restart, a stream — the reader sees `[PERSON_1]`, the vendor never saw more. - The redactors take the table from the call's `RedactionContext` (`createPatternRedactor`, `createPresidioRedactor` and `combineRedactors` do); one of your own that ignores the context replaces as it always did. `createPseudonymTable` is exported for a gate of your own. `checkLiteLlmConfig(…, { pseudonymize: true })` wants an entry that runs `post_call`; without `pseudonymize`, such an entry is an error, because the guardrail refuses answers and LiteLLM fails every call on that. In the service's file it is `redact.pseudonymize: true` (or `{ ttlSeconds, maxCalls }`). ### Routes the gate chooses: "Auto" A person who picks the model has to know what their text is. With `routes`, the gate picks: a request for `auto` goes to the first alias the gate lets it reach — `auto: [chat-eu, chat-local]` sends an internal text to the vendor, pseudonymised, and a password to the model in the house, answered instead of refused. LiteLLM cannot route on a guardrail's answer — the Generic Guardrail API takes back texts and images, never a model or a tag — and its own auto router and tag routing decide after the guardrail ran for the alias asked for: proven against 1.103, a password sent to `auto` reached the vendor, and the guardrail saw `auto`. So the package ships a LiteLLM plugin, `plugins/litellm/disclosure_router.py`, that runs before the guardrail, asks it at `POST /route` and rewrites the request's model; LiteLLM routes on that, and the guardrail entry then decides for the alias chosen, reusing the decision `/route` made for the call — one detection, one record per destination tried. ```yaml # LiteLLM config.yaml — the router first: guardrails run in the order listed guardrails: - guardrail_name: disclosure-router litellm_params: { guardrail: disclosure_router.DisclosureRouter, mode: pre_call, default_on: true } - guardrail_name: disclosure litellm_params: { guardrail: generic_guardrail_api, mode: pre_call, default_on: true, api_base: http://disclosure-guardrail:3000, api_key: os.environ/GUARDRAIL_TOKEN } # plus DISCLOSURE_ROUTE_URL=http://disclosure-guardrail:3000/route and GUARDRAIL_TOKEN in LiteLLM's environment, # and disclosure_router.py next to config.yaml (it ships in this package under plugins/litellm/) ``` - **A refused destination is recorded:** the password above leaves two records, `denied` for the vendor and `sent` for the house. - **The wrong order fails closed:** a guardrail that sees `auto` blocks it — a route alias has no destination of its own. `checkLiteLlmConfig(…, { routes })` warns about a router listed after the guardrail or missing. - **The class still comes from the person's key and team:** the plugin passes them on, and the guardrail entry reads them from LiteLLM as before. - A route alias may also be served in `model_list`, so clients list it; its deployment should be in the house, since it is never reached while the router runs first. ### What the guardrail cannot see: the proxy's routing and its keys LiteLLM runs the guardrail **once, for the alias a client asked for**. A fallback — `chat-local` is down, the proxy tries `chat-eu` — sends the same text on, and the guardrail decided for `chat-local`: in the house, so the text passed unredacted. Held against LiteLLM 1.103 in the integration suite, and it happens two ways: - **a fallback in the proxy's file** (`fallbacks`, `context_window_fallbacks`, `content_policy_fallbacks`, `default_fallbacks`, under `router_settings` or `litellm_settings`); - **a fallback the client names in its own request** — `{"model": "chat-local", "fallbacks": ["chat-eu"]}` — with nothing in the file at all. The guardrail is not told. One switch closes both: **`metadata.disable_fallbacks: true` on every key**. LiteLLM reads it from the key and lets no request override it, and it stops the configured fallbacks too. Two checks hold the rest: ```ts import { checkLiteLlmAccess, checkLiteLlmConfig } from '@octabits-io/disclosure/litellm'; const problems = checkLiteLlmConfig(parse(readFileSync('config.yaml', 'utf8')), { destinations, models }); // + checkLiteLlmAccess({ keys, teams, guardrailName: 'disclosure' }) over GET /key/list and /team/list ``` `checkLiteLlmConfig` — the proxy's `config.yaml` against the guardrail's declaration. Errors, where a request can reach a destination nobody decided for, or the guardrail may not run: - a deployment of an alias declared self-hosted that goes outside the house: an `api_base` that is not loopback, a private address, a single-label name (a Compose service) or `.internal`/`.local`/`.lan`/`.home.arpa`/`.svc` — or no `api_base` on a hosted provider (`insideHosts` names your own domains); - a fallback to a destination that may receive less, or is farther away (into the house is a warning: the record would name the wrong destination); - an alias served and not declared, a wildcard `model_name`, a `model_group_alias` to another destination; - client-side credentials (`allow_client_side_credentials`, `configurable_clientside_auth_params`): a client that sets `api_base` picks the host; - the guardrail entry not `generic_guardrail_api`, not `pre_call` (`during_call` sends while it decides), not `default_on`, or failing open. Warnings: logging callbacks and `store_prompts_in_spend_logs`, which read the original text; no `master_key`, so no request has a team to be declared by. `checkLiteLlmAccess` — what lives in LiteLLM's database, not the file: every key that is not blocked has `disable_fallbacks: true`, and no key or team opts out of the guardrail (`disable_global_guardrails`, `opted_out_global_guardrails`). A request cannot opt out on its own; LiteLLM reads those flags from the key and the team only. ### Who sent it: the actor The record names a destination and a class, and for a proxy the class comes from who asked. `liteLlmActorOf(info, level)` turns what LiteLLM reports into the log's `actor` — `team:support`, `key:open-webui`, `user:ada@example.com` — at the level you choose, falling back to the next coarser one: - `team` names the basis of the declared class, and no person; - `user` is the end user a client names in a header LiteLLM maps (`general_settings.user_header_mappings`, e.g. Open WebUI's `X-OpenWebUI-User-Email` with `ENABLE_FORWARD_USER_INFO_HEADERS=true`). A key or a user usually is a person. The log is then personal data, and in a workplace a record of what each employee sent is monitoring — in Germany a matter for the works council (§ 87(1) no. 6 BetrVG). Settle that before you record more than the team. ### As a service: `litellm-guardrail` For an operator who runs LiteLLM and writes no code, the package's second command serves the handler, configured by a file. `infra/single-host` in the repository runs it as the `gateway` profile, beside LiteLLM and GLiNER behind Presidio's analyzer API, and carries a commented `guardrail.yaml`. ```bash GUARDRAIL_TOKEN=… DATABASE_URL=postgres://… litellm-guardrail --config guardrail.yaml --port 8080 ``` ```yaml destinations: [{ id: local, region: self-hosted }, { id: vendor-eu, region: eu }] models: { chat-local: local, chat-eu: vendor-eu } # every alias in LiteLLM's model_list classes: { teams: { marketing: internal, docs: public } } # anyone else is confidential inside: local # where the detectors and the redactor run detect: patterns: true model: { baseURL: http://model:11434/v1, model: qwen3.6-35b-a3b } # presidio: { url: http://gliner:3000 } # GLiNER as a detector too: see above redact: patterns: true # first: what the patterns find, an IBAN by its shape presidio: { url: http://gliner:3000, entities: [PERSON, LOCATION, DATE_OF_BIRTH] } ``` - `POST /beta/litellm_basic_guardrail_api` is the guardrail; `GET /health/live` and `GET /health/ready` are for the orchestrator. Ready means the log answers and the detector model has answered once: the command asks it at start and every 10 s until it does, because a cold model's first answers can outlast `detect.timeoutMs` and every request that waited for one would be made confidential. - The file is checked at start, every problem listed: an alias to an undeclared destination, a class that is none, an unknown field, a detector or redactor without a self-hosted `inside`. - `--litellm config.yaml` holds the proxy's file to this one before it starts (`checkLiteLlmConfig`): an error stops the service, a warning is logged. Mount the proxy's file read-only beside its own. - `litellm-guardrail check --config guardrail.yaml --litellm config.yaml [--keys-from http://litellm:4000]` runs the same and exits 1 on an error; with `--keys-from` and `LITELLM_MASTER_KEY` it also reads every key and team from the admin API (`checkLiteLlmAccess`). Run it after a change to either file, and after keys are made. - `audit.actor` — `team` (the default), `key`, `user` or `none` — is what each record names as its actor; see *Who sent it* above before you choose `user`. - Each record goes to `disclosure_log` (`audit.table`) with LiteLLM's call id as `correlationId` and the actor; the table is created on start, and a table an earlier version created gets the columns added since, so give the service a database of its own. `--audit stdout` writes JSON lines instead, for a first boot. - `trace: { url }` POSTs each decision's trace (above) to a URL as JSON — a monitor in the house; off unless set. It is sent without waiting: a monitor that is slow or gone never holds up a request, and a trace that cannot be delivered is logged and dropped. - Secrets come from the environment: `GUARDRAIL_TOKEN` (required), `DATABASE_URL`, `DETECTOR_API_KEY`. - It loads optional peers as it needs them: `yaml`, `pg`, and `@ai-sdk/openai-compatible` with `ai` for a detector model. ### Redactors `createPatternRedactor()` replaces what the pattern detector counts — so a call the patterns raised leaves without them — and takes an IBAN by its shape alone, since a mistyped account number is still one: `[IBAN]`, `[TAX_ID_DE]`, `[HEALTH_INSURANCE_ID_DE]`, `[EMAIL]`, `[PHONE]`. `createPresidioRedactor` in `./presidio` replaces every entity the analyzer finds with its type (`[PERSON]`), overlapping entities once, and leaves alone what an earlier redactor put in; Presidio's offsets count code points, and it cuts where Presidio meant. A request that fails throws, and the gate denies the call. `combineRedactors` runs them in order — the exact one first, so what it replaced is not there for the next one to miss: ```ts const redact = combineRedactors( createPatternRedactor(), createPresidioRedactor({ url: 'http://presidio-analyzer:3000', destination: inside, entities: ['PERSON', 'LOCATION', 'DATE_OF_BIRTH'] }), ); ``` Limit Presidio's redaction to personal data with `entities`: content in a sensitive category makes the call `confidential`, which is denied, never redacted. With GLiNER as the recognizer, expect some pronouns and role nouns to go too ("ich" → `[PERSON]`) — readable, not pretty. A redactor only replaces what it finds, so the gate holds the redactions against what the detectors counted and denies the call when they fall short (`redaction_incomplete`) — a passport number or a date of birth `3.7.85` that the model found and neither the patterns nor GLiNER recognised is blocked, not sent. The patterns run first so that the identifiers they know are never the reason. ## The record ```ts { destinationId: 'mistral-eu', region: 'eu', provider: 'mistral.chat', modelId: 'mistral-large-latest', operation: 'generate', // 'generate' | 'stream' | 'embed' | 'tool-result' (an MCP tool's result, @octabits-io/server/mcp) class: 'internal', classDeclared: true, outcome: 'sent', // 'sent' | 'failed' | 'denied' reason: undefined, // the denial reason, for 'denied' payloadDigest: 'sha256:9f2c…', payloadBytes: 1824, redacted: true, redactions: 3, detection: { // present when the guard has a detector detectorId: 'patterns+qwen', detectorVersion: '1+prompt-1', status: 'raised', // 'clear' | 'raised' | 'skipped' | 'failed' raisedFrom: 'public', // the class before the raise findings: [{ type: 'iban', count: 1 }], }, at: new Date(), } ``` `class` is the class the decision was made on, after any raise; `classDeclared` still says whether the call site declared one. `disclosureRecordSchema` is the Zod schema of it. The digest is over a canonical JSON encoding (`canonicalJson`: sorted keys, bytes as base64), so the same content always yields the same digest — keep the content you disclosed elsewhere and `digestPayload` proves later that this is what was sent. The sink is any function; the disclosure log below is the one that keeps records where an audit can read them back. ### The trace: how a call was decided, for someone who has to understand it The record says what was decided and never quotes the text. Someone tuning the detectors in a pilot, or explaining a decision to a security officer, needs the other half: which detector found what, where in the text, what a detector model actually answered, what each redactor replaced, and how long each step took. `trace` on any guard (and on the LiteLLM handler) hears that, once per call, when the gate has decided: ```ts const model = guardLanguageModel(vendorModel, { destination, class: 'internal', redact, detect, audit, trace: (trace) => console.log(JSON.stringify(trace)) }); ``` ```ts { destinationId: 'vendor-eu', region: 'eu', modelId: 'mistral-large-latest', declared: 'internal', classDeclared: true, class: 'confidential', outcome: 'denied', reason: 'confidential_leaves_the_house', ms: 1240, detection: { detectorId: 'patterns+detector.chat/qwen', status: 'raised', ms: 1236, detectors: [ { detectorId: 'patterns', started: true, ms: 0, result: { findings: [{ type: 'email', count: 1 }], … }, matches: [{ type: 'email', text: 'ada@example.com', textIndex: 1, start: 13, end: 28 }] }, { detectorId: 'detector.chat/qwen', started: true, ms: 1236, result: { raise: 'confidential', … }, answer: '{"findings":[{"type":"credentials","kind":"sensitive","count":1}]}' }, ], }, // redaction: { ms, redactions, redactors: [{ redactorId: 'presidio', textIndex, ms, matches: [{ type: 'PERSON', text: 'Ada Krüger', placeholder: '[PERSON_1]', score: 0.94 }] }] } } ``` - **A trace quotes the text it found.** That is its use, and why it is never part of the record: hand it only to something in the house — a console during a pilot, a monitor beside the gateway — and keep it no longer than it is looked at. Nothing is traced unless `trace` is set. - **Each part of a combined detector reports itself**, with its own time. A part with `started` and no `ms` had not answered when the gate stopped waiting — the reason the call was made confidential; `detection.error` says why detection failed. A skipped detection says why (`skipped`). - **The detectors and redactors here fill it in**: the patterns and Presidio with the matched span, offset and score; a model detector with its answer before it was read (at most `MAX_TRACE_ANSWER` characters). A detector of your own gets `context.trace` and may report the same; a redactor gets it on its `RedactionContext`. - **What the trace callback throws is ignored**: it never changes a call. - The LiteLLM handler's trace (`LiteLlmTrace`) adds the request: `path` (`route` or `guardrail`), the call id, the alias, the key, team and user, the candidate a route tried, and `reused` when the guardrail entry took the decision `/route` made. ## The disclosure log A table of every gate decision, with the scope it belongs to and a `correlationId` — your id for what the call served, a workflow run or a proposal — so an audit goes from a reviewed proposal to every model call behind it. The root holds the contract and an in-memory store; `./postgres` is the store over a structural executor (`pg`, PGlite, postgres.js) plus `disclosureLogDdl()`; `./drizzle` is `disclosureLogColumns`, the table for drizzle-kit (`drizzle-orm` optional peer). There is no Drizzle store: a log is one insert and one select, so the SQL store is the store — hand it your pool. ```ts import { createPostgresDisclosureLogStore } from '@octabits-io/disclosure/postgres'; const log = createPostgresDisclosureLogStore({ db: pool, scope: { column: 'org_id' } }); // your scope column const model = guardLanguageModel(base, { destination, class: 'internal', redact, audit: { sink: log.sink({ scopeKey: orgId, correlationId: runId }), onSinkError: (e) => logger.error(e) }, }); const page = await log.list({ scopeKey: orgId, correlationId: runId }); // newest first const period = await log.list({ scopeKey: orgId, since: quarterStart, until: quarterEnd, before: page.value.nextBefore }); ``` - `record(record, context?)` validates against `disclosureRecordSchema` and writes one row — `disclosure_log_invalid` when it does not match, nothing written. `sink(context)` is the same write as a guard sink; it throws on a failed write, so the guard's `onSinkError` hears of it and the call is never changed. - Scope: `{ column, value }` stamps and filters by `value`; `{ column }` stamps each record's `scopeKey` (`disclosure_log_unscoped` without one) and reads across scopes unless `list` gets a `scopeKey` — or a row-level-security policy on the column does the filtering. Omit `scope` in a single-scope deployment. The scope column, its foreign key and its policy are yours. - `actor` (optional, in the context beside `correlationId`) names who the call came from, for a host where nothing the correlation id points at keeps it — a proxy's team, key or user. Where a run or a proposal already names its actor, leave it out. - `list` filters by `correlationId`, `actor`, `destinationId`, `outcome` and a half-open `since`/`until` period; pages of 50 by default (at most 500), newest first, with `nextBefore` as the keyset cursor. - A log created before detection gains its five columns from `disclosureLogDetectionDdl()`, and one created before the actor gains it from `disclosureLogActorDdl()` (`ALTER TABLE … ADD COLUMN IF NOT EXISTS`, both idempotent); a Drizzle host generates the same from `disclosureLogColumns`. - A guarded model built once for the process has no per-request context to hand `sink()`: write through `log.record(record, currentContext())` from a sink of your own, with the context from your request scope. ## Nothing reaches a model around the gate The gate is a control only if there is no other way to a model. `disclosure-check` fails when a provider import exists outside the files you name as your guarded factory: ```json { "scripts": { "lint": "disclosure-check --guarded src/ai/models.ts" } } ``` A model import is any `@ai-sdk/*` provider (not `@ai-sdk/provider` or `@ai-sdk/provider-utils`) and `ai/test`'s mock models, static or dynamic. `--module ` adds your own — a local runtime such as `node-llama-cpp` is a destination too, and so is the client module a `guardModelCall` wraps: name it with `--module` and the file that builds the guard with `--guarded`. Two more ways reach a model with no provider import, and both are findings: the SDK's own `gateway` / `createGateway` imported from `ai`, and a string model id (`model: 'openai/gpt-4o'`) inside an SDK call, which the SDK resolves through its global default provider — Vercel's gateway unless the host set one. A host that wants string ids sets `globalThis.AI_SDK_DEFAULT_PROVIDER` to a provider whose models come out of the guarded factory, and passes `--allow-string-models`. Directories default to `src`; test files are skipped unless `--include-tests`; a `--guarded` path that does not exist fails the check, so the allowlist cannot go stale. `findUnguardedModelImports({ files, guarded })` is the same check as a function, exported from the command's module. It is a lint over source text, not a proof: a model id held in a variable, or a provider reached through a namespace import, is not seen. ## The policy without a model runtime ```ts import { evaluateDisclosure, DISCLOSURE_CLASSES } from '@octabits-io/disclosure'; evaluateDisclosure({ class: 'internal', destination: { id: 'mistral-eu', region: 'eu' }, canRedact: true }); // → ok({ class: 'internal', classDeclared: true, redact: true }) ``` ## Exports | Path | What | |---|---| | `.` (any model call) | `guardModelCall`, and the `ModelCallGuardOptions`, `GuardedModelCall`, `DisclosureGuardOptions`, `Redactor` and `RedactedText` types | | `.` | `DISCLOSURE_CLASSES`, `DISCLOSURE_REGIONS`, `DEFAULT_DISCLOSURE_CLASS`, `evaluateDisclosure`, `isMoreSensitive`, `disclosureDenied`, `disclosureDestinationSchema`, `disclosureRecordSchema`, `DISCLOSURE_DENIAL_REASONS`, `DISCLOSURE_OPERATIONS`, `DISCLOSURE_OUTCOMES`, `digestPayload`, `canonicalJson`, and their types | | `.` (detection) | `detectorPrompt`, `detectorInput`, `parseDetectorAnswer`, `UnreadableDetectorAnswer`, `DETECTOR_PROMPT_VERSION`, `DETECTOR_FINDING_TYPES`, `createPatternDetector`, `combineDetectors`, `createPatternRedactor`, `combineRedactors`, `DETECTION_RAISES`, `DETECTION_STATUSES`, `PATTERN_FINDING_TYPES`, `DEFAULT_DETECT_TIMEOUT_MS`, `MAX_DETECTION_FINDINGS`, `detectionResultSchema`, `detectionFindingSchema`, `detectionFindingTypeSchema`, `disclosureDetectionSchema`, and the `Detector`, `DetectionContext`, `DetectionResult`, `DetectionRaise`, `DetectionFinding`, `DetectionStatus`, `DisclosureDetection`, `PatternFindingType` and `PatternRedactorOptions` types | | `.` (fields) | `disclosureClassOf`, `disclosureFieldClasses`, `DISCLOSURE_META_KEY`, the `DisclosureFieldClass` type | | `.` (routes) | `resolveDisclosureRoute`, `checkDisclosureRoutes`, and the `DisclosureRoutes`, `ResolvedDisclosureRoute`, `DisclosureRouteProblem` and `DisclosureRoutesInvalid` types | | `.` (the trace) | `MAX_TRACE_TEXT`, `MAX_TRACE_ANSWER`, `MAX_TRACE_MATCHES`, and the `DisclosureTrace`, `DetectorTraceEvent`, `RedactorTraceEvent` and `TraceMatch` types | | `.` (the log) | `createInMemoryDisclosureLogStore`, `createDisclosureLogStore`, `toDisclosureLogRow`, `toDisclosureLogQuery`, `disclosureLogFailure`, `DISCLOSURE_LOG_DEFAULT_LIMIT`, `DISCLOSURE_LOG_MAX_LIMIT`, and the `DisclosureLog*` types | | `./ai-sdk` | `guardLanguageModel`, `guardEmbeddingModel`, `routeLanguageModels`, `detectorFromModel` and its `ModelDetectorOptions`, `DisclosureDeniedError`, `isDisclosureDeniedError`, the `LanguageModelRoute`, `ClassRoutedModelsOptions` and `ClassRoutedModels` types, and the `DisclosureGuardOptions`, `Redactor` and `RedactedText` types (the root's, re-exported) | | `./presidio` | `createPresidioDetector`, `createPresidioRedactor`, `PRESIDIO_SENSITIVE_ENTITIES`, the `PresidioDetectorOptions` and `PresidioRedactorOptions` types | | `./litellm` | `createLiteLlmGuardrailHandler`, `liteLlmClassOf`, `liteLlmClassMapSchema`, `checkLiteLlmConfig`, `checkLiteLlmAccess`, `liteLlmActorOf`, `LITELLM_ACTOR_LEVELS`, the `LiteLlmGuardrailOptions`, `LiteLlmRequestInfo`, `LiteLlmClassMap`, `LiteLlmConfigProblem`, `CheckLiteLlmConfigOptions`, `LiteLlmKeyInfo`, `LiteLlmTeamInfo`, `LiteLlmActorLevel` and `LiteLlmTrace` types | | `./postgres` | `createPostgresDisclosureLogStore`, `disclosureLogDdl`, `disclosureLogDetectionDdl`, `disclosureLogActorDdl`, `SqlExecutor` | | `./drizzle` | `disclosureLogColumns` | | `disclosure-check` | the command (`bin`) | | `litellm-guardrail` | the LiteLLM guardrail as a service (`bin`), configured by a file | ## What it does not do - **Classify.** The call site declares the class; the gate enforces it. A classifier model deciding what may leave would be the thing the gate guards against. A detector raises a class; it never sets one. - **Redact.** It runs your redaction, where no caller can skip it — over the system prompt, the messages, tool inputs and results, and the provider options (a vendor's own fields carry content as plain text; they are in the digest too). A tool's schema is not content and is not redacted. `@octabits-io/pii`'s pseudonyms are one redaction; a list of your own system names is another. - **Cover tool definitions and response schemas.** Those are written by your code, not carried in from content, and pass as they are. - **Record who called.** The record names the destination, not the caller. The log keeps the scope and a correlation id beside it; the actor belongs to what the correlation id points at — the run or the proposal, and the ledger entry that applied it. ## Stability 0.x: minors may break and say so; patches never do. The three classes, the rule between them and the record's fields are the contract.