`@octabits-io/intake`

Spreadsheet import as a reviewable proposal. A person uploads a CSV or an .xlsx file; what comes back is not “the import ran” but a Proposal: a create for every new row, field updates for every record that changes, each value cited to its row and column in the file, and every row that was left out named with its reason. A reviewer sees “142 new, 9 changed, 3 rejected”, accepts or rejects row by row, and the host applies what was accepted as one ledger entry — so the whole import reverts as one.

pnpm add @octabits-io/intake @octabits-io/proposal zod      # + read-excel-file for ./xlsx

Subpath	Runs on	Deps	Contents
`.`	any runtime	`@octabits-io/result`; `@octabits-io/proposal` and `zod` peers	the table and its limits, the cell schemas, the column mapping, `validateRows`, `planImport`
`./csv`	any runtime	—	`readCsv`, `parseCsv`, `detectDelimiter`
`./xlsx`	any runtime	`read-excel-file` (optional peer, its `universal` build)	`readXlsx`

No node:* anywhere: the readers take bytes (ArrayBuffer, Uint8Array, Blob), never a path, so the same code reads an upload in a route handler, a worker or a browser.

The pipeline

import { readXlsx } from '@octabits-io/intake/xlsx';
import { importKeys, planImport, suggestMapping, textCell, numberCell, validateRows, type IntakeTarget } from '@octabits-io/intake';

const contacts: IntakeTarget = {
  collection: 'contact',
  key: 'email',                                     // a row whose e-mail you have becomes field updates
  fields: {
    email: { label: 'E-Mail', aliases: ['Mail'], required: true, schema: textCell() },
    name: { label: 'Name', schema: textCell({ maxLength: 200 }) },
    revenue: { label: 'Umsatz', description: 'yearly revenue in euro', schema: numberCell({ decimal: ',' }) },
  },
};

const table = await readXlsx(upload, { name: upload.name });     // Result: refused over a limit, never truncated
if (!table.ok) return table;

let mapping = suggestMapping(table.value.headers, contacts);     // by name, label and alias
// … completeMapping for what is left, then a person confirms: mapping = mapColumn(mapping, 3, 'revenue')

const rows = validateRows(table.value, mapping, contacts);       // refused when a required field has no column
if (!rows.ok) return rows;

const existing = await findContactsByEmail(importKeys(rows.value, contacts));   // your query: Map<key, { id, label?, values }>
const { proposal, summary } = planImport({
  scope: `org:${orgId}`,
  target: contacts,
  rows: rows.value,
  source: table.value.source,
  sourceId: storedUpload.id,                        // the file as you keep it, so a citation resolves
  existing,
});
await pendingProposals.record({                       // reviewed like any other proposal
  kind: 'contact-import',
  proposal,
  scopeKey: `org:${orgId}`,
  summary: `${summary.created} new, ${summary.updated} changed, ${summary.rejected} rejected`,
});

From there it is the ordinary loop, unchanged: the review card shows proposal.operations grouped new / changed with proposal.skipped beside them, the decision comes back, and the host applies it — resolveDecision, detectDrift on the updates’ guards, the writes, one ledger.record({ operations, created }). Reverting that entry deletes what the import created and writes back what it changed (invertOperations).

Limits

An upload is untrusted. Every limit refuses the file; none truncates it or splits it, because one import is one proposal and one revert.

Limit	Default	Checked
`maxBytes`	10 MiB	before reading; on a `Blob`, before its bytes are loaded
`maxRows`	5,000 data rows	while parsing; a CSV stops reading at the first row over
`maxColumns`	200	on the table
`maxUnzippedBytes`	100 MiB	for a workbook, from the zip’s central directory before anything is unpacked; ZIP64 is refused

Pass limits: { … } to a reader to change them. The unzipped-size check is a real bound, not a hint: read-excel-file’s universal build unzips with fflate, which inflates each entry into a buffer of its declared size, so an entry that lies about its size is cut off rather than grown.

Reading

CSV (readCsv): RFC 4180 quoting, any line ending, a byte-order mark dropped. The separator is detected from the header line (; is what Excel writes in German-speaking locales) unless you pass delimiter. Bytes that are not valid UTF-8 are read as Windows-1252, Excel’s other CSV encoding. Every cell is text.

xlsx (readXlsx): one sheet, by name, by 1-based position, or the first. Cells come as the workbook stores them: numbers, booleans, text trimmed, dates as ISO strings (a date without a time as YYYY-MM-DD). A formula cell is the value the spreadsheet application last computed; nothing is evaluated.

Both return an IntakeTable: the header (the first row with anything in it), the data rows with the row number a person sees in their spreadsheet, blank rows dropped, and source with the file name, the sheet and the sha256: digest of the exact bytes read. Another parser plugs in through tableFromGrid(grid, source) and gets the same shape and limits.

Mapping, and what a model sees

suggestMapping matches each header against every field’s name, label and aliases, ignoring case, accents, ß/ss and punctuation (E-Mail, email, e mail are one). Each field maps once, leftmost column first.

What is left can go to a model. mappingRequest(mapping, target) builds the whole payload: the headers nothing matched and the fields still open — no cell, ever. That is a property of the function, not of a prompt. completeMapping sends it through your function, which is where your disclosure gate goes:

import { guardModelCall } from '@octabits-io/disclosure';
import { completeMapping, formatMappingPrompt, mappingAnswerSchema } from '@octabits-io/intake';

const suggest = guardModelCall(
  async (request) => (await generateObject({ model, schema: mappingAnswerSchema, prompt: formatMappingPrompt(request) })).object,
  {
    model: { provider: 'mistral', modelId: 'small' }, destination, class: 'internal', redact, audit,
    structuralKeys: ['fields'],                       // your target's field list: written by your code, not by the file
  },
);
const completed = await completeMapping(mapping, contacts, suggest);   // a denial comes back as the error, mapping unchanged

The headers came from the file, so they go through your redactor; an answer about a header the redactor rewrote names a header the file does not have, and is dropped like any other unknown one. The answer is checked, not trusted: an unknown header or field, a field already taken, or one two headers claim is dropped. Columns the model mapped say by: 'model'; mapColumn records a person’s choice as by: 'person' and moves a field off any other column. Show the mapping before validateRows — the reviewer of the rows should not be the first person to learn that “Umsatz” was read as the customer number.

Rows

validateRows runs every mapped cell through its field’s Zod schema. A row passes whole or is rejected whole, with every issue named (row 6: Umsatz: not a number). A blank cell is left out, never null: an import fills and changes values, it never clears one; a blank required cell rejects the row.

The cell schemas in the root take what spreadsheets actually contain:

Schema	Takes
`textCell({ maxLength? })`	anything, as trimmed text
`numberCell({ decimal? })`	workbook numbers, `1,234.5`, and with `decimal: ','` `1.234,5`
`integerCell({ decimal? })`	the same, whole numbers only
`booleanCell()`	`true`/`false`, yes/no, ja/nein, `1`/`0`, `x` for a ticked box
`dateCell()`	ISO dates and timestamps, `31.12.2026`, `1.2.26`; `12/31` is ambiguous and refused
`choiceCell(['Active', 'Paused'])`	one of a set, case-insensitive, returned in the set’s spelling

A field’s schema may be any z.ZodType<JsonValue, IntakeCell>; refine these or write your own.

The proposal

planImport is pure over data. With a key, it looks each row’s key up in the existing map you pass (importKeys gives you the keys, once each):

no record → create in collection, ref: row-<n>, group new;
a record → proposeFields against its current values, one update per field that differs, each with a drift guard over the current value, group changed; a row that changes nothing counts as unchanged and proposes nothing;
a key an earlier row had → skipped with duplicate_key; the first row wins;
a rejected row → skipped with invalid_row and its issues.

Without a key, every valid row is a create. Every operation’s derivedFrom.citations points at the file: source is your sourceId, digest the bytes’ SHA-256, and locator the place in A1 notation — Kunden!12:12 for a new row, Kunden!C12 for an updated cell — which Excel’s name box jumps to in any language. The proposal’s workflowType is intake, so a host can give it a renderer of its own; summary has the counts for the card’s headline.

The words are the host’s to choose: rowLabel(row) names a new row for the reviewer (Row 12 by default — a product name reads better), and describeSkipped(skip) words a row left out in the reviewer’s language from structured data (invalid_row with its issues, each required or invalid; duplicate_key with the key and the first row that had it). describeSkip is the English default.

What it does not do

Write anything. The host applies the decision and owns every query.
Clear values. A blank cell never becomes null.
Delete what the file does not mention. A sync that removes records is a different operation with a different review.
Split a large file. Over a limit is a refusal; raise the limit or split the file deliberately.
Send a cell to a model. Only headers and field descriptions, through your function.

Stability

0.x: minors may break and say so; patches never do.