Docs / Specification
Milestones
The order in which the corpus is built. A milestone names the operations it closes;
contextful-spec state expands each to its clause set and reports the reach in
status.md
Each milestone’s Reach: line is its acceptance criterion, and its Acceptance: line
names the test in crates/acceptance/ that drives that reach through a built binary.
The acceptance test lands before the first clause of its milestone is pinned, ignored
while the milestone is open; the milestone closes when the ignore comes off, the
test passes, and every operation it names holds a performed clause.
A Depth: operation line specifies a milestone at operation level: its operations carry
refusal and limit clauses, unsettled lines and diagrams, and no behavior clause. Removing
the line when the milestone opens admits behavior clauses again.
Flowchart diagram
Flowchart, top to bottom. 17 nodes, 17 edges.
0 Test-first gate → 1 Authority core [unlocks]
1 Authority core → 2 Store [unlocks]; → 5 Read face [unlocks]; → 14 Assurance [unlocks]
2 Store → 3 Run path [unlocks]; → 5 Read face [unlocks]; → 11 Derive [unlocks]
3 Run path → 4 Ingest [unlocks]
4 Ingest → 10 Cadence [unlocks]; → 15 Point-in-time evaluation [unlocks]
5 Read face → 6 Sync [unlocks]; → 7 Memory [unlocks]; → 8 Accountability [unlocks]; → 12 Operator console [unlocks]
7 Memory → 16 Memory graph walk [unlocks]
8 Accountability → 9 Visibility [unlocks]
9 Visibility → 13 Disclosure [unlocks]
unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks unlocks 0 Test-first gate 1 Authority core 2 Store 3 Run path 4 Ingest 5 Read face 6 Sync 7 Memory 8 Accountability 9 Visibility 10 Cadence 11 Derive 12 Operator console 13 Disclosure 14 Assurance 15 Point-in-time evaluation 16 Memory graph walk
0 — The test-first gate
Reach: A change altering source with no test failing against its base reds the gate before merge.
Acceptance: contextful_acceptance::m00::m00_test_first_gate
1 — The authority core
Operations Intent authority.identifyauthority.identify The subject tuple, its per-member attestation, hygiene at the mint, and the identity links connecting a subject to a source's principals.authority.profileauthority.profile The versioned delegation profile over the attenuable-credential library: admitted elements, reserved facts, the bounded evaluator, the scoped session.authority.grantauthority.grant What a credential says a holder does: actions, table patterns, tenant scope, the template allowlist, ceilings, and resources answering by name.authority.attenuateauthority.attenuate Deriving a narrower child offline, and the narrowing legality of each grant dimension.authority.issueauthority.issue Minting: the persisted lifetime ceiling, the principal a row-landing grant needs, signing custody, and the authoring posture.authority.verifyauthority.verify Admission at a checkpoint: signatures, audience, timestamps, possession proof, key sets, and the admitted-authority value.authority.revokeauthority.revoke Ending a credential's usefulness ahead of expiry: short lifetimes, the denylist, the scoped epoch, key rotation, format withdrawal.authority.exchangeauthority.exchange Exchanging a verified external assertion for a scoped credential under injected verifying material and a declared policy.The subject, delegation, admission, custody and revocation, with the proof package and differential harness beside them. assurance.modelassurance.model The Lean policy package and the Lean protocol state machine, their pins, and the definitions theorems and checks range over.assurance.proveassurance.prove The theorem inventory: layer composition, placement, the evidence floor, the proof targets over the authority mapping, and the mediation relation over statement execution.assurance.audit-assumptionsassurance.audit-assumptions The assumption allowlist, the transitive footprint audit over the elaborated environment, the inventory and the report.assurance.recheckassurance.recheck Rebuilding and re-auditing from pinned source in a credential-free environment.assurance.scope-claimassurance.scope-claim The sentence the assurance claim is allowed to be: named decisions, named specifications and trusted dependencies.assurance.differential-testassurance.differential-test The executable reference models, the case generators, the minimized counterexample corpus and the command running them.—
Reach: An authority is admitted once, travels as a value, and is re-read at the effect about to act.
Acceptance: contextful_acceptance::m01::m01_authority_core
2 — The store
Operations Intent store.lay-outstore.lay-out Store directories, run and snapshot manifests, table pointers, local catalogs and node identifiers.store.initstore.init Project initialization, and project discovery when a command omits --project.store.declarestore.declare Table declarations, primary keys, row ordering, write modes and replacement behavior.store.reservestore.reserve Reserved column and table names, injected columns, _commit_seq assignment and request ledgers.store.reconcilestore.reconcile Schema reconciliation across table files, type promotion, added columns and scan behavior.store.foldstore.fold Compaction passes, their triggers and retention, and snapshot publication through a pointer update.store.indexstore.index Sidecar indexes, bloom filters, row clustering and partitions.store.bound-timestore.bound-time Transaction-time and valid-time columns, their read parameters and the snapshots or rows each parameter selects.store.encryptstore.encrypt At-rest encryption of Parquet, sidecars, ledgers, metadata, machine catalogs and node run-state files.Layout, declaration, reserved columns, reconciliation, commit and compaction, indexes, the two clocks, at-rest encryption.
Reach: A table lands, a scan resolves its file list, and a reader opens the Parquet without the engine.
Acceptance: contextful_acceptance::m02::m02_store
3 — The run path
Operations Intent run.journalrun.journal The journal records step values, returns those values on replay and deletes records after execution retirement.run.itemsrun.items The engine executes isolated input items with independent completion receipts, bounded retries and leased entity keys.run.advancerun.advance The engine commits source positions according to cursor kind and re-reads rows at each poll's boundary.run.suspendrun.suspend The engine suspends runs durably until an external callback resolves an awakeable or its deadline expires.run.retryrun.retry The runner classifies failures and schedules step retries through one retry layer.run.ownrun.own An execution owner binds pending work to a scope, connector build and plan identity.run.cancelrun.cancel The engine stops work at run or pipeline scope through a shared cancellation token and records the remaining durable state.run.recordrun.record The store records run status, owner leases and execution history. History requests select a time window and row ceiling.run.projectrun.project The runner publishes a best-effort live view of run progress to subscribers.Journal, isolated items, cursor, suspension, retry, cancellation and the run record; shared contracts, packaging and coordination. topology.composetopology.compose Execution, storage, queries, access controls and the contracts these components share.topology.packagetopology.package Rust package dependencies and the capabilities each compiled binary includes.topology.coordinatetopology.coordinate Conditional metadata updates, writer leases and the databases that store coordination state.—
Reach: A run crashes mid-step and resumes from its journal without re-issuing a recorded effect.
Acceptance: contextful_acceptance::m03::m03_run_path
4 — Ingest
Operations Intent run.declarerun.declare Pipeline declarations: fields, manifest loading, content hashes, table names and lifecycle commands.run.compilerun.compile The compiler converts a pipeline specification into a versioned plan with a shared schema.run.transformrun.transform The engine applies declared transforms in order to rewrite a batch in place.run.normalizerun.normalize The engine normalizes nested data, assigns identity columns and converts nesting to relational tables where the sink requires it.run.guard-secretsrun.guard-secrets The secret guard masks credential-shaped text in pulled batches, dead letters and registered body results before storage.run.landrun.land The engine processes batches from pull to commit, records ingest counts and isolates decoder failures.run.exportrun.export An export reads committed table rows and delivers them to an operator-declared target with at-least-once delivery.run.backfillrun.backfill The scheduler loads history through chunk plans with fenced leases and supports rewinding a declared range.run.seedrun.seed A seed source bulk-loads history below a declared ordering ceiling before live ingestion.run.modelrun.model The engine builds model tables from SQL over store tables, validates contracts and tests, and records freshness and holds.run.publishrun.publish Published tables: contract identity, snapshot publication, build history, freshness and collection holds.The pipeline declaration and landing sequence, the connector world, and outbound credentials. connector.*—
Reach: A declared pipeline pulls from a real source, lands rows under a cursor and never holds a credential in plaintext, and a model over those rows publishes a build a hold keeps.
Acceptance: contextful_acceptance::m04::m04_ingest
5 — The read face under enforcement
Operations Intent read.registerread.register Registered relations, query tools, templates, the embedded SQL engine and HTTP transport.read.guardread.guard SQL validation, allowed relations, operator statements and template arguments.read.respondread.respond Response fields, cell encoding, row and byte limits, truncation and diagnostic metadata.read.queryread.query The operator's raw statement verb on the command line: its arguments, the relations it registers, its row bound and its output.read.retrieveread.retrieve Candidate rows for ranked retrieval: query tokens, relevance thresholds, per-table queries, deduplication and snippets.read.rankread.rank Candidate ordering by vector and lexical scores, timeframe and reported confidence.read.cacheread.cache Locality of the bytes a read touches, reuse of a resolved session and its connections, and reuse of a result across requests.read.resolve-pinread.resolve-pin Resolution of a named published-model build at read time, and the state each response echoes.read.embedread.embed Embedding implementations, stored model identifiers, query-model checks, build identity and missing-backend errors.Registration, the statement guard, retrieval and ranking, the tool surface; the enforcement layers as one reference monitor. authority.redactauthority.redact Removal and transformation inside the writer, ahead of columnar bytes and the run path's durable record.authority.composeauthority.compose The three layers as one reference monitor, and the order a registered relation applies.authority.filter-rowsauthority.filter-rows The typed row-predicate grammar, subject binding, the tenant equality and the project default.authority.maskauthority.mask Column classes, strategies and combines, guardrails over an exhaustible value space, the pepper, and masked behavior under caller SQL.authority.refuseauthority.refuse The two shapes an unauthorized read takes, the scope guard, and what a refusal discloses.authority.bound-redistributionauthority.bound-redistribution The per-table licence bound applied at push, the root-object allowlist, and the per-credential edge filter.authority.placeauthority.place Inference zones: grammar, composition across grains, floors, incognito, and the serve-time outcome.authority.resistauthority.resist The paths that stay closed to an injected instruction.—
Reach: An agent asks over MCP and receives ranked rows the caller’s authority admits.
Acceptance: contextful_acceptance::m05::m05_read_face
6 — Sync and replicas
Operations Intent store.endpointstore.endpoint Bucket endpoint schemes, transport, request addressing and signing, credentials and backend response handling.store.emitstore.emit Push planning from local files without reading or writing the bucket.store.pushstore.push Uploading local files, constraining bucket prefixes and committing the bucket manifest.store.pullstore.pull Downloading bucket files, verifying digests, advancing table pointers and restoring named generations.store.probestore.probe Testing conditional writes with a sentinel object and selecting the backend's coordination mode.store.mergestore.merge Merging bucket manifests across writers, verifying tombstones and retrying conditional manifest commits.store.leasestore.lease Exclusive pipeline and compaction writer leases, their commit fences, and cursor commits in the commit log.store.replicatestore.replicate Replica refreshes, required snapshot files, local snapshot records and refused reads or writes.store.drainstore.drain Tracking unpushed runs, retrying pushes before reads, pushing during shutdown and reporting status to release checks.store.collectstore.collect Unlisting and deleting the bucket objects no reader needs, and the generation manifests past the retained count.Bucket endpoints and credentials, push planning, the wire format, push and pull, the conditional-write probe, merge, fenced leases, the replica, unpushed runs and bucket collection.
Reach: Two nodes share one bucket and converge without a coordinator.
Acceptance: contextful_acceptance::m06::m06_sync
7 — Memory
Operations Intent read.declareread.declare Memory table schemas, required columns, trust tiers, scopes and relation types.read.synthesizeread.synthesize Claim extraction, entity resolution, consolidation, evidence, deduplication, audit records and coverage gaps.read.reviseread.revise Claim replacement within a validity interval, confidence decay, direct writes, timestamps, expiry, retention and promotion.read.recallread.recall Ranked and keyed claim retrieval, read timestamps, tier ordering, scope and tombstone filters, and evidence checks.read.resolve-entityread.resolve-entity Entity matching, aliases, identity cards, location relationships, edges and ownership queries.read.settleread.settle Prediction registration, observations, required verdict citations and outcome label views.Memory tables, synthesis, revision on one valid-time line, recall, entity resolution and settled outcomes.
Reach: A synthesized belief supersedes its predecessor on new evidence, and a reader sees which grant produced it.
Acceptance: contextful_acceptance::m07::m07_memory
8 — Accountability
Operations Intent disclosure.recorddisclosure.record What a read leaves behind: the span, the hash-linked audit entry, where telemetry lands, and the key custody a log opens under.disclosure.explaindisclosure.explain The access decision and its path, replay over a window with its coverage, and the audience report.disclosure.attestdisclosure.attest Chain verification, signed segment roots, lineage attestations, and the reach of each guarantee.disclosure.erasedisclosure.erase The one erasure verb: subject tombstones, the bounded provenance cascade, and the tenant rewrite.disclosure.receiptdisclosure.receipt The signed artifact a tenant purge returns: its claim, its coverage block, its pseudonym and its idempotence.The audit chain, explanation, attestation, erasure and the receipt.
Reach: An operator answers what a named person could have seen over a past window, from the store, in SQL.
Acceptance: contextful_acceptance::m08::m08_accountability
9 — Visibility
Operations Intent disclosure.mirrordisclosure.mirror The three-way conjunction, the access tables as store data, and the binding that compiles a table's semi-join.disclosure.sweepdisclosure.sweep Observation of one source's permission state: sweep kinds, the observation clock, the coverage watermark, tombstones and revocation.disclosure.reachdisclosure.reach Per-request resolution of a subject to the resources it reaches, the bounded group closure, and the reachable-set cache.disclosure.bound-stalenessdisclosure.bound-staleness The declared age budget on mirrored authorization, the refusal past it, and the one narrowing posture.disclosure.declare-fidelitydisclosure.declare-fidelity The fidelity level a table claims, the source family bounding the claim, and live federation under a reader's delegated credential.disclosure.packdisclosure.pack The reviewed unit landing one source: a manifest fragment plus a connector pin, its access mapping, defaults and container allowlist.Source permissions mirrored as data, the sweep, reachability, the staleness budget and fidelity.
Reach: A revoked grant at the source stops answering within a declared bound.
Acceptance: contextful_acceptance::m09::m09_visibility
10 — Cadence and the operator plane
Operations Intent surface.armsurface.arm The schedule grammar and cron dialect, the trigger adapter and its durability, the wake route, and how an entry computes its next fire.surface.reconcilesurface.reconcile The control source, snapshot pointer, signed pulled-head validation against local declarations, schedule diff, and what a failed poll leaves running.surface.firesurface.fire Job declaration, the closed kind union, same-tick order, the fire watermark, and one-shot evaluation.surface.dispatchsurface.dispatch The bounded fire pool and its exclusion keys, the reconciler's hold on the cadence lease, step results, and the worker adapter with its attempt-fenced callback.surface.editsurface.edit The configuration document, the records it presents read-only, the structured schedule specification, and connector configuration.surface.applysurface.apply Validation, the immutable version claim, the pointer advance, the owner's storage, and the authority an apply carries.surface.residesurface.reside Where the data plane runs, the region allow-set gating placement, and operation with no external reach.surface.register-storesurface.register-store Which stores a deployment serves, how each store's credential and binding names derive, and how a request reaches a store's origin.Schedules, reconciliation, bounded dispatch, configuration apply, deploy targets and published hostnames. topology.deploytopology.deploy Deployment configurations, provider services and comparison against the reference deployment.topology.publish-hostnametopology.publish-hostname Hostname access policies, deployment checks and requests through the routing service to the query container.topology.bound-applicationtopology.bound-application Responsibilities of the engine and applications that use the engine.—
Reach: Due work dispatches into a bounded pool, and a published hostname is probed for the posture it declares.
Acceptance: contextful_acceptance::m10::m10_cadence
11 — The derive tier
Operations Intent run.selectrun.select The derive source selects eligible parent rows each tick and limits each run's work.run.bindrun.bind Engine bindings select registered tasks, configure engine interfaces and constrain media access and returned values.run.execrun.exec The executor runs operator-declared argument arrays as local processes with pinned binaries, restricted environments and bounded runtime and output.run.fetchrun.fetch The fetch driver follows third-party links with host and address checks, bounded redirects, document head scanning and image probes.run.emitrun.emit Derive tasks emit content rows and completion markers with status, attempt counts, citation keys and provenance.run.parse-cuesrun.parse-cues The caption parser reads SubRip and WebVTT through one grammar and groups cues into bounded passages.run.test-enginerun.test-engine The test-engine command builds one derive engine outside a pipeline and runs that engine against one file.Deferred per-row work over landed rows.
Reach: A pipeline reads the words inside a landed document and fills them into the parent row.
Acceptance: contextful_acceptance::m11::m11_derive
12 — The operator console
Operations Intent surface.open-consolesurface.open-console The hosted console's two pages, their API routes, identity gates, page grants and engine capabilities.surface.visualizesurface.visualize The Admin page's operational views: the workflow canvas, run history and data lineage.surface.packagesurface.package The client library, its four deployment shapes and transports, and the credential each shape carries.surface.speaksurface.speak The language every Query-visible string carries, the audience contract, and the deterministic backstops beneath it.surface.groundsurface.ground Where an answer's material comes from: the closed tool set, the two trust layers, the per-reader credential, the web supplement and the sources block.surface.plan-turnsurface.plan-turn The shape of one turn: planner scaffolding, the replanning round, the answerability test, the code path, and the store's prompt overlay.surface.set-vantagesurface.set-vantage A session's time basis: its vantage, the bound each leg carries, the snapshot timeline and the arrivals strip.surface.browsesurface.browse What a store advertises before a question: discovered chips, humanized labels, the insights panel and the file gallery.surface.learnsurface.learn The reading loop's memory: recall ahead of planning, the per-turn distillation and the labels its conclusions inherit.surface.rendersurface.render The three channels, the component union, deterministic component choice, view hints and the sanitizer walk.surface.briefsurface.brief The proactive greeting card: its derivation, payload, matching tiers, client-side clock and the absence it earns.surface.publish-answersurface.publish-answer The moment an answer computed for one operator reaches a place other people read.Independently gated Query and Admin pages, governed answers, and visualized operator workflows.
Reach: An operator asks on Query and receives a sourced answer; Admin visualizes the store’s workflows and admits only operators with its page grant.
Acceptance: contextful_acceptance::m12::m12_console
13 — Disclosure
Operations Intent disclosure.set-modedisclosure.set-mode The two deployment settings a derived result is computed in, the offline diagnostic, and the cross-owner hashed join.disclosure.releasedisclosure.release The ordered statistics release, the write-time disclosure policy, and the per-unit budget reservation.disclosure.suppressdisclosure.suppress The group-size floor, the contributor-share ceiling, and the single signal a withheld group leaves.disclosure.templatedisclosure.template Query templates a releasing principal calls by name: the single-statement rule, the checks they pass, and the truncation flag a capped read carries.disclosure.bound-cohortdisclosure.bound-cohort The line between a per-person table and a cohort table, and the floor a cohort read carries.Aggregate release under a disclosure budget.
Reach: Two parties compare against a benchmark neither can invert.
Acceptance: contextful_acceptance::m13::m13_disclosure
14 — Assurance
Operations Intent assurance.structure-treeassurance.structure-tree One home per capability, the one enforcement decision module, adapters, mirror annotations and derived-artifact checks.assurance.buildassurance.build Target directories, the engine-linked invocation, linked query functions, build targets, release artifacts and the dependency allowlist.assurance.gateassurance.gate Stage order, secrets of record, the crate-graph, row-token, egress and dependency rules, the formal stage, build jobs, free disk, footprint budgets and surface checks.assurance.evaluateassurance.evaluate The read-path quality harness: case format, deterministic metrics and floors, judged dimensions, the reader and the judge.assurance.baselineassurance.baseline Holding a run against committed baselines and absolute floors, intervals, golden custody and case generation.assurance.measureassurance.measure The target ledger: each tracked target, the clause it serves, how it is measured and its record per commit.assurance.releaseassurance.release The release tag's version, the refusals guarding it, the signed annotated tag contextful-ci tag creates, and the local packaging the operator runs from it.The build gate, the build targets, the retrieval-quality harness, the target ledger and release tags.
Reach: The gate holds its resource budget, and the evaluation floors and every gate-tier target are measured on every change.
Acceptance: contextful_acceptance::m14::m14_assurance
15 — Point-in-time evaluation
Operations Intent store.available-timestore.available-time The source availability timestamp, its value at landing and row selection under available_as_of.The declared availability instant, its resolution at landing, vintages, and the available_as_of bound. read.as-of-joinread.as-of-join Point-in-time ASOF JOIN statements and templates that match rows by their availability timestamps.The point-in-time join matching each driving row to the right row available at its instant. run.sweeprun.sweep The scheduler runs one store-driven job per origin across a declared span. Each origin reads data available at that origin under one frozen transaction-time bound.Walk-forward origins under one frozen as_of, carried or fresh state, and writes stamped available at their origin. read.horizonread.horizon Reads under horizon: default time bounds, rejected arguments, hidden metadata, network restrictions and selection of indexes, embedding models and builds.The credential horizon every read inherits, the arguments it refuses, the stamps and statistics it withholds, and the builds and indexes it resolves. store.snapshotstore.snapshot Document capture records, snapshot identifiers for evaluation cases and access to documents captured after the reading horizon.The captured document corpus a case pins, and the standing of a capture after the horizon. assurance.backtestassurance.backtest Historical evaluations, called backtests: bounded cases, model-knowledge controls, label intervals and splits, execution metadata, planted leaks and replayable trajectories. Financial return-series and execution profiles are consumer-selected applications of this flow (domain-neutral).Bounded cases routed to dated checkpoints, leakage stated against a reference, purged and embargoed splits, a trial ledger, planted leaks and a replayable trajectory. assurance.forecastassurance.forecast Probability forecasts over consumer-declared outcomes: freeze instant, settlement, lead time, imputation, coverage, reference baselines and separate scoring tracks. Reference baselines belong to an explicitly selected profile (domain-neutral).Forecast cases frozen before resolution, settled through the label join, scored on tracks that never pool.
Reach: A source observation landed with its valid instant and later publication instant answers no available_as_of read between the two, and answers one after publication.
Acceptance: contextful_acceptance::m15::m15_point_in_time
16 — Memory graph walk
Reach: An agent walks admitted edges from an entity and sees only what its credential can read.
Acceptance: contextful_acceptance::m16::m16_traverse
17 — Delegated bodies
Reach: A store-driven item runs its paid call on a loopback delegate, its answer lands as the item’s rows, and a resume after a restart sends no second request.
Acceptance: contextful_acceptance::m17::m17_delegated
18 — Provider orchestrators
Reach: A backfill chunk’s provider step crashed after half its pages resumes under an in-tree test filler and lands parts byte-identical to the engine runner’s.
Acceptance: contextful_acceptance::m18::m18_orchestrate
19 — Drive connections
Reach: A Picker-selected file creates one tenant-bound Drive pipeline, lands scoped rows, pauses without another fire, and deletes with its credential revoked.
Acceptance: contextful_acceptance::m19::m19_drive_connections
20 — Drive permission custody
Reach: A Drive permission withdrawal stops an enterprise reader within the declared visibility bound, while two connection refreshes resolve one versioned credential.
Acceptance: contextful_acceptance::m20::m20_drive_permissions
21 — Sheet rows
Reach: A selected workbook lands rows from two sheets with distinct keys, and removing a row lands its tombstone.
Acceptance: contextful_acceptance::m21::m21_sheet_rows
22 — Drive change cursor
Reach: A change during the initial walk lands once through the change cursor, and an invalid cursor triggers a complete selected reconciliation.
Acceptance: contextful_acceptance::m22::m22_drive_changes
23 — Workspace delegation
Reach: A delegated Workspace connection reads only as its admitted domain user and refuses a subject outside the approved domain before token minting.
Acceptance: contextful_acceptance::m23::m23_workspace_delegation
24 — Iceberg export
Reach: Two exports of one table commit an append and then an overwrite snapshot to a local Iceberg REST catalog, and a reader of the namespace sees the second export’s rows alone.
Acceptance: contextful_acceptance::m24::m24_iceberg_export
25 — Scoped callback guards
Reach: A guard on one item blocks its owner, ancestors and entity lease while another job’s admission proceeds, and a dead holder’s replay-safe effect releases with its original key while mismatched evidence leaves the guard.
Acceptance: contextful_acceptance::m25::m25_callback_guards