Docs / Specification

A-topology — System topology decisions

Status: accepted

cloudflare-d1 — Cloudflare execution metadata uses D1

Cited bytopology.coordinate.backendstopology.coordinate.backendsSingle-node self-hosting uses a local catalog file. Clusters use Postgres through pg-catalog. Cloudflare execution uses D1 through d1-catalog.run.journal.d1-profilerun.journal.d1-profileCONTEXTFUL_STORAGE_PROFILE=cloudflare-d1 selects D1 for catalog, journal, awakeables and export state through a private Worker outbound binding; R2 holds blob content and corpus objects.

The deciding criterion is one metadata backend across the deployment configuration, the execution binding and its conformance suite. Durable Objects still manage Container lifecycle; D1 replaces only the catalog role.

one-egress — One inference egress and one destination; no vendor SDK, no sink plugin

Cited bytopology.compose.inference-egresstopology.compose.inference-egressModel access leaves the process through one operator-configured endpoint speaking OpenAI-compatible HTTP. The one provider-shaped step serializes a tool's parameter schema into the provider's envelope.topology.compose.vendor-sdktopology.compose.vendor-sdkA model-vendor SDK or second outbound model path raises VendorSdkLinked in the dependency audit. The error names the crate and dependency.run.land.unknown-destinationrun.land.unknown-destinationThe local context store is the only destination, and synthesized artifacts write back through it; any other destination raises PipelineUnknownDestination at assembly, before any row moves.run.land.no-host-armrun.land.no-host-armThe destination world declares no host arm, so a guest supplies no destination.run.export-iceberg.egressrun.export-iceberg.egressEvery catalog and object-store request passes through the mediated client, whose allowlist holds the declared catalog and storage hosts; a table configuration naming another storage host raises ExportStorageUndeclared before any file is written.connector.infer.model-endpointconnector.infer.model-endpointEvery model call resolves to one operator-configured OpenAI-compatible HTTP endpoint declared as a capability. The host owns the credential, rate limit and span; the component owns the prompt template and response schema.

A provider swap is a URL edit, and the host owns credential, rate limit and span. Synthesized output falls under the same grants, masks and retention as landed data.

Revisit: a capability a deployment needs that no OpenAI-compatible endpoint exposes.

build-profiles — Profile capabilities are fixed at build; an absent one is a typed refusal

Cited bycorpus.targets.filecorpus.targets.fileEach deployment provider's target profile is spec/targets/<provider>.toml: named shapes, exactly one marked default, each with a kind, the profiles it hosts, an optional per-invocation wall-clock cap, its exclusions and the primitive filling each role.topology.package.profiletopology.package.profileThree profiles compile from the workspace, each a Cargo feature bundle selected at build time: contextful-edge, contextful-full, contextful-control. Each links the dependencies its role names and nothing else.topology.package.fixed-at-buildtopology.package.fixed-at-buildNo run-time detection widens a running binary into another profile's feature set. Build, release and automation tooling links into no profile.topology.package.profile-leaktopology.package.profile-leakA dependency outside a profile's role raises ProfileRoleLeak, naming the profile and dependency path. Disallowed dependencies include component hosts, execution or evaluation in edge/control; SQL in control; build tooling in any profile.topology.package.capability-absenttopology.package.capability-absentA subcommand requiring an unlinked capability raises ProfileCapabilityAbsent, naming the subcommand and profile. The command performs no action. A subcommand absent from every profile is a usage error.topology.package.component-hosttopology.package.component-hostBuilds that link a component host run component connectors: the full profile and container or worker deployments built from that profile.topology.package.host-missingtopology.package.host-missingDispatching a component connector without a component host raises ComponentHostMissing, naming the connector and profile. No similarly named native source substitutes.topology.package.crdt-leaktopology.package.crdt-leakThe CRDT library in the resolved dependency graph of the edge or full profile raises ProfileDependencyLeak, naming the profile and the path that pulled it. A daemon or replica reads materialized text.read.register.engineread.register.engineEvery profile serving reads links an embedded columnar SQL engine that reads Parquet with standard SQL. External processes read the same files without that embedded engine.read.rank.degradation-not-errorread.rank.degradation-not-errorWithout the lexical backend linked, ranking substitutes a token fallback scoring each query token over the scored-text, recency breaking ties. The result differs in order and still answers.read.rank.caller-embeddingread.rank.caller-embeddingA supplied query_embedding adds per-row cosine fused with the lexical leg; omitting one leaves every projected column as the lexical-only path. Without the vector backend it scores only rows the recency window recalled.read.embed.required-faceread.embed.required-faceBefore the first read, the server rejects a client's require: [...] with RequiredFaceAbsent for any unreported name. An engine reporting no set satisfies no requirement.read.embed.absent-read-backendread.embed.absent-read-backendWithout the embedded SQL engine linked, every read tool raises ReadBackendAbsent rather than answering from a narrower path.run.journal.unwired-capabilityrun.journal.unwired-capabilityRequesting a capability absent from the running profile raises CapabilityUnwired at the first request, before any partial work.

A capability the running profile does not wire is a typed refusal when reached, never a panic, no-op or look-alike fallback.

control-crdt — The control plane edits the configuration as one CRDT document and claims through the shared snapshot package

Cited bytopology.package.control-profiletopology.package.control-profilecontextful-control provides self-hosted team configuration and operator identity. Apply produces canonical TOML. This profile alone links the CRDT library.topology.package.control-team-statetopology.package.control-team-stateThe control profile stores ingest source, model and access-policy configuration in one loro document. Replicas merge concurrent edits. Apply uses this document as the editable configuration.topology.package.apply-hometopology.package.apply-homecontextful-snapshot is the one home of the snapshot directory local-claim claims into; contextful-engine and contextful-control both call it, and it links no run-path package.

Team state is the configuration itself, so replicas merge offline edits to it, and local and control-plane applies share one version claim, pointer and replay store.

public-surfaces — The engine and its consumers meet only at public surfaces

Cited bytopology.compose.three-crossingstopology.compose.three-crossingsExecution, storage and query code share exactly three contracts: the connector interface world, columnar-part-plus-manifest layout and capability-token format. No general internal API connects these components.topology.compose.undeclared-crossingtopology.compose.undeclared-crossingAn execution crate depends on storage or query crates only through crates implementing the three shared contracts. crate-graph checks each dependency.topology.bound-application.domain-neutraltopology.bound-application.domain-neutralThe engine processes declared data, schemas, policies, time bounds and capabilities. Consumer configuration, connectors, components or explicitly selected evaluation profiles supply domain meanings, business rules and sector-specific metrics.surface.package.store-selectorsurface.package.store-selectorThe child's working directory selects the store by walking up to the project manifest; finding none raises StoreSelectorAbsent and exits before writing any protocol framing.

Each shared contract has a version and conformance tests. A behavior becomes shared once a second application needs it; no license check guards a data-plane capability.

deploy-observation — A deploy verifies by observation

Cited bytopology.deploy.unsupported-shapetopology.deploy.unsupported-shapeA target profile claiming a shape its provider does not provide raises TargetShapeUnsupported, naming the shape and the missing primitive.topology.deploy.paritytopology.deploy.parityOne declaration produces byte-identical table parts on every control-plane target; contextful-ci deploy parity compares each target's object-store parts with the reference target's.topology.deploy.parity-divergencetopology.deploy.parity-divergenceA byte difference from the reference output raises ParityDivergence, naming the target, the table and the first differing part.

Each hostname descriptor names its worker, contract version and gate with an explicit acknowledgement, and the deploy judges an anonymous GET / against that gate.

posture-probe — The posture probe runs in build tooling; the gateway caches nothing

Cited bytopology.publish-hostname.posture-mismatchtopology.publish-hostname.posture-mismatchAn unreachable hostname or response outside its declared access policy raises HostnamePostureMismatch, naming the hostname, policy and observed response. contextful-ci deploy probe exits nonzero.topology.publish-hostname.probe-tabletopology.publish-hostname.probe-tableBefore probing hostnames, the deploy tool compares descriptor and probe-table entries. An entry missing from either set raises ProbeTableDrift, naming the hostname and missing side.

The gateway sits in front of a retrieval container that enforces grants per request, so a cached answer comes from inside the container, after enforcement. Profile footprint decided where the probe runs.

domain-crate — The domain crate holds types and ports with no I/O, and every edge points toward it

Cited bytopology.package.domain-cratetopology.package.domain-cratecontextful-core holds the pure domain types and the port traits every adapter implements, performs no I/O, and links into every profile.topology.package.domain-impuritytopology.package.domain-impurityAn async runtime, component host, HTTP client or columnar-format implementation in contextful-core raises DomainCrateImpurity. The error names the dependency and enabling feature.topology.package.dependency-directiontopology.package.dependency-directionA dependency edge from contextful-core toward an adapter crate raises TopologyDependencyInversion, naming both crates and the manifest line.

Revisit: two adapters convert the same shape differently; a port change deferred more than once.

enforcement-stack — Every access to data traverses the enforcement stack, and no configuration disables it

Cited bytopology.compose.mediationtopology.compose.mediationEach function returning or releasing stored rows requires an access-control admission value (row-token). A call omitting that value fails to compile. Access controls have no disable option.

No flag, environment variable or build feature disables access controls. A surface holding the only handle to data reaches it through an adapter over the stack.

Revisit: an adapter through the stack misses a stated throughput requirement.

export-committed — Export reads committed runs and is no second destination

Cited byrun.export.post-commit-readrun.export.post-commit-readExport reads committed runs only, through the read face under the admitted credential, so its grants, row policy and masks hold; no landing waits on an export.

_ingested_at is stamped per run, so a cursor over it skips a run that commits late under an earlier stamp. Export target secrets resolve and leave through the same egress as every other secret.

commit-lock — A commit holds the commit lock to reserve its value and to publish, not to write

Cited bystore.reserve.commit-seqstore.reserve.commit-seqThe engine injects non-null int64 _commit_seq above every value the table holds, so a run committing after a read carries a value above every returned row, whatever its _ingested_at.store.reserve.commit-orderstore.reserve.commit-orderA commit publishes its manifest once every lower _commit_seq of the table publishes or ends landing, so readable runs form a prefix of the commit sequence.store.reserve.commit-lock-spanstore.reserve.commit-lock-spanA commit holds its table's commit lock while assigning _commit_seq and writing its manifest. The commit writes parts outside the lock, except when the table keeps vintage history.store.reserve.commit-waitstore.reserve.commit-waitA commit waits at most 30 s for its table's commit lock or for a lower value's publication, then raises StoreCommitLockTimeout, a transient failure.store.reserve.commit-markerstore.reserve.commit-markerA commit rewrites data/.commit_seq as JSON with format_version and commit_seq. Builds reading a bare integer refuse the table before writing a part. A newer major refuses under format-newer.store.reserve.commit-seq-seedstore.reserve.commit-seq-seedA commit assigns one above the greatest commit_seq in the node's counter, run manifests and current snapshot, so a pulled or counterless store reuses no value.

A lock spanning the Parquet write serializes a table’s landings. A commit instead locks a ticket file naming its value while it writes parts; a dead holder’s ticket unlocks with its process, leaving a gap.

queue-ingress — A managed queue extends ingress behind a port; the writer stays in the engine

Cited bytopology.compose.queue-extensiontopology.compose.queue-extensionWith a queue role, the target carries receive batches, object-arrival keys and post-commit wake-ups through the Queue port. Without that role, the target uses the local spool. Only the engine writes the store.run.export.export-blockrun.export.export-blockAn OTLP [[export]] block takes Block with one landed table and signal = "logs"; contextful export run <name> delivers every row its cursor has not passed.run.export.commit-orderrun.export.commit-orderExport reads rows past its cursor ordered by _commit_seq, then _row_seq. Export does not order by _ingested_at; commit-seq keeps late-committing runs eligible.run.export.deliveryrun.export.deliveryThe exporter sends each batch as one OTLP/HTTP JSON POST through the mediated client. The allowlist contains only the endpoint host. Each header follows hydration-is-just-in-time.run.export.at-least-oncerun.export.at-least-onceA run ending between a target's acknowledgement and the cursor commit resends that batch on the next run; a target deduplicates on contextful.commit_seq and contextful.row_seq.

On a container with an ephemeral disk, the local spool loses a batch on eviction and a cold boot blocks every sender. Single-node and edge shapes keep no external queue.

catalog-read-add-on — A catalog query engine over Iceberg exports is an optional read face beside the Container

Cited byrun.export-iceberg.blockrun.export-iceberg.blockAn Iceberg Block declares format = "iceberg", one landed table, the admitted credential it reads as, a REST catalog URI, a storage endpoint, a warehouse, a namespace, a table_name, a format_version of 2 or 3, and a token reference.run.export-iceberg.file-sizerun.export-iceberg.file-sizeThe exporter writes data files of at most 128 MiB, opening a new file before the next rows carry the current one past that size; one row larger than that fills a file alone.run.export-iceberg.routerun.export-iceberg.routeWhere a deployment adds a catalog query engine, the gateway answers POST /stores/<id>/query from a reader whose audience key maps to a namespace through that engine, and sends every other caller to the Container.

A Container query face that sleeps pays a full hydration on its next read: 145 s for 1 GB in 8,157 objects, answering 503 until ready. Basin SQL over an Iceberg export of 10M rows answers a first read in 3.6 s and a warm one in 0.7 to 1.6 s at p50; a warm Container over 1.27M rows answers in 0.2 to 0.4 s. Read latency, idle cost and the frontier a reader sees decide.

At Workers Paid list prices, an always-on standard-1 Container costs about $27.5 a month whatever its query count. Basin SQL bills $2.50 per TB scanned with a 10 MB minimum per query, so a small query costs $25 per million, beside R2 writes and retained snapshot copies. They meet near 1.1 million queries a month. A 4,000-file table raises warm p50 1.2 to 2.1 times against 40 files and takes 24 s cold, so file size bounds latency and cost.

Revisit when a catalog engine’s warm p50 falls below 300 ms.

aws-fargate — AWS hosts every process on Fargate, and Lambda hosts none

Cited bytopology.package.edge-eligibilitytopology.package.edge-eligibilityThe edge profile is the one profile a function-class target hosts. Execution on such a deployment runs on a worker target.

Serving is long-lived by design: serve holds a hydrated snapshot set, refreshes it on an interval and answers concurrent requests from one process. A function instance freezes between requests, handles one request at a time and hydrates on each cold start, so one host lifecycle across every AWS process decides this.

Revisit: many small stores whose idle serving cost outweighs a second runtime’s upkeep, once a small task or a service scaled to zero no longer answers it.