contract run · 30-run.md · 31-pipeline.md · 32-derive.md · guide · card
Runs, pipelines and derive — card
Generated by contextful-spec state from the contract’s files and spec/terms/run.toml; not authored.
Operations
| Operation | Covers | Clauses | Proved |
|---|---|---|---|
journal | The journal records step values, returns those values on replay and deletes records after execution retirement. | 52 | 4 |
callback-guard | A D1 callback guard under d1-guard: the rows it protects, the effect identity it records, its release after the holder stops, and what an operator sees of it. | 16 | 0 |
items | The engine executes isolated input items with independent completion receipts, bounded retries and leased entity keys. | 13 | 0 |
advance | The engine commits source positions according to cursor kind and re-reads rows at each poll’s boundary. | 15 | 0 |
suspend | The engine suspends runs durably until an external callback resolves an awakeable or its deadline expires. | 13 | 0 |
retry | The runner classifies failures and schedules step retries through one retry layer. | 21 | 0 |
own | An execution owner binds pending work to a scope, connector build and plan identity. | 40 | 0 |
cancel | The engine stops work at run or pipeline scope through a shared cancellation token and records the remaining durable state. | 17 | 0 |
record | The store records run status, owner leases and execution history. History requests select a time window and row ceiling. | 28 | 0 |
project | The runner publishes a best-effort live view of run progress to subscribers. | 16 | 0 |
sweep | The scheduler runs one store-driven job per origin across a declared span. Each origin reads data available at that origin under one frozen transaction-time bound. | 10 | 0 |
orchestrate | Each deployment supplies a durable orchestrator implementing the execution interface. This section specifies provider steps and the engine behavior inside each step. | 8 | 0 |
declare | Pipeline declarations: fields, manifest loading, content hashes, table names and lifecycle commands. | 20 | 0 |
compile | The compiler converts a pipeline specification into a versioned plan with a shared schema. | 11 | 0 |
transform | The engine applies declared transforms in order to rewrite a batch in place. | 8 | 0 |
normalize | The engine normalizes nested data, assigns identity columns and converts nesting to relational tables where the sink requires it. | 10 | 0 |
guard-secrets | The secret guard masks credential-shaped text in pulled batches, dead letters and registered body results before storage. | 10 | 0 |
land | The engine processes batches from pull to commit, records ingest counts and isolates decoder failures. | 57 | 0 |
export | An export reads committed table rows and delivers them to an operator-declared target with at-least-once delivery. | 59 | 0 |
export-iceberg | An Iceberg export publishes one audience’s admitted rows of one landed table as an Apache Iceberg table in a REST catalog, which a query engine reads without reaching the store. | 30 | 0 |
backfill | The scheduler loads history through chunk plans with fenced leases and supports rewinding a declared range. | 11 | 0 |
seed | A seed source bulk-loads history below a declared ordering ceiling before live ingestion. | 13 | 0 |
model | The engine builds model tables from SQL over store tables, validates contracts and tests, and records freshness and holds. | 34 | 0 |
publish | Published tables: contract identity, snapshot publication, build history, freshness and collection holds. | 17 | 0 |
select | The derive source selects eligible parent rows each tick and limits each run’s work. | 30 | 0 |
bind | Engine bindings select registered tasks, configure engine interfaces and constrain media access and returned values. | 23 | 0 |
exec | The executor runs operator-declared argument arrays as local processes with pinned binaries, restricted environments and bounded runtime and output. | 19 | 0 |
fetch | The fetch driver follows third-party links with host and address checks, bounded redirects, document head scanning and image probes. | 11 | 0 |
emit | Derive tasks emit content rows and completion markers with status, attempt counts, citation keys and provenance. | 22 | 0 |
parse-cues | The caption parser reads SubRip and WebVTT through one grammar and groups cues into bounded passages. | 6 | 0 |
test-engine | The test-engine command builds one derive engine outside a pipeline and runs that engine against one file. | 1 | 0 |
Error cases
| Error | When | Clause |
|---|---|---|
BlobMissing | A journal row’s blob reference resolves to no stored blob. | run.journal.missing-blob |
CapabilityUnwired | A capability the running profile links no implementation for. | run.journal.unwired-capability |
JournalRedactionConflict | Write-path redaction declared over a source whose pulls are journaled. | run.journal.redacting-source |
StorageBindingUnavailable | A D1 binding request received no HTTP answer within its attempts. | run.journal.d1-transport |
StorageCapabilityMissing | A disposable execution profile lacks a required durable binding or adapter. | run.journal.storage-incomplete |
StorageCorpusUnavailable | A storage preflight’s corpus request received no answer or a server error. | run.journal.corpus-unavailable |
StorageProfileUnsupported | The storage profile or its encryption pairing is unsupported. | run.journal.storage-unsupported |
ExecutionSchemaIncompatible | The durable execution schema differs from the executable’s supported schema. | run.journal.execution-schema |
ExecutionRecoveryMismatch | A complete execution backup or its project recovery identity is missing. | run.journal.execution-recovery |
RunInputTruncated | A store-driven input statement the read face truncated at its row ceiling. | run.journal.input-truncated |
ExecutionGuardHeld | Reconciliation of a guard whose holder may still be running its callback. | run.callback-guard.live-holder |
ExecutionGuardEvidenceMismatch | Reconciliation evidence that carries no recorded key, or an idempotency key past its window. | run.callback-guard.evidence |
CursorFieldMismatch | A stored position’s field differs from the declared incremental field. | run.advance.field-rename |
CursorPositionUnorderable | A clock value with no ordering, or a position type changing mid-pass. | run.advance.unorderable-position |
AwakeableAlreadyResolved | A second resolution carrying a different payload. | run.suspend.conflicting-resolution |
AwakeableTimedOut | A resolution past the suspension’s deadline. | run.suspend.expired-token |
AwakeableUnknown | A resume token with no registry row. | run.suspend.unknown-token |
RunRetryTagUnretryable | A retry_on naming a tag outside the two retryable ones. | run.retry.unretryable-tag |
StepFailed | A step closed on a terminal failure, carrying its label and tag. | run.retry.step-failed |
ExecutionPinMismatch | A connector identity, component world, plan hash, plan reference or host identity moved under a pending owner. | run.own.pinned-plan-changed |
ExecutionRewindNotPending | A rewind naming an execution no pending owner holds. | run.own.rewind-not-pending |
ExecutionRewindPlanCurrent | An unforced rewind of an owner no attempt has asked another plan reference of. | run.own.rewind-plan-current |
ExecutionRewindUnsafe | A retirement of an owner a live attempt, a waiting attempt or a pending awakeable still holds. | run.own.rewind-unsafe |
RunStagedBytesExceeded | A run whose staged parts pass the per-run staged-bytes bound. | run.own.staged-bytes |
RunCheckpointIdTooLong | A checkpointing run whose next commit’s run id passes the store’s run-id bound. | run.own.checkpoint-id-length |
CancelTargetNotInFlight | A stop matching no pending, running or waiting run. | run.cancel.not-in-flight |
CancelUnauthorized | A credentialed stop whose grants cover no execute over the recorded pipeline. | run.cancel.stop-unauthorized |
SiteIdUnresolved | A site id with no declaration, two in one place, or an unset variable. | run.record.site-id-unresolved |
HistoryBoundSpelling | A history window bound outside the two accepted spellings. | run.record.bound-spelling |
RunMetadataTooLarge | A snapshot metadata write past its serialized bound. | run.project.metadata-too-large |
RunStreamUnauthorized | A run-stream request with no valid credential before the upgrade. | run.project.unauthenticated-upgrade |
RunSweepOriginAfterOpen | A sweep whose last origin falls after the transaction-time bound every origin reads at. | run.sweep.origin-after-open |
PipelineManifestMissing | A validation that read no manifest file. | run.declare.manifest-missing |
PipelineDuplicateId | One pipeline id declared twice. | run.declare.duplicate-id |
PipelineSpecInvalid | A manifest file the canonical type cannot deserialize. | run.declare.spec-invalid |
PipelineTableNameCollision | Two tables whose destination names fold to one spelling. | run.declare.table-name-collision |
PipelineUnboundTableName | A destination table named in a spelling the fold does not produce. | run.declare.unbound-table-name |
PipelineReplaceUnsupported | The replacing write mode beside a windowed or chunked load. | run.declare.replace-unsupported |
PipelineUnknownConfigKey | A source config key outside the set the source enumerates. | run.declare.config-key |
PipelineInlineStepBody | A plan step whose body is not a connector reference. | run.compile.inline-step-body |
PipelineUndeclaredControlFlow | A data-dependent conditional or loop in a workflow body. | run.compile.control-flow |
PipelineNodeIdCollision | One node id repeated in a plan. | run.compile.node-id-collision |
PipelineParallelUnjoined | A parallel node no join rejoins, or read around its join. | run.compile.unjoined-parallel |
PipelineTransformArity | A chain operation emitting more rows than it consumed. | run.transform.arity |
PipelineTransformColumnMissing | A chain operation naming a column the batch lacks. | run.transform.column-missing |
PipelineNormalizeModeUnknown | A normalize mode outside native and relational. | run.normalize.mode-unknown |
PipelineListIndexMissing | A relational child table without a list index. | run.normalize.list-index-missing |
PipelineUnknownDestination | A destination other than the local store. | run.land.unknown-destination |
PipelineTypeDeclaredLate | A pull declaring a type for a column an earlier staged batch of its run carried undeclared. | run.land.late-type |
ReceiveBatchTooLarge | A delivered body past its source’s max_batch_bytes. | run.land.receive-batch-too-large |
ReceiveBackpressure | A delivered batch arriving while every queue slot of its pipeline is held. | run.land.receive-backpressure |
ReceiveWriteNotGranted | A delivered batch under a credential lacking write over one of its pipeline’s tables. | run.land.receive-write-not-granted |
ReceiveBatchKeyExpired | A batch under a key whose first attempt is older than its source’s key_retention_secs. | run.land.receive-key-expired |
ReceiveBatchKeyMissing | A delivered batch naming no batch key. | run.land.receive-key-missing |
ReceivePipelineUnknown | A delivery to a pipeline id no declaration names as a receive source. | run.land.receive-pipeline-unknown |
PipelineUnreadableInput | Input a parser cannot read, named by path and position. | run.land.unreadable-input |
PipelinePartialParse | A read covering part of a multi-part input. | run.land.partial-parse |
PipelineParseCrashed | A non-zero exit or fatal signal from the decode process. | run.land.parse-crashed |
PipelineTableFailed | One table’s pull failing inside a fire. | run.land.table-failed |
ExportSignalUnknown | An export block naming a signal the built-in arm does not map. | run.export.signal-unknown |
ExportCommitSeqMissing | A table holding rows no cursor passes, their _commit_seq null. | run.export.commit-seq-missing |
ExecutionWriterFenced | An exporter no longer owns its durable writer lease. | run.export.writer-fenced |
ExportDeliveryRefused | An OTLP target answering other than 2xx, or unreachable. | run.export.delivery-refused |
ExportIdentityChanged | A pending typed export under another read identity, or its target redirected under the same name. | run.export.typed-identity-changed |
ExportEventTooLarge | One typed event exceeding the delivery request ceiling. | run.export.typed-event-too-large |
PublicationHeaderTooLarge | A coordinated group header leaving no room for one event under the typed request ceiling. | run.export.group-request-bound |
PublicationInputsBusy | A coordinated export whose publication inputs a running unit or another export still holds past the wait bound. | run.export.group-quiesce-wait |
PublicationIncompatible | A coordinated publication, predecessor, membership or checkpoint differs from its durable binding. | run.export.group-incompatible |
PublicationStorageUnsupported | A storage profile lacks the coordinated publication outbox capability. | run.export.group-storage |
PublicationDeliveryBounded | A bounded coordinated delivery stopped before its next batch with its group still pending. | run.export.group-within-exit |
ExportStorageUndeclared | An Iceberg table configuration naming a storage host its export block does not declare. | run.export-iceberg.egress |
ExportTypeUnsupported | An Iceberg export column holding nanosecond timestamps under format version 2. | run.export-iceberg.type-refused |
ExportCommitConflict | An Iceberg snapshot commit still conflicting after its last attempt. | run.export-iceberg.commit-retry |
ExportSchemaChanged | An Iceberg export source whose columns no longer match the table’s current schema. | run.export-iceberg.schema-changed |
ExportNamespaceShared | Two Iceberg exports sharing a namespace across credentials, a credential across namespaces, or one table. | run.export-iceberg.namespace-shared |
ExportQueryRefused | A Basin reader request the gateway answers from no engine: a named namespace, unknown template, bad table or shared-warehouse SQL. | run.export-iceberg.query-refused |
PipelineRewindWindowInvalid | An inverted, empty or scale-incomparable rewind window. | run.backfill.rewind-invalid |
PipelineSeedDeclarationMissing | A seeded table lacking a key or an event-time ordering. | run.seed.declaration-missing |
PipelineSeedCeilingBreached | A seeded row stamped at or past the ceiling. | run.seed.ceiling-breached |
PipelineSeedCeilingUnevaluable | A batch whose stamp cannot be compared with the ceiling. | run.seed.ceiling-unevaluable |
PipelineSeedSourceChanged | A seed source whose fingerprint differs from the committed one. | run.seed.source-changed |
PipelineSeedCompactionMissing | A seeded table no scheduled, enabled fold covers. | run.seed.compaction-cadence |
PipelineUnknownBlock | A manifest top-level key outside the enumerated blocks. | run.model.top-level-block |
ModelInputRestricted | A build reading a table that declares a disclosure key. | run.model.restricted-input |
ModelContractUndeclared | A published model declaring no contract. | run.model.contract-required |
ModelTestFailed | A model test returning a row. | run.model.test-failed |
ModelUndeclared | A build naming no declared model. | run.model.unknown-model |
ModelCycle | Model statements returning to a model through other models. | run.model.model-cycle |
ModelBuildUnknown | A hold naming a build no committed manifest records. | run.model.hold-unknown-build |
ModelBuildCollected | A hold naming a build whose parts a collection removed. | run.model.hold-collected |
PipelineContractMismatch | A materialization failing its declared contract. | run.publish.contract-mismatch |
DeriveConfigKeyMissing | A required derive config key absent or blank. | run.select.required-key |
DeriveNoStoreRoot | A derive source built without a store root or pipeline id. | run.select.no-store-root |
DeriveForeignOutputTable | Configuration overriding the build-owned anti-join output identity. | run.select.foreign-output-table |
DeriveUnitIncomplete | A parent row without a key or media value. | run.select.incomplete-unit |
DeriveJournaledPull | A derive pipeline configured to journal its pulls. | run.select.journaled-pull |
DeriveUnmeteredGrant | A shared-quota grant on a derive task sending through no metered client. | run.select.unmetered-grant |
DeriveMeteredClient | A derive task reaching the network outside the client the engine hands it. | run.select.metered-client |
DeriveParentsDiverge | A derive pipeline whose derive parents lie on no one dependent run. | run.select.parents-diverge |
DeriveCycle | Derive source-table or join dependencies returning to a pipeline. | run.select.derive-cycle |
DeriveAfterConflict | A derive child whose explicit predecessor differs from its derive parent. | run.select.derive-after-conflict |
DeriveEngineUnbound | A pipeline naming an engine the machine has not defined. | run.bind.unbound-engine |
DeriveCommandInManifest | An executable key on the request side of the binding split. | run.bind.command-in-manifest |
DeriveUnknownTask | A task naming neither a built-in nor a registered task. | run.bind.unknown-task |
DeriveTaskNameTaken | A host task registered under a name already taken. | run.bind.host-task |
DeriveJoinRepeated | A host task whose joins name one table twice. | run.bind.join-repeated |
DeriveDriverMismatch | A driver and task pairing the tier does not serve. | run.bind.driver-mismatch |
DeriveEndpointHostNotBare | An engine host carrying a path, query, port or scheme. | run.bind.endpoint-host-bare |
DeriveAttachUnbound | Credential headers declared on a binding with no exec vendor host. | run.bind.attach-unbound |
DeriveRemoteUrlUnsupported | A remote address handed to an engine that declines them. | run.bind.remote-url-unsupported |
DeriveMediaUnreadable | A media value neither an address nor a readable local file. | run.bind.media-unreadable |
DeriveMediaOutsideRoot | A local media path escaping the binding’s media root. | run.bind.media-root |
DeriveConfidenceOutOfRange | A confidence outside the normalized interval. | run.bind.confidence-range |
DeriveLinkEngineUnavailable | A link engine returning the engine-unavailable variant. | run.bind.engine-unavailable |
DeriveAdvisoryZone | A row zone taken from the operator’s advisory key. | run.bind.advisory-zone |
DeriveLocalityWider | An engine reaching wider than its source table’s zones. | run.bind.locality-wider |
DeriveShellCommand | A step command given as a string, not an argument array. | run.exec.shell-command |
DeriveEnvNameInvalid | An environment allowlist name outside the permitted characters. | run.exec.env-name |
DeriveEnvCredential | An exec step environment value embedding a credential reference. | run.exec.env-credential |
DeriveStepExit | A chain step ending non-zero. | run.exec.non-zero-exit |
DeriveStepTimeout | A chain outrunning its deadline. | run.exec.deadline-elapsed |
DeriveOutputCap | A step’s captured output crossing its bound. | run.exec.output-cap |
DeriveStepProducedNothing | A preprocess step exiting zero without its output file. | run.exec.silent-step |
DeriveBinaryMissing | A declared command that is not an executable file. | run.exec.missing-binary |
DeriveUnpinnedPath | A path-form command with no content digest. | run.exec.unpinned-path |
DeriveDigestMismatch | A pinned file whose bytes differ from its digest. | run.exec.digest-mismatch |
DeriveStepConditionUnknown | A preprocess condition naming no known condition. | run.exec.step-condition |
DeriveSchemeUnsupported | A followed address with a scheme other than http or https. | run.fetch.scheme |
DeriveAddressLiteral | A followed host written as an address literal. | run.fetch.address-literal |
DeriveFetchBindingKey | A process-spawning key on a fetch binding. | run.fetch.binding-key |
DeriveCharsetUnsupported | A document declaring a character set other than UTF-8. | run.fetch.charset |
DeriveBytesNotUtf8 | An undeclared document failing UTF-8 validation. | run.fetch.not-utf8 |
DeriveFailureOffTable | Per-unit failure state recorded outside the output table. | run.emit.failure-off-table |
DeriveUnitStatusUnknown | A unit status outside the four. | run.emit.unit-status |
DeriveReservedDiscriminator | A derive table declaring the reserved genre column. | run.emit.reserved-discriminator |
DeriveSettledUnitRevived | A settled unit re-entering the outstanding set. | run.emit.settled-revived |
DeriveUnredactedError | A last_error written without address redaction. | run.emit.unredacted-error |
DerivePrimaryKeyMissing | A derive output table without the unit-and-sequence key. | run.emit.primary-key |
DeriveOutputTablesMismatch | A host-task pipeline or unit reaching tables other than its task’s. | run.emit.output-tables |
DeriveCueOutOfOrder | A caption block starting before the one already accepted. | run.parse-cues.backward-cue |
DeriveTestEngineUnsupported | The single-file verb handed a fetch engine. | run.test-engine.unsupported-driver |
Bounds
| Bound | Value | Basis | Clause |
|---|---|---|---|
journal-inline-cutoff | 1 MiB | chosen | run.journal.inline-cutoff |
blob-sweep-grace | 1 h | chosen | run.journal.blob-sweep |
blob-sweep-interval | 24 h | chosen | run.journal.blob-sweep |
d1-request-timeout | 15 s | chosen | run.journal.d1-transport |
d1-transport-attempts | 3 attempts | chosen | run.journal.d1-transport |
allowed-lateness-default | 0 s | chosen | run.advance.allowed-lateness |
retry-base-delay | 100 ms | chosen | run.retry.default-policy |
retry-default-attempts | 5 attempts | chosen | run.retry.default-policy |
retry-jitter-ceiling | 250 ms | chosen | run.retry.default-policy |
retry-max-delay | 30 s | chosen | run.retry.default-policy |
retry-single-attempt | 1 attempts | chosen | run.retry.single-attempt |
retry-after-ceiling | 300 s | chosen | run.retry.retry-after |
staged-bytes-per-run | 1 GiB | chosen | run.own.staged-bytes |
checkpoint-bytes | 512 MiB | chosen | run.own.checkpoint-commit |
cancel-poll-interval | 500 ms | chosen | run.cancel.poll-interval |
owner-lease-renewal | 10 s | chosen | run.record.owner-lease |
owner-lease-ttl | 30 s | chosen | run.record.owner-lease |
site-id-max-length | 64 chars | chosen | run.record.site-id-length |
site-id-min-length | 1 chars | chosen | run.record.site-id-length |
describe-history-ceiling | 500 rows | chosen | run.record.describe-window |
describe-history-default | 5 rows | chosen | run.record.describe-window |
export-history-ceiling | 5000 rows | chosen | run.record.export-ceiling |
export-history-default | 500 rows | chosen | run.record.export-ceiling |
run-error-cap | 2 KiB | chosen | run.record.error-cap |
projection-coalesce-window | 100 ms | chosen | run.project.coalescing |
run-metadata-bytes | 16384 B | chosen | run.project.metadata-too-large |
broadcast-ring-entries | 256 entries | chosen | run.project.broadcast-ring |
channel-eviction-delay | 60 s | chosen | run.project.channel-eviction |
plan-version-prefix | 16 chars | chosen | run.compile.plan-version |
normalize-nesting-depth | 5 levels | chosen | run.normalize.nesting-depth |
llm-candidate-min-tail | 32 B | chosen | run.guard-secrets.llm-admission |
receive-group-bytes | 64 MiB | chosen | run.land.receive-group |
receive-retry-after | 1 s | chosen | run.land.receive-retry-after |
receive-push-wait | 30 s | chosen | run.land.receive-push-wait |
parse-child-deadline | 60 s | chosen | run.land.parse-deadline |
parse-child-memory | 512 MiB | chosen | run.land.parse-memory |
export-batch-rows | 500 rows | chosen | run.export.batch-rows |
typed-export-request-bytes | 65536 B | chosen | run.export.typed-byte-limit |
publication-completions | 1 completion | chosen | run.export.group-source-latest |
publication-quiesce-wait | 300 s | chosen | run.export.group-quiesce-wait |
d1-group-state-bytes | 8 MiB | chosen | run.export.group-storage |
d1-group-row-bytes | 64 KiB | chosen | run.export.group-d1-spill |
iceberg-data-file-bytes | 128 MiB | chosen | run.export-iceberg.file-size |
iceberg-part-bytes | 16 MiB | chosen | run.export-iceberg.multipart |
iceberg-row-group-bytes | 16 MiB | chosen | run.export-iceberg.memory |
iceberg-commit-attempts | 3 attempts | chosen | run.export-iceberg.commit-retry |
chunk-lease-renewal | 60 s | chosen | run.backfill.chunk-lease |
chunk-lease-ttl | 300 s | chosen | run.backfill.chunk-lease |
derive-rows-per-run | 25 rows | chosen | run.select.rows-per-run |
derive-attempts-per-unit | 3 attempts | chosen | run.select.attempts-per-unit |
derive-seconds-per-run | 300 s | chosen | run.select.seconds-per-run |
derive-memory-scan-rows | 1000000 rows | chosen | run.select.parent-scan |
derive-upstream-excerpt | 4 KiB | chosen | run.bind.upstream-excerpt |
derive-chain-deadline | 1800 s | chosen | run.exec.chain-deadline |
derive-captured-output | 8 MiB | chosen | run.exec.captured-output |
derive-audit-entries | 64 entries | chosen | run.exec.audit-entries |
derive-step-error-excerpt | 4 KiB | chosen | run.exec.step-error-excerpt |
derive-engine-id-prefix | 12 chars | chosen | run.exec.engine-id |
derive-redirect-chain | 5 hops | chosen | run.fetch.redirect-chain |
derive-document-prefix | 1 MiB | chosen | run.fetch.document-prefix |
derive-probe-prefix | 64 KiB | chosen | run.fetch.probe-prefix |
derive-hop-timeout | 20 s | chosen | run.fetch.hop-timeout |
derive-retry-after-default | 60 s | chosen | run.fetch.retry-after-default |
derive-empty-ceiling | 1 attempts | chosen | run.emit.attempts |
derive-passage-chars | 600 chars | chosen | run.parse-cues.passage-chars |
derive-passage-span | 60 s | chosen | run.parse-cues.passage-span |
derive-passage-bytes | 8 KiB | chosen | run.parse-cues.passage-bytes |
derive-passages-per-doc | 2000 rows | chosen | run.parse-cues.passages-per-document |
Types
Each struct or enum a {{type:<path>}} pointer names, with the keys serde reads and their doc comments.
contextful_cli::context::SessionBatch
The struct at crates/contextful-cli/src/context.rs, named by run.land.session-input.
| Key | Doc |
|---|---|
run_id | |
rows | |
now |
contextful_core::export::Block
The struct at crates/contextful-core/src/export.rs, named by run.export.export-block, run.export.typed-block, run.export-iceberg.block.
| Key | Doc |
|---|---|
name | |
table | |
endpoint | |
signal | |
format | |
key | |
schedule | |
headers | |
credential | |
catalog | |
storage | |
warehouse | |
namespace | |
table_name | |
format_version | |
token |
contextful_core::export::DeliveryProgress
The struct at crates/contextful-core/src/export.rs, named by run.export.status.
| Key | Doc |
|---|---|
pending | Whether a staged publication or group awaits acknowledgement. |
pending_events | Staged events no acknowledgement has passed. |
acknowledged_sequence | The sequence of the last acknowledged event, or null. |
version | Committed writes of the outbox: stages, acknowledgements and restores. |
last_ack_at | The instant of the last acknowledgement, or null. |
contextful_core::export::publication::PublicationDeclaration
The struct at crates/contextful-core/src/export/publication.rs, named by run.export.group-block.
| Key | Doc |
|---|---|
name | |
job | |
members |
contextful_core::pipeline::declare::PipelineSpec
The struct at crates/contextful-core/src/pipeline/declare.rs, named by run.declare.pipeline-spec.
| Key | Doc |
|---|---|
id | |
source | |
tables | |
destination | |
schedule | |
priority | The order among units due at one instant, highest first; absent is 0 (surface.dispatch.admission-order). |
after | The entry this one lands after: a landing step of that entry’s dependent run (surface.dispatch.not-a-head). |
incremental | The stream clock field (run.declare.incremental-field). |
transforms | |
redaction | |
journal | |
normalize | |
backfill | |
seed | |
queries | |
on_table_error |
contextful_core::pipeline::declare::SourceBlock
The struct at crates/contextful-core/src/pipeline/declare.rs, named by run.declare.pipeline-spec.
| Key | Doc |
|---|---|
name | |
config |
contextful_core::pipeline::model::BuildEntry
The struct at crates/contextful-core/src/pipeline/model.rs, named by run.publish.build-entry.
| Key | Doc |
|---|---|
build_id | |
started_at | |
completed_at | |
status | |
contract_version | |
schema_fingerprint | |
partitions_unfilled | |
disclosure_digest |
contextful_core::pipeline::model::ContractColumn
The struct at crates/contextful-core/src/pipeline/model.rs, named by run.model.contract-block.
| Key | Doc |
|---|---|
name | |
type | |
nullable |
contextful_core::pipeline::model::HoldRecord
The struct at crates/contextful-core/src/pipeline/model.rs, named by run.publish.hold.
| Key | Doc |
|---|---|
build_id | |
principal | |
placed_at | |
expires_at |
contextful_core::pipeline::model::ModelSpec
The struct at crates/contextful-core/src/pipeline/model.rs, named by run.model.model-block.
| Key | Doc |
|---|---|
id | |
sql | |
sql_file | |
disclosure_opt_out | |
disclosure | |
materialized | |
unique_key | |
publish | |
contract | |
freshness | |
test |
contextful_core::pipeline::model::PublishSection
The struct at crates/contextful-core/src/pipeline/model.rs, named by run.publish.manifest-section.
| Key | Doc |
|---|---|
contract_version | |
schema_fingerprint | |
build_id | |
build_started_at | |
last_built_at | |
watermark | |
max_lag | |
last_build_status | |
withheld_cells | |
disclosure_digest | |
partitions_failed | |
semantics_version | |
fingerprint_recipe |
contextful_core::pipeline::transform::TransformOp
The enum at crates/contextful-core/src/pipeline/transform.rs, named by run.transform.chain.
| Key | Doc |
|---|---|
select | Fix the outgoing column set by name. |
rename | Map an incoming name onto an outgoing one. |
cast | Rewrite one column’s type: string, int64, float64 or boolean, or a binary or vector type (run.transform.typed-cast). |
filter | Keep the rows whose single column equals a value, or by whether a column holds a value at all (run.transform.presence-filter). A filter takes exactly one form. |
extract | Copy the value at an RFC 6901 pointer into a named column (run.transform.extract). |
contextful_core::run::advance::CursorKind
The enum at crates/contextful-core/src/run/advance.rs, named by run.advance.cursor-kind, run.advance.concurrency-by-kind, run.backfill.chunk-plan.
| Key | Doc |
|---|---|
monotonic | A timestamp, autoincrement or watermark. |
opaque-token | A vendor continuation token. |
snapshot-id | A log sequence number or version id. |
contextful_core::run::backfill::BackfillBlock
The struct at crates/contextful-core/src/run/backfill.rs, named by run.backfill.block.
| Key | Doc |
|---|---|
max_chunks | The bound on a plan’s chunk count. |
chunk_size | One clock-ranged chunk’s width in cursor units. |
start_from | The floor of the planned range. |
contextful_core::run::backfill::ChunkRow
The struct at crates/contextful-core/src/run/backfill.rs, named by run.backfill.chunk.
| Key | Doc |
|---|---|
pipeline_id | |
table | |
chunk | |
ordinal | The chunk’s place in its plan; a plan resumes at the lowest ordinal not done. |
predicate | The half-open range of cursor positions the chunk pulls. |
status | |
attempts | How many times a worker claimed the chunk. |
started_at | |
completed_at | |
cursor_committed | Whether the chunk’s commit cached its position. |
sequential | Whether the chunk waits on every earlier chunk of its plan (run.backfill.chunk-plan). |
lease | The last lease a worker took on the chunk; a done row keeps the committing one. |
contextful_core::run::backfill::PipelinePhase
The enum at crates/contextful-core/src/run/backfill.rs, named by run.backfill.phase.
| Key | Doc |
|---|---|
planning | |
seeding | |
backfilling | |
streaming |
contextful_core::run::derive::config::Binding
The struct at crates/contextful-core/src/run/derive/config.rs, named by run.bind.command-in-manifest, run.fetch.binding-key.
| Key | Doc |
|---|---|
driver | |
timeout_secs | |
max_output_bytes | |
zone | Advisory: the adapter, not this key, writes a row’s zone. |
media_root | The directory local media resolves under, relative to the working directory. |
preprocess | |
engine | |
env | Variable name to value template, the only environment a step receives. |
allow_hosts | |
allow_image_hosts | |
endpoint_host | |
attach | Header name to value template the exec proxy attaches to each request toward endpoint_host (connector.attach.exec-proxy-attach). |
request_timeout_secs | |
max_document_bytes | |
max_probe_bytes |
contextful_core::run::drive::ItemTerminal
The enum at crates/contextful-core/src/run/drive.rs, named by run.items.terminal-outcome.
| Key | Doc |
|---|---|
succeeded | |
failed | A failure no retry changes. |
exhausted | A retryable failure on the attempt that spent the item budget. |
contextful_core::run::failure::FailureTag
The enum at crates/contextful-core/src/run/failure.rs, named by run.retry.failure-taxonomy.
| Key | Doc |
|---|---|
Transient | |
Permanent | |
SchemaIncompatible | |
AuthExpired | |
RateLimited | |
Config | |
Storage | |
UnknownConnector | |
SecretNotFound | |
Canceled |
contextful_core::run::nodes::NodeKind
The enum at crates/contextful-core/src/run/nodes.rs, named by run.compile.plan-node.
| Key | Doc |
|---|---|
step | One connector pull, recorded once, under an optional retry schedule. |
sleep | A durable wait, spelled as a span such as 30s or 5m. |
awaitEvent | A suspension on an awakeable, with an optional time-to-live span. |
branch | An arm chosen by a predecessor’s recorded output: the label-to-node-id map. |
parallel | Fan-out bodies, each a node id run as one branch. |
join | The explicit join node rejoining one parallel node’s bodies. |
contextful_core::run::ports::DeadLetter
The struct at crates/contextful-core/src/run/ports.rs, named by run.guard-secrets.dead-letter-reason, run.land.dead-letter, run.land.dead-letter-occupied.
| Key | Doc |
|---|---|
key | |
reason | |
sha256 |
contextful_core::run::project::Snapshot
The struct at crates/contextful-core/src/run/project.rs, named by run.project.wire-snapshot.
| Key | Doc |
|---|---|
run_id | |
workflow_id | |
status | |
steps | |
metadata | |
started_at | |
ended_at | |
error | |
version |
contextful_core::run::project::StepStatus
The enum at crates/contextful-core/src/run/project.rs, named by run.project.step-view.
| Key | Doc |
|---|---|
pending | |
running | |
completed | |
failed | |
retrying |
contextful_core::run::project::StepView
The struct at crates/contextful-core/src/run/project.rs, named by run.project.step-view.
| Key | Doc |
|---|---|
label | |
status | |
attempts | |
started_at | |
ended_at | |
output | |
failure |
contextful_core::run::record::InputBounds
The struct at crates/contextful-core/src/run/record.rs, named by run.record.sweep-bounds.
| Key | Doc |
|---|---|
as_of | |
snapshots | |
rows | |
available_as_of | |
valid_as_of | |
sweep_id | |
origin |
contextful_core::run::record::RunRow
The struct at crates/contextful-core/src/run/record.rs, named by run.record.columns, run.land.ingest-tally.
| Key | Doc |
|---|---|
run_id | |
pipeline_id | |
table | |
site_id | |
status | |
owner | |
started_at | |
ended_at | |
rows | Rows and bytes measured at the destination (run.record.skipped-count). |
bytes | |
batches | Batches the run landed. |
skipped | Inputs the run’s pulls declared skipped, summed (run.record.skipped-count). |
fetched | Rows the run’s pulls carried into the transform chain, summed (run.land.ingest-tally). |
kept | Rows the transform chain kept for staging, summed; fetched - kept is the rows its filter operations dropped (run.land.ingest-tally). |
declined | What the run’s directory walks declined, tallied by extension and summed over its pulls (connector.source.declined-tally). |
audit | Captured output the run’s pulls recorded, in pull order (run.exec.audit-entries). |
error_kind | |
error_message | |
connector_id | |
connector_version | |
connector_hash | |
trace_id | |
phase | |
pipeline_phase | The pipeline phase the run ran under; a seed chunk’s run records seeding, its only attribution beside the load id (run.seed.one-land-path). Absent for every other run. |
execution_id | The execution whose journal this attempt resolves against (run.own.execution-id-keys-the-journal). |
plan_ref | Immutable plan identity admitted at this attempt’s open, retained after retirement. |
stop | |
host_scope | The host scope an execution opened under; None for a table or chunk run, whose pipeline and table name its scope (run.own.host-scope). |
input | A store-driven run’s input (run.record.sweep-bounds); None for every other run. |
failed_branches | Each fan-out branch that failed into a join declaring allow_partial, with its label and failure tag (run.journal.fan-out-join). |
credential_providers | Each logical name the run hydrated, mapped to the adapter that answered it, names only; a row recorded without the field decodes as empty (connector.record.provider-attribution). |
bound_hits | Every declared resource bound a connector struck, in the order struck, a hit a later attempt recovered from included; a row recorded without the list decodes as empty (run.retry.bound-hit-recorded). |
next_attempt_at | A waiting attempt’s next-attempt instant (run.retry.waiting-state); absent on every other row. |
unpushed | A run committed on a disposable disk whose run manifest no committed push has carried (store.drain.mark); absent on every other row. |
contextful_core::run::record::RunStatus
The enum at crates/contextful-core/src/run/record.rs, named by run.cancel.not-in-flight, run.record.status-set.
| Key | Doc |
|---|---|
pending | |
running | |
waiting | |
success | |
partial_failure | |
failed | |
canceled |
contextful_core::run::sweep::Sweep
The struct at crates/contextful-core/src/run/sweep.rs, named by run.sweep.origins.
| Key | Doc |
|---|---|
start | |
end | |
step | A <n><unit> interval in the schedule grammar’s units. |
state |
contextful_core::store::declare::TableDecl
The struct at crates/contextful-core/src/store/declare.rs, named by store.declare.table-block, store.declare.erasure-metadata, run.model.restricted-input, run.publish.disclosure-digest.
| Key | Doc |
|---|---|
name | |
primary_key | |
order_by | |
write_mode | |
replicate | |
subject_id | |
erasure_key | |
referenced_by | |
on_erase | |
class | |
policy | |
redaction | |
visibility | |
valid_time | |
available_time | The availability instant an available_as_of read compares (store.available-time.declaration). |
snapshot | A snapshot corpus’s capture columns (store.snapshot.capture). |
cluster_by | |
partition_by | |
bloom_filter | Columns whose Parquet parts carry a bloom filter (store.index.kinds). |
retain_runs | |
fold_max_unfolded_runs | The unfolded run count that fires the table’s pass in place of 50 (store.fold.declared-trigger); zero refuses the block. |
fold_max_unfolded_bytes | The part bytes, summed over the unfolded runs’ manifests, that fire the table’s pass (store.fold.declared-bytes); zero refuses the block. |
allowed_lateness | The window a monotonic poll of this table re-reads behind its stored position, spelled as a span (run.advance.allowed-lateness). |
retain_rows | |
retain_versions | A derive output table keeping each task version’s rows current under that version (run.emit.version-retained). |
columns | Column types every landing reads (store.declare.table-block), spelled as [ColumnType::parse] reads them; an unreadable spelling refuses the block. |
indexes | Sidecar declarations (store.index.declaration). |
agent_description | |
agent_hint | |
column_hints | Hints for named columns in a table description (read.register.describe-payload). |
example_queries | |
content_hash_column | The column carrying each row’s content hash, the row key a ranked read keeps one row per (read.retrieve.row-key-dedup). |
result_cache | The time to live of a cached read result touching the table, which opts the table into the result cache (read.cache.cache-is-opt-in), spelled as retain_runs. |
private | A table whose rows never enter the result cache (read.cache.cache-is-opt-in). |
retrieve | What a ranked read scores the table’s rows on (read.retrieve.scored-text). |
contextful_core::surface::delegate::Answer
The enum at crates/contextful-core/src/surface/delegate.rs, named by run.items.delegated-item, surface.delegate.outcome.
| Key | Doc |
|---|---|
ok | Rows per declared output table; they land as the item’s rows. |
defer | Park the item until retry_at, spending no attempt (surface.delegate.defer-parks). |
transient | Spend one failed attempt. |
permanent | Write a terminal receipt. |