Docs / Specification

contract store · 10-store.md · 11-snapshot.md · 12-time.md · 13-replica.md · 14-drain.md · 15-endpoint.md · 16-collect.md · 17-merge.md · 18-pull.md · guide · card

Unpushed runs

Replacing a disposable disk loses committed runs that have not reached the bucket. The execution database marks each run from opening until a push carries it, and commands, pipeline serve shutdown and release checks read the mark.

drain

Tracking unpushed runs, retrying pushes before reads, pushing during shutdown and reporting status to release checks.

• marktesttestUnder postgres or cloudflare-d1, opening a run marks its row unpushed with the node before landing commits; closing clears the mark when no table the push includes holds a manifest of the run.
  Why

  A-store#unpushed-run-marked
• record-unmarkedtestThe store’s run record keeps each run row without unpushed; only the execution database holds the mark.
  Why

  because a rebuilt execution database restores rows from the run record
• cleartesttestA committed push clears each mark whose run manifest its included tables held at push start or its committed bucket manifest lists. It sets lost on other marks with no local manifest and no live owner, naming them on stderr.
  Why

  because no retry restores a commit the store has lost
• before-readtestUnder these profiles, before-runstore.pull.before-runWith [sync] pull_before_run = true, run start, pipeline run, mcp and serve pull the tables table-scope selects before their first read, a failed pull stopping the command. pushes after pulling for unlost marks on its node with a local manifest or expired owner lease. A failed push names those runs and stops the command before its first read.
  Why

  A-store#unpushed-run-marked
• deadlinetestOn a stop signal, pipeline serve with push_after_run ends its children, then pushes for its node’s marks meeting before-readstore.drain.before-readUnder these profiles, before-run pushes after pulling for unlost marks on its node with a local manifest or expired owner lease. A failed push names those runs and stops the command before its first read., for at most 15 s.
  Why

  because a platform kills a stopping process after its grace period
• unfinishedtestA shutdown push failing or exceeding deadlinestore.drain.deadlineOn a stop signal, pipeline serve with push_after_run ends its children, then pushes for its node's marks meeting before-read, for at most 15 s. keeps the run marks and names the runs on stderr, and pipeline serve exits non-zero.
  Why

  because a release step reading a clean exit as a carried store discards the missed runs
• statustesttestcontextful sync status prints JSON error, the counts open and lost of marked runs, and runs, each with pipeline, table, node, end instant and lost; any run not lost reports SyncRunsUnpushed, exiting 3.
  Why

  because a release step replacing a container discards each unpushed run
• status-summarytestWithout --all, runs lists at most 20 runs of each kind, lost and not, in run id order; the counts still number every marked run.
  Why

  because a caller keeping a bounded tail of stdout parses the whole document, and the counts carry what the list leaves out
• status-alltestsync status --all lists every marked run in runs.
  Why

  because an operator auditing every mark reads the full list on request, apart from the release gate’s bounded read
• status-unreadtestA sync status failing to read the execution database prints runs as null and error as SyncStatusUnread with its cause, exiting 1.
  Why

  because a release gate tells a store holding unpushed runs from a check that read nothing
• prunetestA committed push clears each lost mark, any node’s, more than 7 d after clearstore.drain.clearA committed push clears each mark whose run manifest its included tables held at push start or its committed bucket manifest lists. It sets lost on other marks with no local manifest and no live owner, naming them on stderr. set it, keeping the run row; a mark lost without its instant takes the push’s.
  Why

  because only a push sets lost, so pruning there bounds the marks to one window of losses while status stays a read

Scenarios

• store.drain.markstore.drain.markUnder postgres or cloudflare-d1, opening a run marks its row unpushed with the node before landing commits; closing clears the mark when no table the push includes holds a manifest of the run.: WHEN a process terminates after a run’s local commit and before its push, THEN the run row stays marked and the next push uploads the run.
• store.drain.before-readstore.drain.before-readUnder these profiles, before-run pushes after pulling for unlost marks on its node with a local manifest or expired owner lease. A failed push names those runs and stops the command before its first read.: WHEN the bucket refuses a run’s push and then accepts a push from run start, THEN that start pushes the marked run before opening its own run.
• store.drain.clearstore.drain.clearA committed push clears each mark whose run manifest its included tables held at push start or its committed bucket manifest lists. It sets lost on other marks with no local manifest and no live owner, naming them on stderr.: WHEN a replacement disk holding none of a marked run’s files pushes, THEN the run turns lost and sync status lists it without refusing.
• store.drain.clearstore.drain.clearA committed push clears each mark whose run manifest its included tables held at push start or its committed bucket manifest lists. It sets lost on other marks with no local manifest and no live owner, naming them on stderr.: WHEN a disk whose pull_tables excludes a marked run’s table pushes while the bucket lists that run’s manifest, THEN the mark clears and the run never turns lost.
• store.drain.status-summarystore.drain.status-summaryWithout --all, runs lists at most 20 runs of each kind, lost and not, in run id order; the counts still number every marked run.: WHEN the execution database holds 300 lost marks and one open mark, THEN sync status lists the open run and 20 lost runs, counts 1 open and 300 lost, and exits 3.
• store.drain.prunestore.drain.pruneA committed push clears each lost mark, any node's, more than 7 d after clear set it, keeping the run row; a mark lost without its instant takes the push's.: WHEN a push settles while one mark turned lost 8 d before and another 6 d before, THEN the push clears the first mark and the second stays listed.
• store.drain.unfinishedstore.drain.unfinishedA shutdown push failing or exceeding deadline keeps the run marks and names the runs on stderr, and pipeline serve exits non-zero.: WHEN a stopping serve’s push waits on a push guard past 15 s, THEN serve exits non-zero and the run stays marked.