Docs / Specification

contract store · 10-store.md · 11-snapshot.md · 12-time.md · 13-replica.md · 14-drain.md · 15-endpoint.md · 16-collect.md · 17-merge.md · 18-pull.md · guide · card

Time bounds

The engine stamps transaction time at commit; a table declaration names its valid-time and availability-time columns. Each bound resolves before access controls select rows.

bound-time

Transaction-time and valid-time columns, their read parameters and the snapshots or rows each parameter selects.

• two-clockstestTransaction time is _ingested_at; valid time is a declared pair of table columns, which the engine never infers, and it stamps no second transaction timestamp.
• valid-time-declarationtest[pipeline.tables.valid_time] declares from and an optional to; a table declaring from alone treats each row as valid from that instant onward.
• valid-time-typetestA declared valid-time column whose type is not a timestamp raises StoreValidTimeNotTimestamp at declaration.
  Why

  P1
• instant-comparisontestEvery bound compares instants as timestamps, not strings; a date-only literal resolves to the start of the next day, exclusive.
• as-oftestInside each table’s FROM-source, as_of selects the newest reachable snapshot created at or before the bound plus the committed runs at or before it that the snapshot omits.
  Why

  because filtering current files by ingest stamp returns different rows before and after a fold
• row-age-cutofftestA read of a table with retain-rowsstore.declare.retain-rowsretain_rows = { column = "<name>", age = "<n>d" } takes _ingested_at or a declared Timestamp column; another column or a null value raises StoreRetentionColumnInvalid before landing, and a malformed age refuses the declaration. excludes rows older than its wall-clock cutoff, as_of reads of retained snapshots included, and the result cache reuses no such statement.
  Why

  because a historical snapshot does not restore a row after its declared age passes
• as-of-unretainedtesttesttestAn as_of before the oldest snapshot retention kept, or selecting a snapshot collected-snapshotstore.fold.collected-snapshotA snapshot counts as collected while the bucket's _collected.json stands beside it or, on a store reading through no part source, a part it names is absent; no as_of read, pin or hold resolves to it. counts or whose successors fold a run absent here, raises StoreAsOfUnretained, naming the oldest answerable instant.
  Why

  P4
• valid-as-oftestvalid_as_of filters the same inner source with from <= valid_as_of AND (to IS NULL OR to > valid_as_of) over the declared pair.
• valid-time-undeclaredtestA valid_as_of read against a table declaring no pair raises StoreValidTimeUndeclared, naming the table.
  Why

  P1
• pin-boundtestA read carrying as_of and a build pin resolves each table named by both by earlier-bound-winsread.resolve-pin.earlier-bound-winsA pin and the store's transaction-time bound are upper bounds on one clock; a table named by both resolves to the earlier..
• unbounded-latesttestA read carrying neither bound resolves each table to its current snapshot plus the committed runs it omits.
• beneath-enforcementtestBoth bounds apply before the row predicate, column mask and tenant filter, so they only narrow the rows access controls admit.
  Why

  P5
• covering-versionstestFor an unkeyed table, a valid-time bound returns every version whose interval contains the requested instant.
• interval-rowstestA keyed table declaring valid_time stores one row per key and valid-time interval, deduplicating on its primary key and from, so a fold keeps every interval.
  Why

  A-store#validity-interval-row
• echotestA bounded read returns contextful.bounds as {as_of?, valid_as_of?, inclusive}, each instant RFC 3339 UTC with nine fractional digits and a Z; an unbounded read omits it.

unsettled: Does an as_of read selecting a held snapshot older than a collected one read that snapshot, rather than refuse under as-of-unretainedstore.bound-time.as-of-unretainedAn as_of before the oldest snapshot retention kept, or selecting a snapshot collected-snapshot counts or whose successors fold a run absent here, raises StoreAsOfUnretained, naming the oldest answerable instant.? owner: store affects: store.bound-time

available-time

The source availability timestamp, its value at landing and row selection under available_as_of.

• declarationtesttest[pipeline.tables.available_time] declares AvailableTime, omitted from canonical serialization when unset. column holds the source availability instant as timestamptz, which landing keeps and availability bounds compare alone; zone defaults to UTC.
  Why

  A-store#availability-declared-column
• lag-filltesttestAt landing, available_time.lag fills a null availability column with valid-time to, else from, plus the duration lag.classes maps the row’s lag.class_column value to, and injects _available_imputed true; a class without an entry leaves the instant null.
  Why

  because a source can publish different record classes after different declared delays
• live-sincetesttestA date-typed availability value resolves at landing to the start of the next day in the declared zone; rows available before available_time.live_since, when the source began publishing live, get injected _available_reconstructed true.
  Why

  because a date gives no publication time within its day
• available-as-oftesttestavailable_as_of filters the inner source with <column> <= available_as_of under beneath-enforcementstore.bound-time.beneath-enforcementBoth bounds apply before the row predicate, column mask and tenant filter, so they only narrow the rows access controls admit., combining with as-ofstore.bound-time.as-ofInside each table's FROM-source, as_of selects the newest reachable snapshot created at or before the bound plus the committed runs at or before it that the snapshot omits. and valid-as-ofstore.bound-time.valid-as-ofvalid_as_of filters the same inner source with from <= valid_as_of AND (to IS NULL OR to > valid_as_of) over the declared pair. when the read supplies them.
  Why

  A-store#availability-declared-column, P5
• bound-echotestA read carrying available_as_of adds it, in the same instant form, to the object echostore.bound-time.echoA bounded read returns contextful.bounds as {as_of?, valid_as_of?, inclusive}, each instant RFC 3339 UTC with nine fractional digits and a Z; an unbounded read omits it. returns.
• undeclaredtesttestAn available_as_of read of a table declaring no available_time raises StoreAvailableTimeUndeclared, naming the table; a landing row resolving to a null instant raises StoreAvailableTimeNull, naming the table and run.
  Why

  because a missing availability instant either hides a backfill or leaks it to every bound
• vintage-rowstesttesttestA keyed table declaring available_time is a vintage table, keeping every vintage per primary key, valid-time from and availability instant; a read returns the latest at or before any available_as_of, or under vintage = "first" the earliest.
  Why

  A-store#restatement-vintage-row, A-store#adjustment-factors-apply, A-read#join-matches-availability
• vintage-hashtestA landing row whose content_hash_column value equals that of its key’s latest vintage lands no new vintage.
  Why

  because a re-pulled unchanged window otherwise lands a false restatement
• vintage-validationtesttestA vintage table declaring no content_hash_column raises StoreVintageHashUndeclared, and one declaring retain-rowsstore.declare.retain-rowsretain_rows = { column = "<name>", age = "<n>d" } takes _ingested_at or a declared Timestamp column; another column or a null value raises StoreRetentionColumnInvalid before landing, and a malformed age refuses the declaration. raises StoreVintageRetainRows, at validation, naming the table.
  Why

  because a re-pull without a hash reads as a restatement, and a row-age cutoff shrinks a past answer
• vintage-dedup-keytesttesttesttestA vintage table reads through dedup-viewstore.declare.dedup-viewA keyed table reads through ROW_NUMBER() OVER (PARTITION BY <pk> ORDER BY <order_by> DESC, _ingested_at DESC, _run_id DESC, _row_seq DESC) = 1 over the current snapshot, if any, unioned with the committed runs it omits., or interval-rowsstore.bound-time.interval-rowsA keyed table declaring valid_time stores one row per key and valid-time interval, deduplicating on its primary key and from, so a fold keeps every interval. under valid_time, and folds under dedup-viewstore.declare.dedup-viewA keyed table reads through ROW_NUMBER() OVER (PARTITION BY <pk> ORDER BY <order_by> DESC, _ingested_at DESC, _run_id DESC, _row_seq DESC) = 1 over the current snapshot, if any, unioned with the committed runs it omits. or valid-time-linestore.fold.valid-time-lineA keyed table declaring valid_time partitions on the key and valid-time line, keeping one row per line., each adding the availability column to the key and keeping every vintage; an unclustered valid_time fold sorts by from, then primary key.
  Why

  A-store#restatement-vintage-row, A-store#validity-interval-row, A-store#adjustment-factors-apply
• vintage-scopetestA vintage landing compares only its own keys’ stored vintages, skipping row groups that text or signed-integer key bloom filters, or footer bounds on text, integer or temporal keys or temporal valid time, exclude.
  Why

  A-store#vintage-landing-reads
• adjustment-factorstesttestavailable_time.adjustment keeps stored values raw; an available_as_of read multiplies each adjust_columns column by the product of factor_column vintages available by the bound, over factor rows sharing its factor-table primary_key values with a later valid-time from.
  Why

  because applying a correction published later exposes information unavailable at an earlier read bound

Scenarios

• store.available-time.available-as-ofstore.available-time.available-as-ofavailable_as_of filters the inner source with <column> <= available_as_of under beneath-enforcement, combining with as-of and valid-as-of when the read supplies them.: WHEN a 2026 backfill lands a first-quarter measurement with availability date 2014-04-30, THEN available_as_of 2014-05-01 returns the row and available_as_of 2014-04-29 returns none.
• store.available-time.vintage-rowsstore.available-time.vintage-rowsA keyed table declaring available_time is a vintage table, keeping every vintage per primary key, valid-time from and availability instant; a read returns the latest at or before any available_as_of, or under vintage = "first" the earliest.: WHEN a first-quarter measurement lands at 17149.6 available 2014-04-30, 17101.3 available 2014-05-29 and 17016.0 available 2014-06-25, THEN available_as_of 2014-06-01 returns 17101.3, an unbounded read returns 17016.0 before and after a fold, and vintage = "first" returns 17149.6.
• store.available-time.vintage-hashstore.available-time.vintage-hashA landing row whose content_hash_column value equals that of its key's latest vintage lands no new vintage.: WHEN a re-pull lands the measurement at 17016.0 with an unchanged content hash, THEN the table still holds three measurement vintages.
• store.available-time.vintage-scopestore.available-time.vintage-scopeA vintage landing compares only its own keys' stored vintages, skipping row groups that text or signed-integer key bloom filters, or footer bounds on text, integer or temporal keys or temporal valid time, exclude.: WHEN day 101 of 100 entities lands over 100 stored daily runs, THEN its comparison decodes no stored row; WHEN a revision of day 50 lands, THEN the comparison decodes only day 50’s 100 rows.

unsettled: Does vintage = "first" count a same-day correction as a revision of the initial release? owner: store affects: store.available-time