Docs / Specification

contract disclosure · 60-visibility.md · 61-disclosure.md · 62-accountability.md · guide · card

Source-faithful visibility

A workspace serving a whole organization answers each reader inside the audience the originating source drew. This file holds the mirrored permission state, its observation clock, the resolution of a subject to the resources it reaches, the age budget on that state, the fidelity a table claims, and the pack that lands a source.

Mirrored permission state, from the source’s permission endpoint to the reader’s view:

mirror

The three-way conjunction, the access tables as store data, and the binding that compiles a table’s semi-join.

• unbound-tabletestOn an organization-wide face, a table with no visibility block raises VisibilityUnboundTable from diagnose and from the serving guardrail at start, naming the table.
  Why

  P5
• face-scopetest[project] face_scope = "organization" makes every face the project serves organization-wide; absent or "personal", its faces are personal, and another value does not load.
  Why

  because a local store serves its owner first, so the organization-wide guardrails bind where a project declares an organization audience
• incomplete-bindingtestA binding at a servable level lacking a declared sweep for its source or a max_acl_staleness raises VisibilityBindingIncomplete.
  Why

  P5
• binding-keystestA table’s visibility block declares RawBinding; on_stale takes refuse, the default, or public_only.
• memory-bindingtestA memory [[table]] takes a visibility block under the keys of binding-keysdisclosure.mirror.binding-keysA table's visibility block declares RawBinding; on_stale takes refuse, the default, or public_only., parsed as a pipeline table’s block, so an organization-wide face serves it bound.
• grants-tabletestA sweep lands a source’s access data as ordinary rows of _visibility/<source>/resources, grants, principals, group_members, identity_links, tombstones and freshness, through the land path and its write lock.
  Why

  A-disclosure#grants-rows-under
• unmirrorable-openingtestA resource whose access list the sweep could not mirror opens only through a grant row the store owner lands carrying expires_at; the row confers nothing past that instant, and no manifest key opens a resource.
  Why

  A-disclosure#grants-rows-under

sweep

Observation of one source’s permission state: sweep kinds, the observation clock, the coverage watermark, tombstones and revocation.

• coverage-watermarktestA sweep run that failed or skipped any governed resource of its source and advances watermark_at raises VisibilityWatermarkUncovered, and the watermark stays where it stood.
  Why

  A-disclosure#mirrored-authorization-carries
• ungapped-streamtestAdvancing watermark_at from an event stream the mapping has not declared gap-detectable raises VisibilityUngappedStream.
  Why

  A-disclosure#mirrored-authorization-carries
• orphan-granttestA grant landing with no resource row, or on a resource of the unknown class, raises VisibilityOrphanGrant at commit and joins no reachable set.
  Why

  P5
• sweep-blocktestAn [[acl_sweep]] block declares one source’s sweep as RawBlock: enumerate names the table whose rows are its governed resources, and each cadence is { every = <budget> }.
• cadence-grammartestA sweep cadence’s every takes the grammar of budget-grammardisclosure.bound-staleness.budget-grammarmax_acl_staleness is a positive integer with one suffix from s, m, h, d. A compound form, a bare number or another suffix raises VisibilityBudgetMalformed, naming the table and the text read..
• sweep-verbtestvisibility sweep --source <s> --kind full|incremental|webhook --grants <file> [--directory <file>] runs one sweep of a declared source from its files and commits as one run.
• grants-filetestA --grants file is the run’s read of every resource enumerate names: one JSON object per line, a GrantLine or resource_id with "unread": true; an unmarked resource with no grant line holds none.
• bare-resourcetestA --grants line carrying resource_id alone names a resource read holding no grant, so a webhook run naming it withdraws every grant that resource held.
• directory-filetestA --directory file is the identity provider’s directory read, one JSON object per line: a LinkLine or a Membership; a run without one keeps the current rows.
• full-runtestA full run replaces the source’s resource and grant rows with those it read, writes a tombstone per grant it no longer observes, and sets last_full_sweep_at.
• incremental-runtestAn incremental run replaces the grants of each resource it read, tombstoning those withdrawn, leaves every other resource’s grants standing, and sets last_incremental_at.
• webhook-runtestA webhook run applies one delivery of the source’s events stream: it replaces the grants of each resource its file names, leaves every other resource standing, and moves watermark_at to the instant it began reading.
• empty-deliverytestA webhook run whose file names no resource lands nothing, and watermark_at stays where it stood.
  Why

  because an empty delivery reads no resource, so it carries no evidence the source’s grants stand
• superseded-runtestBefore it lands, a run collects every run of its source’s resources, grants, principals, group_members and identity_links but the current one, so each of those tables holds at most two runs.
• watermark-advancetestA run reading every governed resource sets the source’s watermark_at to the instant it began reading; lag is now minus watermark_at.
• sweep-epochtestA committed run advances its source’s epoch by one.
  Why

  A-disclosure#mirrored-authorization-carries

unsettled: Does visibility sweep given no --grants file read the source’s permission endpoint under the block’s credential? owner: disclosure affects: disclosure.sweep

reach

Per-request resolution of a subject to the resources it reaches, the bounded group closure, and the reachable-set cache.

• subject-principalstestA subject user://<p> reaches the user principal spelled <p> and each principal an identity_links row maps to it, then the groups those principals belong to.
• closure-walktestThe closure walks the group graph with a visited set, ending any cycle, to a depth of max_group_depth, default 8 hops, and a breadth of 10000 nodes.
• closure-overflowtestA closure reaching its depth or node bound raises VisibilityClosureDepth with the non-retryable HTTP status 422 and returns no set.
  Why

  A-disclosure#grants-rows-under
• cache-capacitytestThe reachable-set cache holds at most 100000 entries and evicts the least recently used.
• degraded-uncachedtestRetaining a reachable set or result produced past a budget, or under the narrowing posture, raises VisibilityDegradedCached.
  Why

  A-disclosure#mirrored-authorization-carries
• semi-jointestThe semi-join opening relation-orderauthority.compose.relation-orderA registered relation applies, in order: mirrored permission semi-join, tenant equality, credential predicate, table policy, project default, table zone, then the column projection carrying masks and zone nulls. admits a row of a bound table when its resource_key value names a resource with a read-conferring, unexpired, untombstoned grant in the reader’s reachable set.
• read-leveltestA grant confers read when its level, mapped through the level table of the pack mapping its table, or taken verbatim where no pack maps that table, is read, write or admin.
• public-granttestA grant whose principal_kind, mapped through the principal_kind table of the pack mapping its table, or taken verbatim where that table names no such kind, is public joins every subject’s reachable set.
• cache-epochstestThe reachable-set cache keys on the subject, the directory epoch and the epoch of each source the read touches, so a sweep invalidates only sets reading its source.
  Why

  A-disclosure#mirrored-authorization-carries

bound-staleness

The declared age budget on mirrored authorization, the refusal past it, and the one narrowing posture.

• budget-grammartestmax_acl_staleness is a positive integer with one suffix from s, m, h, d. A compound form, a bare number or another suffix raises VisibilityBudgetMalformed, naming the table and the text read.
  Why

  A-disclosure#mirrored-authorization-carries
• access-staletestA read raises VisibilityAccessStale (HTTP 503, carrying source, lag, budget and last observation) when a touched table’s source lag exceeds its budget, the source has no completed full run, or the public-status watermark is past budget.
  Why

  A-disclosure#mirrored-authorization-carries
• budget-below-cadencetestA budget tighter than the sweep cadence its source sustains raises VisibilityBudgetUnreachable at diagnose, naming both figures.
  Why

  A-disclosure#mirrored-authorization-carries
• clock-offsettestCONTEXTFUL_NOW_OFFSET_SECS, a non-negative integer, moves the instant a staleness check reads as now that many seconds later, so it can refuse a read and never extends a budget.
  Why

  because a test observes a budget elapse without waiting, and an offset that only ages state cannot admit a read
• offset-malformedtestA CONTEXTFUL_NOW_OFFSET_SECS value other than a non-negative integer stops the process before it serves, naming the variable.
• public-watermarktestA table’s public-status watermark is the oldest acl_observed_at among its source’s resources holding a read-conferring, unexpired public grant; with no such resource it lags zero.
  Why

  because an incremental run over the public resources refreshes each one’s observation, so public status stays current while the full watermark lags
• public-onlytestPast its budget, a public_only table whose source has a completed full run and a public-status watermark within budget serves only rows whose resource holds a read-conferring, unexpired public grant.
• degraded-blocktestA read touching a table served under public-onlydisclosure.bound-staleness.public-onlyPast its budget, a public_only table whose source has a completed full run and a public-status watermark within budget serves only rows whose resource holds a read-conferring, unexpired public grant. answers with a contextful.visibility block holding degraded true and the narrowed tables under narrowed.

declare-fidelity

The fidelity level a table claims, the source family bounding the claim, and live federation under a reader’s delegated credential.

• family-boundtestA person-container table at a servable level, or a directory table above excluded, raises VisibilityFamilyBound, and the manifest declaring it does not load.
  Why

  A-disclosure#fidelity-bounded-what
• family-undeclaredtestA mapping landing a table without naming its family raises VisibilityFamilyUndeclared.
  Why

  A-disclosure#fidelity-bounded-what
• federated-registeredtestRegistering a federated table on a read face raises VisibilityFederatedRegistered.
  Why

  A-disclosure#fidelity-bounded-what

unsettled: Does a grants mapping refuse a roster read from content, a sharing engine’s inputs, or a federated result cached without its subject? owner: disclosure affects: disclosure.declare-fidelity

pack

The reviewed unit landing one source: a manifest fragment plus a connector pin, its access mapping, defaults and container allowlist.

• mapping-absenttestA pack landing a table with neither an access mapping nor excluded raises VisibilityMappingAbsent at diagnose, naming the pack and the table.
  Why

  A-disclosure#fidelity-bounded-what
• asserts-accesstestAn access-shaped key the mapping schema does not define raises VisibilityPackAssertsAccess by name. Fields outside the mapping are fetch inputs and reach no authorization decision.
  Why

  A-disclosure#fidelity-bounded-what
• any-of-allowlisttestAn allowlist evaluated as any-of raises VisibilityAllowlistAnyOf at load.
  Why

  P1
• unsupported-coarsetestDeclaring coarse where the source emits no container or workspace signal raises VisibilityUnsupportedCoarse.
  Why

  A-disclosure#fidelity-bounded-what
• pack-blocktestA [pack.<name>] block names the tables it lands, its excluded tables, one access.<table> mapping per mapped table, and an optional container allowlist of containers with evaluate = "all-of".
• mapping-keystestAn access mapping holds RawMapping and no other key.

drive-sweep

Mapping Drive file permissions into the governed source’s visibility rows.

• resource-setA Drive sweep enumerates each file row of its bound connection as one governed resource under sweep-blockdisclosure.sweep.sweep-blockAn [[acl_sweep]] block declares one source's sweep as RawBlock: enumerate names the table whose rows are its governed resources, and each cadence is { every = <budget> }..
• permission-pagesFor each governed file, the sweep follows every permissions.list page under the connection credential before its grants replace prior grants.
  Why

  A-disclosure#drive-permission-coverage
• grant-mappingUser, group, domain and anyone permissions map to distinct principal kinds; the mapping retains Drive permission id and role beside the source resource id.
• inherited-accessA shared-drive permission records its inheritance source when Drive supplies one; a direct permission and an inherited permission remain distinct grants.
• unreadable-resourceA refused or incomplete permission read marks its resource unread and advances no coverage watermark under coverage-watermarkdisclosure.sweep.coverage-watermarkA sweep run that failed or skipped any governed resource of its source and advances watermark_at raises VisibilityWatermarkUncovered, and the watermark stays where it stood..
  Why

  A-disclosure#drive-permission-coverage
• identity-linkAn email or domain from Drive becomes a reachable subject only through a verified identity link under subject-principalsdisclosure.reach.subject-principalsA subject user://<p> reaches the user principal spelled <p> and each principal an identity_links row maps to it, then the groups those principals belong to..

Scenarios

• disclosure.drive-sweep.permission-pagesdisclosure.drive-sweep.permission-pagesFor each governed file, the sweep follows every permissions.list page under the connection credential before its grants replace prior grants.: WHEN a file’s permission response has a next-page token, THEN the sweep reads that page before replacing its grants.
• disclosure.drive-sweep.unreadable-resourcedisclosure.drive-sweep.unreadable-resourceA refused or incomplete permission read marks its resource unread and advances no coverage watermark under coverage-watermark.: WHEN one file’s permissions refuse, THEN no complete sweep watermark advances.

Shapes

The access tables, each at _visibility/<source>/<name>. Every instant is an engine UTC instant taken when the sweep read the source.

CREATE TABLE resources (
  resource_id       VARCHAR NOT NULL,
  source            VARCHAR NOT NULL,
  resource_kind     VARCHAR NOT NULL,
  container_id      VARCHAR,
  visibility_class  VARCHAR NOT NULL,   -- mirrored | coarse | federated | unknown
  acl_observed_at   TIMESTAMP NOT NULL,
  acl_sweep_kind    VARCHAR NOT NULL,   -- full | incremental | webhook
  acl_sweep_id      VARCHAR NOT NULL
);

CREATE TABLE grants (
  resource_id       VARCHAR NOT NULL,
  principal         VARCHAR NOT NULL,
  principal_kind    VARCHAR NOT NULL,   -- user | group | team | channel | org | public
  level             VARCHAR NOT NULL,   -- source spelling, projected by the mapping
  expires_at        TIMESTAMP,          -- set on an operator-landed opening
  acl_observed_at   TIMESTAMP NOT NULL,
  acl_sweep_kind    VARCHAR NOT NULL,
  acl_sweep_id      VARCHAR NOT NULL
);

CREATE TABLE principals (
  principal         VARCHAR NOT NULL,
  principal_kind    VARCHAR NOT NULL,
  source_label      VARCHAR
);

CREATE TABLE group_members (
  "group"           VARCHAR NOT NULL,
  member            VARCHAR NOT NULL,
  member_kind       VARCHAR NOT NULL,
  acl_observed_at   TIMESTAMP NOT NULL
);

CREATE TABLE identity_links (
  source_principal  VARCHAR NOT NULL,
  subject           VARCHAR NOT NULL,
  method            VARCHAR NOT NULL,   -- scim_email | oidc_sub | operator_asserted
  confidence        DOUBLE
);

CREATE TABLE tombstones (
  scope             VARCHAR NOT NULL,   -- resource | grant | principal
  resource_id       VARCHAR,
  principal         VARCHAR,
  level             VARCHAR,
  revoked_at        TIMESTAMP NOT NULL
);

CREATE TABLE freshness (
  source              VARCHAR NOT NULL,
  epoch               BIGINT NOT NULL,      -- one per committed run
  sweep_id            VARCHAR NOT NULL,     -- the run whose resources and grants rows are current
  sweep_kind          VARCHAR NOT NULL,
  watermark_at        TIMESTAMP,            -- null until a run reads every governed resource
  last_full_sweep_at  TIMESTAMP,
  last_incremental_at TIMESTAMP,
  lag_seconds         BIGINT,
  budget_seconds      BIGINT
);

A bound content table, the sweep feeding it, and a pack’s mapping:

[pipeline.tables.pages.visibility]
source             = "wiki"
resource_key       = "page_id"
resource_kind      = "page"
fidelity           = "mirrored"
family             = "item-exception"
max_acl_staleness  = "15m"
on_stale           = "refuse"

[[acl_sweep]]
source        = "wiki"
credential    = "secret://wiki/permission-reader"
enumerate     = "pages"
full          = { every = "1h" }
incremental   = { every = "5m" }
events        = { stream = "wiki.permissions", gap_detectable = true }

[pack.wiki]
tables         = ["pages", "drafts"]
excluded       = ["drafts"]
allowlist      = { containers = ["eng", "ops"], evaluate = "all-of" }

[pack.wiki.access.pages]
resource_key   = "page_id"
resource_kind  = "page"
family         = "item-exception"
principal_kind = { user = "user", group = "group", workspace = "org", anyone = "public" }
level          = { viewer = "read", commenter = "read", editor = "write", owner = "admin" }

The levels each source family permits:

The envelope a bound read returns:

{
  "rows": 42,
  "visibility": {
    "fidelity": "coarse",
    "grain": "teamspace",
    "acl_lag_seconds": 212,
    "acl_budget_seconds": 900,
    "degraded": false,
    "group_closure_depth": 3,
    "federated_legs": [{ "source": "drive", "outcome": "timeout", "citations": 0 }]
  }
}