Invocation#
transcript-lake [--data-dir <path>] stream [--json]Purpose#
Continuously catch up supported sources and commit masked appends into the Lake and Oko projection.
Inputs and options#
- No positional input is required or accepted.
--jsonchanges each status line to one JSON record.- At least one supported source root must already exist.
Output and state effect#
Mutating and long-running. Installs recursive watches before catch-up, writes masked canonical events, cursors, stream state, and Oko projection updates beneath the selected root, and stops cleanly on SIGINT or SIGTERM. Each commit takes the single state-writer lease.
On a modified source, the stream verifies the previously consumed prefix before resuming. A rewritten or truncated source, or a legacy cursor without proof, replays through masking without duplicating retained canonical occurrences. Recovery also restores missing Oko rows and preserves historical records the vendor removed. Its diagnostic names the source, reason, previous offset, and observed bytes; per-runtime summaries include replayedFiles. Recovery advances the cursor only after the whole source succeeds, so a later failure remains safe to retry.
A file that vanished after it was named is skipped. A session file that cannot be inspected or streamed, and a notified path inside a runtime's root whose type cannot be read (so no adapter can tell whether it is a session), are failures of that runtime: each is named on stderr with its cause and counted in that runtime's and the pass's failures, which the stream state records; a catch-up with failures leaves the state degraded. Every error a run returns once it knows its data directory (no supported source root, a root it cannot watch, the stop handler, a catch-up that cannot run, such as one with no masking declaration in TRANSCRIPT_LAKE_SECRET_FORMATS) writes the state failed with its error and failedAt before the process exits non-zero with the same error; each later start writes catching-up carrying that failure as lastFailure until a catch-up completes and writes running or degraded. The real journey tests/stream/failure.rs (cargo test --test stream-failure) starts the real stream with no session root, with no declaration twice and with one, and reads each state and status.
Exact refusals#
- With no supported roots it refuses exactly
stream found no supported source roots on this machine. - Unreadable or corrupt cursor state is refused rather than reset.
- Recovery against damaged canonical evidence refuses as
cannot replay source against corrupt canonical partition <path> at line <line>: <cause>. - A canonical write failure names
cannot commit canonical partition <path>: <cause>. - Writer contention is refused as
state writer lock is held by <host> pid <pid>. - Any positional argument is refused as
stream accepts flags only. - An unrecognized
--watis refused asunknown stream flag: --wat. - Repeating
--jsonis refused asstream received duplicate --json.
All long flags are exact and single-use. A value flag never consumes another --flag. Global --data-dir may appear anywhere in the invocation, must appear once, and selects an absolute root.