Onboarding — Transcript Lake

This guide moves a new operator from a clean macOS environment to the first observable Transcript Lake result. Core streaming is local, requires no account or credential, and never modifies vendor transcript stores.

Choose the channel#

Development: a build from the canonical source checkout; its receipt must name the revision and any uncommitted changes.
Candidate or stable: an exact version, artifact digest, source revision, and publication receipt from Stado. No immutable Transcript Lake release is currently published.

Do not treat a main archive as an immutable release.

Prerequisites#

Required for every installation#

macOS.
A Rust toolchain (cargo and rustc) at version 1.85 or newer, to build the CLI. Running the built binary needs nothing else.
Local read permission for at least one supported transcript store if you expect non-empty streaming.
Writable local storage for LAKE_DATA.

Check:

uname -s
cargo --version
printf '%s\n' "${LAKE_DATA:-$HOME/.transcript-lake}"

Expected: Darwin, a cargo version of 1.85 or newer, and the intended local state path.

Required only for SQL and compaction#

DuckDB CLI 1.5.x available as duckdb.

duckdb --version

A missing DuckDB binary does not prevent streaming, status, masking, or Oko projection.

Required only for Oko integration#

A compatible oko-cli on PATH, or OKO_CLI set to its executable path.

command -v "${OKO_CLI:-oko-cli}"

Oko is optional. Its absence must not prevent core streaming.

Install an immutable release#

Use this path only after stado release status transcript-lake --json identifies a published artifact. Select the exact version, darwin-arm64 platform, archive coordinate, and SHA-256 from that receipt. A source checkout or a mutable channel name does not identify immutable bytes.

stado release status transcript-lake --json
stado storage get <exact-artifact-uri> <archive-file>
shasum -a 256 <archive-file>
tar -xzf <archive-file>
install -m 755 bin/transcript-lake "$HOME/.local/bin/transcript-lake"
transcript-lake --version

Compare the printed digest with the receipt before extracting or installing. The manifest stages bin/transcript-lake; the installed version must match the selected receipt. Missing provenance, a wrong platform, or a digest mismatch is a refusal, not a reason to choose another download.

These commands do not create a Lake or restart a supervised stream. Coordinate an existing service's desired binary and lifecycle through Stado's service declaration. No immutable release currently exists, so the placeholders deliberately do not name invented assets.

Development installation#

Maintainers may install from the public source tree:

git clone https://github.com/wisent-ai/transcript-lake.git
cd transcript-lake
cargo install --path .
transcript-lake --version

cargo install places the binary in ~/.cargo/bin, which must be on PATH. To build without installing, run cargo build --release and invoke target/release/transcript-lake directly.

This selects mutable development source and is not a production release coordinate.

Zero state#

Run without arguments before choosing a data path:

transcript-lake

Expected: purpose, safe starting commands, global flags, default state path, and a help URL. This command must not create files, read transcript stores, contact external services, or start background work.

Minimum configuration#

Transcript Lake defaults to ~/.transcript-lake. Select another root for one invocation without editing shell configuration:

transcript-lake --data-dir "$HOME/.transcript-lake" paths

Automation may instead set LAKE_DATA. Global --data-dir wins for that process. Use an operator-owned local path; Transcript Lake does not edit shell profiles.

There are no core credentials. OKO_CLI is optional and needed only when asking Transcript Lake to invoke Oko.

First successful workflow#

Starting state#

Transcript Lake is installed.
LAKE_DATA identifies a directory the operator owns.
No Transcript Lake stream or other writer is using that directory.
Zero or more supported coding-agent stores may exist under the current user's home directory.

Inspect without mutation#

transcript-lake paths
transcript-lake sources
transcript-lake doctor
transcript-lake status

Expected on a clean setup: resolved paths, zero or more discovered runtime stores, optional-dependency warnings rather than core failure, an absent healthy zero-state, no partitions, no cursors, and no live stream. The Oko freshness line may report that no index or projection exists.

All four commands are read-only. doctor exits non-zero only for corrupt authoritative metadata or a broken installed adapter; missing DuckDB/Oko and absent source stores are warnings.

Adopt an existing source#

Choose one exact runtime/root pair printed by sources, then run the first-use action:

transcript-lake onboarding --source claude --root "$HOME/.claude/projects"
# The same reusable operation after onboarding:
transcript-lake adopt --source claude --root "$HOME/.claude/projects" --json

Adoptable runtimes are claude, codex, omp, droid, and kimi. The root must resolve to the exact root that runtime's adapter discovered. Before taking the writer lease, Transcript Lake opens every candidate and validates the complete supported JSON or JSONL schema. Unknown record types and malformed complete records reject the whole input; a final unterminated JSONL line is retained as pending for a later pass. Factory *.settings.json is validated as one JSON object rather than line-by-line.

Only after preflight succeeds does the normal masking, canonical partition, cursor, and Oko projection boundary ingest the source. The selected native identity (runtime plus canonical root) is then persisted in LAKE_DATA/sources.json. Existing events are preserved, repeat adoption is idempotent through byte cursors and native identity, and older adopted source declarations remain retained. The JSON receipt reports candidate files/events, imported and unchanged files, pending lines, imported events, and the retained Lake root, each as ingestion reported it; an ingestion summary missing one of its counts is refused naming it. No transcript or credential contents are printed. --skip-source leaves an empty usable Lake without recording source completion.

Missing, non-discovered, empty, unreadable, malformed, or unsupported sources exit non-zero. Selection is not changed unless canonical data was actually accepted; an invalid input does not partially mutate Lake state.

Start the stream#

Run in the foreground:

transcript-lake stream

When LAKE_DATA/sources.json selects a source, the process watches and catches up only that exact runtime/root pair. Before any source has been adopted it retains the legacy behavior of watching every supported discovered root. A source append is read directly from its durable byte cursor, masked, committed to canonical partitions and the affected Oko session projection, and then checkpointed. There is no scan interval, timer, or refresh subprocess.

For an always-on development installation from the source checkout:

scripts/install-stream-service.sh

The script installs the release-profile binary outside ~/Documents, removes the obsolete timer/watch LaunchAgents, and loads one KeepAlive LaunchAgent named com.wisent.transcript-lake-stream. Its combined structured log is ~/Library/Logs/transcript-lake-stream.log.

Side effects remain inside the selected LAKE_DATA: cursors, masked NDJSON partitions, stream status, and Oko session projections. Vendor stores remain unchanged, and the stream makes no network request.

Observe and use the result#

transcript-lake status
transcript-lake sessions --limit 20
transcript-lake events --type tool_call --limit 20
transcript-lake search "refactor" --limit 20
transcript-lake show <session-id>
transcript-lake label add <session-id> --aspect reviewed --value yes
transcript-lake label aspects
transcript-lake stats --days 7

Expected: status has a live stream record and cursor count. When supported events exist, partition counts, recent sessions/events, and statistics are non-empty, and text search returns the newest events containing the literal term. show reads one whole conversation back in chronological order, closing with a rendered N of M footer. label add validates the session against the Lake and appends one operator annotation beneath LAKE_DATA/labels. The analytics commands require DuckDB. Add --json for automation.

Safe recovery and derived cleanup#

Source rewrites, truncations, and legacy cursors recover automatically against retained canonical history. Read the recovery diagnostic and inspect the actual restored events. For damaged cursor or canonical state, preserve the current root and reconstruct elsewhere:

transcript-lake rebuild --to "$HOME/.transcript-lake-rebuild"
transcript-lake --data-dir "$HOME/.transcript-lake-rebuild" doctor
transcript-lake --data-dir "$HOME/.transcript-lake-rebuild" status

Preview and optionally remove only rebuildable derived data:

transcript-lake clean --target all
transcript-lake clean --target all --apply

clean never removes authoritative NDJSON, cursors, stream state, or vendor transcripts.

Safe reset and uninstall#

Stop the stream using the same root. Preserve evidence by moving rather than deleting the state:

STAMP=$(date -u '+%Y%m%dT%H%M%SZ')
mv "$LAKE_DATA" "$LAKE_DATA.reset-$STAMP"

A later stream starts from zero. Delete the retained directory only after confirming it is no longer needed. This does not remove vendor transcripts.

Uninstall the global development or release package:

cargo uninstall transcript-lake

A release archive installed by hand is removed the same way it was placed: rm "$HOME/.local/bin/transcript-lake".

Uninstallation leaves LAKE_DATA intact. Remove or retain that state as a separate operator decision.

Common failures#

Symptom	Meaning	Corrective action
`transcript-lake: command not found`	The installed binary is not on `PATH`	Add the install directory (`~/.cargo/bin` after `cargo install`) to `PATH`, then rerun `transcript-lake --version`
`unknown command` or `unknown ... flag`	Input is outside the public CLI contract	Run `transcript-lake --help` and correct the command before retrying
Permission error reading a vendor path	Current user cannot read that source	Correct ownership for the same user; do not elevate the stream
Permission error beneath `LAKE_DATA`	State root is not writable	Select an owned local path and restart; the failed source cursor did not advance
`duckdb failed to start`	Optional SQL dependency is absent	Install compatible DuckDB; streaming and status remain available
`oko-cli is not on PATH`	Optional Oko integration is unavailable	Install Oko or set `OKO_CLI`; core Lake state remains valid
Empty partition inventory while streaming	No supported complete records have arrived	Confirm an agent is appending within a root reported by `sources`
Cursor store is unreadable	Cursor JSON was damaged	Preserve the current Lake and run `rebuild --to <empty-path>`; never edit cursors or replay into existing partitions
Source-prefix recovery diagnostic	The source was rewritten, shortened, or retained a legacy cursor without proof	Read the source path, reason, offset, and observed size. Recovery preserves masked history automatically; inspect the recovered events and Oko projection. A corrupt canonical partition is a separate refusal.
Another writer holds the state lock	A mutation already owns this root	Let that writer finish or diagnose its process; do not delete a live lock
Interrupted stream	The supervised process stopped	Inspect the owning supervisor's declaration and failure. On the next supervised start, verified appends resume and unfinished source recovery repeats its retained-history comparison.
Historical replay needs too much space	Entire selected history was rebuilt	Retain current evidence and rebuild to a larger empty root or select one runtime

Errors name the failed dependency or path. A source-local failure is logged and leaves that cursor unchanged; fatal startup or authoritative-state failure exits non-zero for the supervisor to restart.

Machine onboarding#

Automation should:

install an exact archive and verify its SHA-256 digest;
pass an explicit absolute --data-dir or set LAKE_DATA;
invoke transcript-lake doctor --json before mutation;
supervise one transcript-lake stream --json process and parse its JSONL lifecycle records;
treat process exit as a fatal lifecycle event and each failure record as source-local;
serialize compaction, reconstruction, and applied cleanup against the stream writer lease;
use status --json, sources --json, and bounded analytics commands for machine diagnostics;
retain selected version, source commit, archive digest, stream log, and cleanup ownership.

Continuous freshness requires exactly one explicit supervised stream. Retention, backups, and access control for the data directory remain operator-managed.