Architecture

Lifeline's architecture: the deterministic event model, reducers, recall, projection, and the store-is-source cutover.

lifeline-context 0.6.0 · licensed FSL-1.1-MIT (source-available; converts to MIT after two years). Technical document. For the why behind each decision, the source is LIFELINE.md (the decision entries cited in brackets, e.g.: #0002).

10,000-foot view

Lifeline is an event-sourced reasoning ledger. Everything is an immutable, content-addressed Entry, chained in a DAG. The "current truth" and the context an AI reads are derived projections of that ledger — never the source. Three layers of memory, all anchored in the same ledger:

                      Layer 3 · Recall (semantic)
                      anchored embeddings → "what is relevant to the task"
                                  ▲
   Entry (append) ──▶ Layer 1 · Ledger ──reduce──▶ Layer 2 · State ──assemble──▶ payload
                      (hashed DAG,                   (current truth,                (markdown,
                       source of truth)               via reducers)                  by budget)
                                  │
                                  └── projection ──▶ LIFELINE.md (diffable view)

Module map (lifeline/):

1. The event model — deterministic content-addressing [#0002, #0003]

id = sha256( kind \n author \n agent \n provider \n model \n summary \n body.strip()
             \n "|".join(sorted(parents)) \n )   # canonical form ends in a trailing \n

• ts (timestamp) and dedup_key stay out of the hash. Consequence: the same content + the same parents produce the same id on any machine, at any time. This is what makes dedup and merge across nodes/users possible (we are more content-pure than git, whose commit-sha includes the timestamp).
• parents is sorted in the hash → identity is invariant to the order in which the parents were listed (a merge of A+B = of B+A).
• pydantic strict=True. The id is sealed in a model_validator; verify() recomputes and compares (per-entry tamper-evidence).

2. Layer 1 — the Ledger (SQLiteEventStore)

Append-only. entries table (unique id, JSON payload, kind, ts, dedup_key, parents) + edges table (DAG) + unique index on dedup_key. WAL on. append() is idempotent: violating the PK (id) or dedup_key → silently ignored (it solves the "split-brain" problem trivially). stream() returns in causal insertion order (single-writer). It is the runtime source of truth.

EventStore is an abstract port — the SQLiteEventStore is the local adapter; the SupabaseEventStore (shipped in cloud.py) implements the same interface without touching the core. It is the cloud seam, selected with --store supabase / LIFELINE_STORE=supabase.

3. Layer 2 — the State (StateEngine + reducers)

Folds the stream into "current truth" via pure reducers (state, entry) -> state. The default reducer ledger_projection produces: identity (project), decisions in effect, open_items, latest, contributors (authorship aggregated by provider/model), kinds, and superseded.

Integrity gate [gap #G3]: reduce() calls entry.verify() on every entry before folding it. If the stored id doesn't match its content (a tampered .db), the entry is dropped from the truth (never served as a decision) and its id is collected in integrity_broken, which the assembler surfaces as an explicit warning. (ts stays out of the hash by Law #3, so clock tampering is not covered here — a declared product limit; use git as the external notary.)

Status is a projection, not an FSM [#0002]: no execution state machine; the "state" is what the reducers compute. Custom reducers can be registered.

Supersession [#0018, gap #G8]: a correction supersedes its parents — they leave the current truth. Crucially, superseded is not a monotonically-growing set; it's the fixpoint of the correction graph (_effective_superseded). A correction only supersedes while it is itself active; if a later correction supersedes it, its parents come back. The reducer iterates to a stable set, so reverting a reversal restores the original. Append-only throughout: closing/reverting is a new entry, never an edit.

4. Layer 3 — Recall (SemanticRecall + Embedder)

0.6: on SQLite the default is FTSRecall (FTS5/BM25 — §8); the embedder path below remains the seam for dense recall and the fallback where FTS5 is missing.

Embedder is pluggable [#0015]: one model per index. Default LexicalEmbedder — sparse term-frequency, exact cosine, deterministic, no dependency. (We tried hashing first; the test caught a bucket collision producing false relevance → sparse TF gives an exact 0 without a shared token.) SemanticRecall.search(query, k) returns top-k anchored to the event (Law #1) — the vector is only an index; getting the match wrong does not turn into a hallucination. A dense semantic embedder (SentenceTransformerEmbedder, #0029) plugs in behind the same interface — opt-in via the [embeddings] extra; select it with LIFELINE_EMBEDDER=dense (or make_embedder(...)). The default stays lexical (zero-dependency).

Abstention floor [gap #G5]: search() drops any hit below embedder.min_score, so a weak match returns nothing rather than noise. The lexical embedder uses an exact 0.0 floor (no shared token → exactly 0, which already abstains honestly); the dense embedder defaults to 0.3 (LIFELINE_RECALL_MIN_SCORE overrides). Recall would rather say nothing than anchor to a coincidence.

5. Assembly (ContextAssembler)

Renders markdown [#0010] (more token-efficient than JSON/YAML for LLMs) within a budget (chars, or tokens since 0.6). Budget priority: header + "Relevant" (if there is a query) + "Open" + "Resume" + "Recent" + footer are always included; the decisions fill the rest. Since 0.6 the packing is HEAD+TAIL (pinned + oldest in force + newest; the middle omitted with an explicit marker listing the ids, the why excerpt shrinking first — §8), and the payload is split into a stable prefix and a volatile tail (Law #6 — truncation is never silent). Superseded items appear marked [closed/reverted] under "Recent".

6. Projection and the store-is-source cutover [#0014, #0020, #0022]

render_ledger_markdown(store) generates LIFELINE.md from the ledger. ingest_markdown does the reverse path. The two form a fixed point (proven): store → markdown → store reproduces the same ids, and the 2nd render is byte-identical.

This unified the two hashing schemes that coexisted (markdown chain vs Entry id) into the content-addressed id. Git artifact decision [#0022]: what gets committed is the text (LIFELINE.md, diffable/mergeable in PRs); the .lifeline/*.db is the runtime store but is gitignored and rebuildable (lifeline migrate). No contradiction with §2: the .db is the local runtime source of truth, while the committed text is the durable, shareable form — text↔store are losslessly interconvertible, so the real source of truth is the set of content-addressed entries, materialized as both. (In cloud mode the Supabase store is the source; the text follows.)

7. MCP surface (mcp_server.py) [Law #7]

Closes the loop without a human. Resource lifeline://project/context (read) + tools lifeline_append, lifeline_recontextualize, lifeline_recall (write/recall). DB via the env LIFELINE_DB (default .lifeline/ledger.db, relative to the cwd → each project its own line). The same surface runs locally (stdio) and remotely (HTTP/SSE, lifeline-mcp-remote, with an optional OAuth Resource Server) — see docs/MCP_REMOTE.md.

Invariants (summary)

1. No memory without an immutable anchor. 2. Append-only. 3. Deterministic content-addressing.
2. Provider-agnostic storage; per-provider delivery. 5. Why > what. 6. Budget first-class (explicit truncation). 7. MCP-native.

8. 0.6 — long horizon: checkpoints, threads, policy, mesh, ranking, delivery

Working state (checkpoint). A kind, not a table: the body carries thread: plus list blocks goal / done / next / blocked / tried (blocks.py, regex-parsed, order fixed → deterministic body). The reducer keeps the latest live checkpoint per thread (derived supersession — no correction needed; a correction on a checkpoint restores the previous one; a closed open drops its thread's checkpoint). The assembler always includes ## Resume (where work stopped) in the volatile tail. Records reasoning only — it executes nothing (the non-goals hold).

Threads = branches. thread: <name|open id> in the body is the only thing that survives the hash and the round-trip, so it is the membership. log --thread chains to the thread's head (threads[key].head in the state), which makes the DAG a real tree; the first entry of an open's thread hangs off the open. context --thread pins the thread's decisions, filters Resume, and shows the thread's Recent — the brief a sub-agent gets (lifeline_brief).

Write policy. .lifeline/policy.json (resolved from the db's directory, never the cwd) maps agent × kind → direct | propose | deny; precedence `agents[a][kind] > agents[a]["*"] > kinds[kind]

default; an invalid file fails loud. All MCP writes go through one gate (_write):proposeis the HITL path;directseals the entry and writes an audit row into the proposals table (status='auto',entry_id) —lifeline auditreads it;approve all` ignores those rows.

Mesh of lines. promote writes origin:\n line: …\n id: <full id> (an anchor verifiable against the source line; the 0.5 prose note is read as legacy and never re-promoted). A parent may live in another line (--parents research:<prefix> resolves the prefix in that line; parents stays a list of ids, so the canonical form is untouched). verify resolves missing parents in the sibling lines of the same .lifeline/ and reports them as external anchors (default; --no-mesh is the strict single-line check). where and lines --mesh expose the welds. The MCP tools take line= (validated name; db = <server db dir>/<line>.db), so one server serves the whole mesh.

Ranking and layout. When the decisions don't fit, the assembler keeps the pinned ones, the oldest in force (foundation, foundation_share of the budget) and the newest, omitting the middle with an explicit marker that lists the ids; before omitting, the why excerpt shrinks (320 → 160 → 96 chars). Pins are anchored notes (pin: <id>, parents=[id]), unpinned by a correction. The payload is ContextParts(stable, volatile): identity + decisions first (what a provider's prompt cache reuses), open / resume / relevant / recent / footer last.

Budget and delivery (Law #4). budget_tokens measures with a deterministic, language-aware estimate (tokens.py; tiktoken opt-in via the [tokens] extra). deliver(parts, fmt) wraps the same markdown in the provider's system-message shape — anthropic puts cache_control on the stable block; openai sends one system message (stable first); gemini uses system_instruction.

Recall (Layer 3, default). FTSRecall over SQLite FTS5 (lifeline_fts, unicode61 remove_diacritics, incremental by seq, pin notes skipped): build_match turns free text into a safe FTS query (quoted tokens, stopwords dropped, prefix only for 7+ chars, PT↔EN synonyms in OR); BM25 weights the summary 3×; scores are relative to the best hit and min_score=0.25 is the abstention floor. Without FTS5 it falls back to the lexical embedder; dense stays opt-in.

Store and cache. One aiosqlite connection per store (per-loop write lock, synchronous=NORMAL in WAL), closed on exit before the non-daemon thread join; migrate ingests in one transaction; log appends the new block to LIFELINE.md when the file is exactly the projection of the previous n−1 entries (else full regen). StateEngine.reduce() is cached in-process by (store.identity(), reducers) and versioned by (head id, count) — content-addressing makes that version exact; the declared limit (an in-place tamper with the same head and count) is caught by verify, exam, or reduce(fresh=True).