Skip to main content

​
System of record

The GitHub repo (markdown + frontmatter) is the system of record. The Postgres/PGLite database is a derived cache. We do not back up the database — we rebuild it from the repo. This document is the canonical reference for that contract. Every code path that writes user-knowledge state should match the pattern described here. The CI gate at scripts/check-system-of-record.sh enforces it programmatically.

​
Why this matters

The DB is a derived index over the markdown content. It exists to make search fast, to dedup embedding-similar claims, to materialize the cross-page graph. None of that data is irreplaceable — as long as the markdown is intact, modusbrain sync && modusbrain extract all rebuilds the entire DB from scratch. This means:
  • Disaster recovery is one command. If your DB volume corrupts, if Postgres eats itself, if PGLite’s WASM lock wedges — you don’t need a backup. You wipe the DB, re-import from your brain repo, and the derived state regenerates. v0.32.3 ships modusbrain rebuild --confirm-destructive as the documented one-liner.
  • Multi-machine sync is git. Your brain is a repo. Push from one machine, pull from another, and the second machine’s DB rebuilds on its next sync. No “back up the database” step.
  • Privacy is in your hands. Sensitive entity pages can be gitignored (via modusbrain.yml db_only paths or per-page) and they stay on disk but not in git. The fence respects whatever git tracking choice you make at the page level.
  • Cross-agent collaboration is possible. Multiple agents can write to the same brain because the fence is the merge point, not the DB. Git handles concurrent edits the way git handles concurrent edits.

​
The three categories

Every table in the modusbrain schema belongs to exactly one of three categories. The category determines how it gets rebuilt during disaster recovery.

​
FS-canonical (markdown is the source of truth)

These are user-authored knowledge. The DB row is a derived index over the markdown — wipe the table and modusbrain extract rebuilds it identically. The CI gate keeps direct DB writes from drifting away from the markdown contract.

​
Derived from FS but not user-authored

These hold derived state that’s automatically reconstructible from the markdown but not directly authored as markdown by the user. The chunker + embedder rebuild these on import.

​
DB-only by design (named exceptions)

These hold runtime / infrastructure state that’s intentionally not in the repo. The architectural rule still holds — these aren’t “user knowledge” — but they’re DB-only by design. A new derived table that holds user-knowledge MUST land FS-first. If you’re tempted to add one as “DB-only for now,” the structural question is: does it belong in this DB-only-by-design list? If not, it’s FS-canonical and needs a fence (or frontmatter field) plus a reconciler.

​
The privacy boundary

Private knowledge in a fence still lives in the markdown file. If the user commits the page to git, the private data lands in git too. This is the existing operational model — we don’t infer git policy. For untrusted readers (remote MCP, subagent), the v0.32.2 release ships a 3-layer strip:
  1. Layer A (chunker): src/core/chunkers/recursive.ts calls stripFactsFence({keepVisibility: ['world']}) + stripTakesFence before chunking. Private fact text never reaches content_chunks.chunk_text, embeddings, or search results.
  2. Layer B (get_page): when ctx.remote === true, the response body has both fences stripped (private rows from facts; entire takes fence). Local CLI (ctx.remote === false) sees the full fence.
  3. Layer C (git tracking): the user decides whether to commit the entity page. modusbrain.yml db_only paths are gitignored automatically; per-page choices via the user’s normal git workflow.
For universally-private entities (a friend’s name, an investor’s internal notes), mark the entity page’s directory as db_only in modusbrain.yml. The file stays on disk but never lands in git.

​
The forget contract

modusbrain forget <id> and the MCP forget_fact op rewrite the fence row with strikethrough + valid_until = today + context: "forgotten: <reason>". The DB’s expired_at = valid_until + now() derivation reconstructs the forget state on every rebuild because the fence is canonical. Strikethrough has two semantics distinguished by context:
  • ~~claim~~ + context: "superseded by #N" → row was replaced by a newer row in the same fence
  • ~~claim~~ + context: "forgotten: <reason>" → row was retracted via the forget op
Both encodings keep the row in the markdown for audit history. To permanently delete a fact, edit the fence directly in markdown and remove the row. The next extract_facts cycle wipes the DB row.

​
Disaster recovery

The promise the rule makes:
The invariant E2E test at test/e2e/system-of-record-invariant.test.ts exercises this exact flow on every CI run.

​
Rule for new code

When you add a new user-knowledge category:
  1. Define the markdown shape. Fence (<!--- modusbrain:NAME:begin --> ... :end --> table) or frontmatter field.
  2. Build a parser that produces structured data from markdown. See src/core/fence-shared.ts for the shared primitives.
  3. Build a writer that round-trips: parse + edit + render produces byte-identical markdown for identical input.
  4. Add the engine method that takes parsed data and stamps a derived table. The method gets an entry in the CI gate’s banned-direct-call list.
  5. Add a reconciler: a cycle phase that walks pages, parses the fence, and rebuilds the derived table from scratch. The reconciler is the only legitimate call site for the engine method; // modusbrain-allow-direct-insert: <reason> annotates it explicitly.
  6. Add a round-trip test in test/e2e/system-of-record-invariant.test.ts that proves DELETE + reconcile rebuilds the table byte-identically.
The CI gate at scripts/check-system-of-record.sh fails any PR that adds a new direct call to a derived-table writer outside the reconciler / migration layer without the explicit allow-list comment.
  • ~/.claude/plans/system-instruction-you-are-working-expressive-pony.md — the v0.32.2 design plan (decisions D1-D22 + Q1-Q8, Codex round 1 and round 2 finds)
  • skills/migrations/v0.32.2.md — the agent-facing migration guide
  • CHANGELOG.md v0.32.2 entry — the release manifesto
  • scripts/check-system-of-record.sh — the CI gate that enforces the rule