ModusBrain v0: Postgres-Native Personal Knowledge Brain
Historical design doc. This is the original v0 spec from before PGLite landed. Several forward-looking sections — most notably the SQLite engine plan — were superseded by PGLite (embedded Postgres via WASM), which uses the same SQL dialect as Postgres and eliminates the need for a separate FTS5/sqlite-vss translation layer. Kept here for historical context; seeENGINES.mdfor the current engine architecture and theCHANGELOG.mdfor the actual implementation history.
What this is
ModusBrain is a compiled intelligence system. Not a note-taking app. Not “chat with your notes.” Every page is an intelligence assessment. Above the line: compiled truth (your current best understanding, rewritten when evidence changes). Below the line: timeline (append-only evidence trail). AI agents maintain the brain. MCP clients query it. The intelligence lives in fat markdown skills, not application code. The core insight: personal knowledge at scale is an intelligence problem, not a storage problem.Why it exists
A 7,471-file / 2.3GB markdown wiki is choking git. Git doesn’t scale past ~5K files for wiki-style use. The compiled truth + timeline model (Karpathy-style knowledge pages) is right, but it needs a real database underneath. There’s already a production-grade RAG system (Ruby on Rails, Postgres + pgvector) with 3-tier chunking, hybrid search with RRF, multi-query expansion, and 4-layer dedup. ModusBrain ports these proven patterns to a standalone Bun + TypeScript tool.The knowledge model
Architecture decisions
v0 stack
What we chose and why
Postgres over SQLite. We have 3+ years of proven RAG patterns running on Postgres. tsvector for full-text search, pgvector HNSW for semantic search, pg_trgm for fuzzy slug matching. Porting these to SQLite would mean reimplementing search from scratch. SQLite is a future pluggable engine for lightweight open source users (seedocs/ENGINES.md).
Supabase over self-hosted. Zero maintenance. The brain should be infrastructure that AI agents use, not something you administer. Free tier has pgvector but only 500MB (not enough for 7K+ pages with embeddings, which need ~750MB). Pro tier at $25/mo gives 8GB. No Docker, no self-hosted Postgres in v1.
Full port over minimal viable. The patterns are proven. The port is mechanical. Shipping the full 3-tier chunking + hybrid search + 4-layer dedup means world-class RAG from day one. “We’ll add that later” means rebuilding everything later.
Library-first distribution. modusbrain is an npm package. OpenClaw installs it as a dependency (bun add modusbrain), imports the engine directly. Zero-overhead function calls, shared connection pool, TypeScript types. The CLI and MCP server are thin wrappers over the same engine.
Trigger-based tsvector (not generated column). To include timeline_entries content in full-text search, the tsvector needs to span multiple tables. Generated columns can’t do cross-table references. A trigger on pages + timeline_entries updates the search_vector.
Auto-embed during import. No separate embed step. modusbrain import chunks and embeds in one pass. Progress bar shows status. --no-embed flag for users who want to defer. embedded_at column enables modusbrain embed --stale for backfill.
Distribution model
- Library:
src/core/index.ts(BrainEngine interface, PostgresEngine, types) - CLI binary:
src/cli.ts
First-time experience
Path 1: OpenClaw user (primary)
OpenClaw is the AI orchestrator that uses modusbrain as its knowledge backend. This is the most common install path.clawhub install modusbrain:
- Installs the
modusbrainnpm package - Ships SKILL.md files (ingest, query, maintain, enrich, briefing, migrate)
- Registers brain tools with the orchestrator
- Runs
modusbrain init --supabaseon first use (guided wizard)
Path 2: CLI user (standalone)
Path 3: MCP user (Claude Code, Cursor)
The init wizard in detail
modusbrain init --supabase runs through these steps:
CLI commands
Database schema
9 tables in Postgres + pgvector:pages.slug: UNIQUE constraint (implicit B-tree)pages.type: B-treepages.search_vector: GIN (full-text search)pages.frontmatter: GIN (JSONB queries)pages.title: GIN with pg_trgm (fuzzy slug resolution)content_chunks.embedding: HNSW with cosine ops (vector search)content_chunks.page_id: B-treelinks.from_page_id,links.to_page_id: B-treetags.tag,tags.page_id: B-treetimeline_entries.page_id,timeline_entries.date: B-tree
Search architecture
Chunking strategies
Dispatch: compiled_truth gets semantic chunker. Timeline gets recursive chunker. Override with
--chunker flag or chunk_strategy in frontmatter.
Skills (fat markdown, no code)
Each skill is a markdown file that AI agents (Claude Code, OpenClaw) read and follow. The skill contains the workflow, heuristics, and quality rules. No skill logic is in the binary.CEO scope expansions (accepted for v0)
- CLI/MCP parity with drift tests. Both interfaces are thin wrappers over the engine. Tests assert identical output.
- Smart slug resolution. Fuzzy matching via pg_trgm for reads. Writes require exact slugs.
modusbrain get "dont scale"resolves toconcepts/do-things-that-dont-scale. - Brain health dashboard.
modusbrain healthshows page count, embed coverage, stale pages, orphans, dead links. - Normalized timeline.
timeline_entriestable only (no TEXT column).detailfield supports markdown. - Page version control.
page_versionstable stores full snapshots (compiled_truth + frontmatter + links + tags).modusbrain history,modusbrain diff,modusbrain revertcommands. Revert re-chunks and re-embeds. - Typed links + graph traversal.
link_typecolumn (knows, invested_in, works_at, etc.).modusbrain graphuses recursive CTE with max depth (default 5, configurable via--depth). - Trigger.dev data cleanup jobs. Daily embed backfill, weekly stale detection + orphan audit + tag consistency.
- Stale alert annotations. Search results flag pages where compiled_truth is older than latest timeline entry.
- Timeline merge on ingest. Same event created across all mentioned entities.
Security model (v0)
Single-user, local-only:- Supabase service role key in
~/.modusbrain/config.json(0600 permissions) - MCP stdio transport is inherently local (client spawns
modusbrain serveas subprocess) - No multi-user, no RLS, no OAuth in v0
- Multi-user path (future): Supabase RLS + per-user API keys
Upgrade mechanism
modusbrain upgrade detects the installation method and updates accordingly:
Version check: compare local version against latest GitHub release tag.
Storage and cost estimates
Storage (~750MB for 7,471 pages)
Supabase free tier (500MB) won’t fit. Supabase Pro ($25/mo, 8GB) is the starting point.
Embedding cost (~$4-5 for initial import)
Budget alternative:
modusbrain import --chunker recursive skips sentence-level embeddings, then modusbrain embed --rechunk --chunker semantic upgrades later.
Serverless operations stack
Verification checklist
modusbrain import /data/brain/migrates all 7,471 files losslesslymodusbrain exportround-trips to semantically identical markdownmodusbrain query "what does PG say about doing things that don't scale?"returns relevant hybrid search resultsmodusbrain servestarts MCP server connectable by Claude Code- All 3 chunkers produce correct output with test fixtures
modusbrain init --supabaseworks end-to-endbun testpasses all testsclawhub install modusbraininstalls the skill and runs guided setupbun add modusbrain+import { PostgresEngine } from 'modusbrain'works in external project- Drift tests pass: CLI and MCP produce identical results
modusbrain healthoutputs accurate brain health metrics- Migration skill successfully imports an Obsidian vault
Future plans
Seedocs/ENGINES.md for the pluggable engine architecture and future backend plans.
v1 candidates (deferred from v0)
modusbrain asknatural language CLI alias. Trivial to add. P1 TODO.- Intelligence compiler. Treat every fact as a first-class claim with source span, entity links, validity window, confidence, and contradiction status. “What changed, why, and what evidence would flip it again?” From Codex review. Builds on compiled truth model.
- Active skills via Trigger.dev. Application-specific briefings, meeting prep. Belongs in OpenClaw, not generic brain infra.
- Multi-user access. Supabase RLS + per-user API keys. v0 is single-user.
- SQLite engine. Superseded by PGLite (embedded Postgres 17 via WASM) before v1. See
ENGINES.mdfor the current engine architecture. - Docker Compose for self-hosted Postgres. Community PRs welcome.
- Web UI. Optional Vercel-hosted dashboard for browsing brain pages.
Interface abstraction principle
All operations go throughBrainEngine. The engine interface is the contract. Postgres-specific features (tsvector, pgvector HNSW, pg_trgm, recursive CTEs) are implementation details inside PostgresEngine. The interface exposes capabilities, not SQL.
This means:
- A SQLite engine can implement
searchKeywordusing FTS5 instead of tsvector - A SQLite engine can implement
searchVectorusing sqlite-vss instead of pgvector - A future DuckDB engine could implement analytics-heavy workloads
- The CLI, MCP server, and library consumers never know which engine runs underneath
ENGINES.md for the full interface spec. (The original SQLite engine plan was superseded by PGLite; the contract-first BrainEngine interface made that swap clean.)