Skip to main content

​
Progress events

Canonical reference for the JSONL progress stream that modusbrain writes to stderr when a bulk command runs with --progress-json. Stable from v0.15.2. Additive changes only; no renames or removals without a major version bump. Most humans won’t read this page. Agents parsing progress will.

​
When do I get these events?

Any of these commands stream events when --progress-json is set:
  • modusbrain doctor (DB checks, JSONB integrity, markdown body completeness, integrity sample)
  • modusbrain orphans
  • modusbrain embed
  • modusbrain files sync
  • modusbrain export
  • modusbrain extract [links|timeline|all] (fs or db source)
  • modusbrain import
  • modusbrain sync
  • modusbrain migrate --to …
  • modusbrain repair-jsonb
  • modusbrain check-backlinks
  • modusbrain lint
  • modusbrain integrity auto
  • modusbrain eval
  • modusbrain apply-migrations (the orchestrator + every child command)
Non-bulk commands (stats, graph-query, get, put, etc.) don’t emit events — they return in under a second.

​
Channel

  • Progress events: stderr, one JSON object per line, \n-terminated.
  • Data results (--json payloads from each command): stdout.
  • Final human summaries: stdout.
Agents can safely capture stdout for their result parsing and read stderr separately for progress.

​
Flags

Global flags: parsed by src/core/cli-options.ts before command dispatch, so modusbrain --progress-json doctor works the same as modusbrain doctor --progress-json (the latter also works — per-command parsers see the flag via the shared CliOptions singleton).

​
Event types

Every event is a single-line JSON object with these common fields:

​
start

Emitted when a phase begins.
Optional fields:
  • total — the total item count if known at start.

​
tick

Emitted periodically during iteration. Time- and item-gated: the reporter won’t emit more often than minIntervalMs (default 1000) and minItems (default max(10, ceil(total/100))).
Fields:
  • done — items completed in this phase.
  • total — total items, if known. Omitted when the scan doesn’t have a total up front (e.g. a streaming iterator).
  • pct — done/total * 100, one decimal. Omitted when total is unknown.
  • eta_ms — projected ms until done === total, from the observed rate. Omitted when total is unknown.
  • note — optional string with the current item (e.g. a slug or filename).

​
heartbeat

Emitted for long-running single operations that don’t iterate (e.g. SELECT against a 50K-row table). No done, no total — just a signal that work is still happening.

​
finish

Emitted when a phase completes normally.

​
abort

Emitted by a single process-level SIGINT/SIGTERM handler that tracks every live phase. After abort, no further events emit for that phase.

​
Phase names

Phases use snake_case.dot.path naming. A fresh reporter starts at the root; child() composition appends to the parent’s current phase, so a sync that calls import emits sync.import.<file>, not import.<file>. Stable phase names shipped in v0.15.2:
  • doctor.db_checks (umbrella for all DB-side doctor checks)
  • orphans.scan
  • embed.pages
  • extract.links_fs, extract.timeline_fs, extract.links_db, extract.timeline_db
  • import.files
  • sync.deletes, sync.renames, sync.imports
  • migrate.copy_pages, migrate.copy_links
  • repair_jsonb.run, repair_jsonb.<table>.<column>
  • backlinks.scan
  • lint.pages
  • integrity.auto
  • eval.single, eval.ab
  • export.pages
  • files.sync
Sub-phases exposed via child():
  • sync.import.files — nested inside a sync
  • apply_migrations.v0_12_2.jsonb_repair — nested inside the orchestrator

​
Subprocess inheritance

When a parent CLI spawns modusbrain … child processes (mostly in src/commands/migrations/*), global flags (--quiet, --progress-json, --progress-interval) are propagated to the child’s argv via the childGlobalFlags() helper in src/core/cli-options.ts. Child stderr passes straight through stdio: 'inherit' so the event stream is one merged JSONL feed on the parent’s stderr. One exception: the orchestrator phase in migrations/v0_12_2.ts that captures child stdout (repair-jsonb --dry-run --json for verification) does not pass --progress-json to avoid any risk of stdout pollution breaking the orchestrator’s JSON.parse. Its stdio is explicit: ['ignore', 'pipe', 'inherit'] so stderr still flows through.

​
Minion jobs

modusbrain jobs work (the Minion worker daemon) keeps progress in the DB, not on stderr. Each Minion handler that runs a bulk core (embed, sync, extract, import, backlinks) calls job.updateProgress({done, total, …}) per iteration. Agents read per-job progress via the get_job_progress MCP operation or modusbrain jobs get <id>. The jobs work daemon itself emits coarse one-line-per-job stderr output for liveness only. Per-page detail lives in the DB.

​
Compatibility

  • Added: only. A new event type, a new field, a new phase name — all safe. Agents must ignore unknown fields and unknown event types.
  • Removed/renamed: never without a major version bump.
  • Schema changes: announced in CHANGELOG.md and in skills/migrations/v<next>.md.
If your agent depends on this schema and something surprises you, open an issue with the event you received and what you expected.