Enrichment Pipeline
Goal
Enrich brain pages from external APIs with tiered spend — full pipeline for key people, light touch for passing mentions, raw data preserved for auditability.What the User Gets
Without this: brain pages are thin shells with only what the user manually typed, API calls are wasted on nobodies, and enrichment data vanishes after the agent session ends. With this: key people have rich, multi-source portraits; spend scales to importance; raw API responses are preserved for re-processing; and cross-references connect the entire graph.Implementation
Tricky Spots
- Don’t overwrite human-written assessments. If the user wrote an Assessment section with their own read on someone, API enrichment NEVER overwrites it. API data goes into State, Contact, Timeline. The user’s assessment is sacrosanct.
- Don’t re-enrich the same page more than once per week. Check
put_raw_datatimestamps before running the pipeline again. Enrichment is expensive and data doesn’t change that fast. - LinkedIn connection count < 20 means wrong person. Crustdata sometimes returns a different person with the same name. If the LinkedIn profile has fewer than 20 connections, it’s almost certainly a false match. Discard it.
- X/Twitter is the most underrated data source. When you have someone’s handle, their tweets reveal beliefs, what they’re building, hobby horses, network (reply patterns), and trajectory (posting frequency, tone shifts). This is richer than LinkedIn for “What They Believe” and “What Makes Them Tick.”
- Cross-references are not optional. After enriching a person, update their company page. After enriching a company, update founder pages. An enriched page without cross-links is a dead end in the graph.
How to Verify
- Enrich a Tier 1 person. Run
modusbrain get <slug>and confirm the page has Executive Summary, State, What They Believe, Contact, and Timeline sections populated from multiple sources. - Run
modusbrain get_raw_data <slug>. Confirm raw API responses are stored withsources.{provider}.fetched_attimestamps. - Run
modusbrain get_links <slug>. Confirm cross-reference links exist to the person’s company page, deal pages, and related entities. - Check a page that was enriched AND has a user-written Assessment. Confirm the Assessment section was preserved, not overwritten by API data.
- Try to re-enrich the same person. Confirm the system checks the
fetched_attimestamp and skips if less than a week old.
Part of the ModusBrain Skillpack.