The problem
Diligence takes teams of associates weeks of reading filings, news, dockets and data-room financials, and the worst issues hide between sources. Autonomous agents could help, but they are the hardest AI pattern to control, so high-stakes work needs human checkpoints.
What I built
- Agent Core: a lead agent (
CLAUDE.md) running a bounded Plan, Act, Observe loop (at most three iterations). - Four specialists in parallel: SEC filings, news, litigation and financials. One source each, read-only.
- Independent citation verifier: re-opens every cited source and recomputes the numbers behind each Critical and High finding.
- Working memory as plain files: plan, findings register, conflicts, run log and every unedited agent report. Fully auditable.
- Three human checkpoints: approve the plan, review the red flags, sign off.
- Testable guardrails: no web access, protected sources, a prompt-injection rule, approval to open the answer key, 69 automated checks.
Key decisions
- Specialists by source, not topic, so connecting sources is the lead agent's job. The Critical issues only appear there.
- Separation of duties: analysts cannot change anything; a separate verifier checks them, like a deal-team reviewer.
- Measured, not assumed: 18 planted issues, 4 distractors and an answer key the analysts and verifiers never see.
- Planned like a product: specs and a phased plan with tests for each phase, agreed before building.
Real filings: two backtests
Two backtests ran on a real company's SEC filings as of a cutoff date (2018 and 2014), with a hypothetical deal and a key of warning signs the later public record showed mattered. Limits first: one company each, picked by a session that knew the outcome; no distractors, so false alarms are untested; the companies and reports stay private. Found: 10 of 10 signs, then 5 of 8 plus 3 in part (81.3% scored, 62.5% on full finds alone). Both: 0 unsupported findings, 0 hindsight leaks.
Architecture: Agent Core, four specialists, working memory, verifier and human checkpoints.
What the system concluded
All six Falcon runs: do not proceed at $24.00 per share. The committee decides.
- The valuation used guidance the target had already replaced: about 18.3x on current guidance, not the 11.5x in the deal model.
- A patent case turned against the target in July, with an injunction sought on robots behind about 42% of revenue. The next quarterly filing left the ruling out.
- A customer worth 31% of revenue can leave after a change of control, and is building its own robots.
What I learned
- The verifier earned its place: 17 of 61 findings needed corrected wording before the report (one claimed 18 months of negative free cash flow; it was 12).
- Three runs, not one: the same misses repeated, so they were design gaps, not luck. I fixed them, and run 4 found all 18.
- Then quality: Critical must name its test, duplicates fold into one parent, cross-source findings are self-checked. Reports fell from 38 to 13 pages; recall held at 100%.
- Then generalization: four held-out targets, 96% to 100% of Critical and High issues found, nothing invented. Exact severity is the open gap: 77.8%, 88.9%, 72.2%, 72.2%.