How we verify

The mechanism behind every report — plain-spoken, no marketing.

A summary of an SEC filing is only useful if the numbers are right. A hallucinated figure or a misquoted paragraph in a financial context isn't a rough edge — it's a defect that makes the whole report worse than not reading anything. So we built this the other way around: instead of asking the AI to "be careful," we assume the model will occasionally get things wrong and catch it before you see the report.

Here's what actually happens between the SEC posting a filing and you reading a report.

Every number traces to the source

Load-bearing numbers — revenue, net income, offering prices, share counts, dilution, red-flag figures — are not written by the AI. They come from one of two deterministic sources:

  • SEC XBRL data — the machine-readable financial facts the company itself files. When we display "$416.2B in FY2025 revenue," that value came directly from the company's own XBRL submission, not from a Claude interpretation of the filing.
  • Regex extraction from the filing text — for figures not tagged in XBRL (IPO pricing, dilution tables, trust balances, vote results), we extract them from the filing's actual prose using pattern-matching, then present them verbatim.

The IPO snapshot table on prospectus reports is 100% deterministic — every row is pulled from source text or SEC XBRL, not generated by the model. The label "Prospectus" or "SEC XBRL" on each row tells you which.

Every quote is verbatim, every quote is linked

When a report says "The company disclosed…" and shows a quoted passage, that passage is extracted verbatim from the filing — not paraphrased. The "verify on EDGAR →" link on every quote opens SEC.gov and highlights the exact paragraph, so you can confirm in one click. If the model tries to introduce a specific claim that isn't present in the source, a source-anchoring check catches it before publication.

Cross-surface consistency checks

We run over 100 deterministic checks against every report before it's published. The most load-bearing families:

  • Headline reconciliation — the number in the headline must match the number in the body must match the quoted source. A report whose headline says "margin expanded 60 bp" while the body computes 30 bp is caught.
  • Direction-of-transaction checks — if the source says a company acquired an entity, the report cannot describe the same transaction as a divestiture on any surface (bullets, narrative, section diffs).
  • Magnitude checks — buyback per-share prices, dividend amounts, and other repeated figures must match across every mention in the same report.
  • Fiscal-period attribution — a claim like "raised guidance twice in FY2026" is checked against the actual period boundaries of the events being counted.
  • Substance vs disclosure-label — an 8-K's "Item 5.02 — Departure of Directors" heading does not automatically produce a "CEO departed" red flag; the check requires the source to actually state a substantive leadership change at the public entity.

The publish-gate: reports we can't verify are hidden

When any of the blocking checks fails, the report is withheld from public and paying users. You'll see a "Report withheld pending review" page instead of the report itself. This is not a technical bug — it's the mechanism working. A staff reviewer sees the failed check, fixes the underlying issue, and the report re-publishes.

The tradeoff is deliberate: shipping a report we can't verify is worse for you than delaying it. On a typical day, most reports pass all gates and publish immediately; the ones that don't get a human look before you see them.

What we do not claim

The AI still writes the narrative — the connective analysis, materiality judgments, and interpretation. That prose is grounded to the deterministic figures and quoted source text, but interpretive framing ("this suggests management is de-risking the pipeline") is model output, and you should read it as one analyst's read, not a machine-verified fact. The verifiable layer is the numbers, the quotes, and the sources. The framing is commentary.

We also do not claim zero errors. AI drifts; new filing styles surface new failure modes; our validators catch what we've seen before, not what we haven't. When you find something wrong, tell us — we build a new deterministic check for that failure mode and add it to the gate. That's how the check list grew from ~10 to over 100 in the first six months.

What this means for you

If you're citing a FilingDiff report in your own work: cite the numbers with confidence, cite the quoted paragraphs with confidence (they're verbatim from EDGAR), and treat the narrative as one analyst's read. If you find a number that doesn't reconcile with the source, tell us — it's how we make the next report better.

FilingDiff is not investment advice. All source data comes from public SEC EDGAR filings.