# Verify Claims Against Primary Sources Agent output, review findings, and secondary documents are untrusted input. Trace every load-bearing or disputed claim to a primary source before you act on it: the shipped code, the filesystem, a live tool run, or the exact revision. ## Key points ### Agent output - Treat agent prose as untrusted input. Two agents gave contradictory verdicts on one codebase because they analyzed different scopes. Trust agreement on narrow claims over one agent's confidence. - Verify agent inputs too. A hallucinated name can originate in the plan document, one hop before the executing agent. - Design fact-check passes to separate verified from unverifiable claims, and verify the negatives. One summary read blocked while the claimed reversal never happened. - Check provenance, not just wording: a correct mechanism can carry fabricated anchors. Verify pinned SHAs with `git cat-file`. - Verify AI-reviewer claims like any reviewer: one case had both sides wrong, and one real citation pointed at the corrected measurement, not the error analysis. ### Review findings - Run a verify lane that tries to refute findings. A refuted Critical finding is as valuable as a confirmed one. - Attribute damage before blaming the change. A local formatter hook can mangle the worktree. `git status` shows the truth. Re-verify against a moved head before posting. - A checked-out worktree can hold a stale revision. Fetch files at the exact PR-head SHA before you write a suggestion. - Green CI does not validate the core algorithm. Extract it into a `-race` harness. Reproduce the bug on main, then confirm the head passes it. - Review state has a shelf life of hours. Same-day merges invalidate drafted comments. Re-derive from the live diff. - Verify critique details before posting: one "new dependency" claim was false against `go.mod`, and one backoff snippet inverted a time subtraction. - For a red CI shard, compare sibling PRs on the same base and replay locally. A sibling failure is pre-existing or infra. - Hand verification overturns verdicts. A live reproduction reclassified a review as salvageable, and one PR's motivating claim failed a throwaway repro. ### Version claims - Verify API claims against the shipped source, not the proposal or prior notes. One Go 1.25 feature shipped a different API shape. - Settle runtime claims against the toolchain source under `GOROOT`, and versions and dates with `proxy.golang.org/<module>/@latest`. - Check unreleased-version claims against the open proposal, not the deprecation notice. One replacement did not exist yet. - Re-check version-specific behavior on the current toolchain. One documented benchmark behavior reversed between Go releases. - Read vendor docs at the release tag, never at `main`, and confirm README version strings against the releases page. - Verify quotations against the upstream comment, not memory. One quote named the wrong person and described fixed behavior as current. ### Docs and notes - Verify docs against the repository source and the filesystem, not the docs site. One review refuted 13 drafted claims that overstated prerequisites. One PR body called a text file a symlink. - Default search hides files from audits. Use `rg --hidden` or read ignored paths directly. - Verify wire-format objections against the canonical serializer, not the internal struct. - Verify status claims against the live repository. One logbook line claimed a release that was retracted 30 minutes later. Handover notes listed merged PRs as open. - Fact-check numbers against live tool output, and calibrate feedback against the commit timeline. Manual review missed a factor-of-10 error. - Check quote-claim fit. A verbatim quote, or a grep match without section context, can support one sub-claim while the compound claim falls. - Re-verify claims of absence by independent search. A false negative is the most dangerous fact-check error. - Prove patterns empirically: an A/B repro of a one-line diff, or the upstream CLI against testdata fixtures. - Approving a doc approves its contradictions. Capacity claims need a source a reader can open. ## How to apply 1. Reproduce first: a throwaway repro, a `-race` harness for concurrency claims, or the upstream CLI on testdata fixtures. 2. Pin the revision: fetch files at the PR-head SHA, read vendor docs at the release tag, check PR state with `gh pr view --json mergedAt`. 3. Try to refute before you post: re-derive from the live diff, re-verify against a moved head, and attribute worktree damage with `git status`. ## Observations - [practice] trace disputed agent claims to the shipped source before you trust a verdict #verification (source: agent sessions, 2026-07) - [failure-mode] a hallucinated name entered through the plan document, one hop before the executing agent #agent-trust (source: agent sessions, 2026-06) - [failure-mode] a docs page stated an overridable degradation path as a hard prerequisite #docs (source: agent sessions, 2026-08) - [practice] the module proxy settles version and date claims in one fetch #go (source: agent sessions, 2026-07) - [practice] re-verify claims of absence by independent search #fact-checking (source: agent sessions, 2026-09) - [failure-mode] two mechanisms failing identically point at the shared path #debugging (source: agent sessions, 2026-06) ## Relations - relates_to [[resources/documentation/Go/Benchmarks/Go Benchmarks - Three Questions Before You Trust a Benchmark]] (a runnable check before you trust a number) - relates_to [[resources/concepts/Cross-Domain Pattern Analysis]] (verify-first across review, research, docs, and versions) - relates_to [[resources/documentation/Agents/Agents - Plan First, Fresh Sessions, Write It Down]] (the canon that treats agent output as untrusted input) ## Sources - Personal agent-session notes, February 2026 to November 2026