Research · AI Agent Research
Watching the agents work
Each round leaves a record of how the AI systems behaved: what they found, what they got wrong, and what checking caught. This page collects that record. It is a catalogue, not a benchmark: one project, small samples, and roles that are not symmetric. Claude edits every report, so its errors surface mostly through the other models’ audits and the research director’s checks, while the researchers’ errors surface through Claude’s verification.
The protocol’s own rule: a scorecard from any one round does not show that one model is more reliable than another. Behaviour changed between rounds, and the next round could reverse a pattern.
Models used
Names as written in each round’s records. Where a record names only the product, the exact model version was not logged at the time.
| Round | ChatGPT | Gemini | Claude |
|---|---|---|---|
| R01 | ChatGPT Deep Research | Gemini Deep Research | not recorded |
| R02 | ChatGPT Deep Research | Gemini Deep Research | not recorded |
| R03 | not recorded | Gemini Deep Research | Claude Fable 5 |
| R04 | not recorded | Gemini Deep Research | not recorded |
| R05 | ChatGPT Deep Research | Gemini Deep Research | Claude Fable |
| R06 | not recorded | Gemini Deep Research | Claude Fable 5.1 |
| R07 | ChatGPT Deep Research | Gemini Deep Research | Claude Fable 5.1 |
| R08 | ChatGPT Deep Research · not recorded (auditor) | Gemini Deep Research | Claude Fable 5 |
| MED01 | not recorded (P1) · GPT-5.6 Sol (P2) · not recorded (auditor) | not recorded (P1) · Gemini Spark (P2) · not recorded (auditor) | not recorded |
| MED02 | — | — | not recorded |
Raw passes and audits, by round
Published as received, errors included. Each file opens beside a link to its round’s critique.
| Round | ChatGPT | Gemini | Audits and rulings |
|---|---|---|---|
| R01 Organic, Regenerative & Sustainable Agriculture | pending | ||
| R02 The Global Cannabis Cultivation Landscape | pending | ||
| R03 Legal & Illicit Cannabis Production: What Can Actually Be Known | queued next | ||
| R04 The Environmental Footprint of Cannabis: A Map, Not a Number | pending | ||
| R05 Indoor vs. Greenhouse/Light-Dep vs. Sun-Grown Cannabis | pending · 4 pre-audit findings logged | ||
| R06 How Cannabis Is Fed: Nutrient Systems, Not Labels | pending | ||
| R07 Where Cannabis Roots Live: Root-Zone Systems, Not Labels | pending | ||
| R08 Living Soil & Indoor No-Till: What the Evidence Actually Supports | |||
| MED01 Historical Evidence Landscape of Medical Cannabis | |||
| MED02 The Endocannabinoid System and Cannabinoid Pharmacology | — | — | not started |
Verification tallies
From each round’s critique. Identity: does the citation resolve to one real, matching paper. Support: does the paper say what was attributed to it. A dash means the record does not state that number.
| Round | Researcher | Identities checked | Identity failures | Nonexistent | Wrong paper | Support partial / no | Notes |
|---|---|---|---|---|---|---|---|
| R01 | ChatGPT | — | 0 | 0 | — | 0 | No existence or claim-support failures; one title paraphrase (DOI correct). |
| R01 | Gemini | — | 0 | 0 | — | 2 | No existence failures; 2 claim-support failures; 5–6 precision flags. All four replacement citations in the correction round carried a wrong journal or DOI. |
| R02 | ChatGPT | — | 0 | 0 | — | 0 | No existence, claim-support or citation-identity failures. |
| R02 | Gemini | — | 3 | 0 | — | 0 | 3 identity failures (2 right paper at the wrong address, 1 unresolved). Correction round: 4 of 5 returned citations failed identity. |
| R03 | ChatGPT | 21 | 0 | 0 | 0 | 0 | 15 verified, 5 partial (host moved, district misattributed, online-first year, two values not re-extractable). |
| R03 | Gemini | 15 | 7 | 3 | 4 | 3 | 3 DOIs absent from the registry, 4 pointing at a different paper; 3 claim-support overreaches. First export truncated. Correction round added 3 new identity failures. |
| R04 | ChatGPT | 39 | 0 | 0 | 0 | 4 | All 39 DOIs resolve; 4 self-flagged partial identities; 4 partial claim support (direction confirmed, exact value not re-extractable). |
| R04 | Gemini | 29 | 7 | 3 | 3 | 13 | 3 nonexistent DOIs, 3 wrong paper, 1 altered citation on a real DOI; 9 unsourced magnitudes. Correction round: 5 accepted, 2 partial, 2 new altered citations. |
| R05 | ChatGPT | 8 | 0 | 0 | 0 | — | 8 of 8 load-bearing citations confirmed. |
| R05 | Gemini | — | 3 | 1 | — | — | 3 citations refuted or unlocatable (1 not found, 1 altered author list, 1 misattributed review) plus 5 unsourced aggregate figures; the replacement offered in follow-up also failed. |
| R06 | ChatGPT | 21 | 1 | 0 | 1 | 4 | 1 author-year misattribution; every DOI and PMID resolved; claim support 17 yes, 3 partial, 1 no. |
| R06 | Gemini | 28 | 6 | 1 | 5 | 18 | 5 wrong paper, 1 nonexistent; claim support 7 yes, 11 partial, 7 no, 9 unverifiable. Correction round (26 items): 11 accepted, 11 partial, 4 rejected. |
| R07 | ChatGPT | 31 | 0 | 0 | 0 | 1 | All 31 identities verified; claim support 24 yes, 1 partial. |
| R07 | Gemini | 23 | 2 | 0 | 2 | 12 | 3 altered citations, 2 wrong paper, 2 unresolved; claim support 0 yes, 5 partial, 7 no, 4 uncheckable. Correction round: 20 accepted, 3 partial, 1 rejected. |
| R08 | ChatGPT | 15 | 0 | 0 | 0 | 2 | 14 verified, 1 partial (two co-authors omitted); claim support 13 yes, 2 partial. |
| R08 | Gemini | 31 | 9 | 2 | — | 18 | 18 verified, 4 partial, 9 failed (2 existence-level); claim support 11 yes, 9 partial, 9 no. The audit later removed one "no": it was a verifier error. |
| MED01 · P1 | ChatGPT | 39 | 1 | 0 | 1 | — | 1 wrong-paper PMID (not load-bearing); correction round 3/3 resolved, 0 new failures. |
| MED01 · P1 | Gemini | 29 | 3 | 1 | 1 | — | 1 DOI returning 404 for a real paper, 1 wrong paper, 1 wrong pages; PubMed count not reproducible. Correction round: 5 new wrong-paper PMIDs and 1 wrong-paper DOI. |
| MED01 · P2 | ChatGPT | 54 | 0 | 0 | 0 | — | 54 unique PMIDs, none pointing at the wrong paper; 1 of 9 official-record items mischaracterised. Correction round 3/3, 0 new failures. |
| MED01 · P2 | Gemini | 62 | 27 | 1 | 26 | — | 27 of 62 PMIDs wrong or nonexistent; also 7 nonexistent and 3 wrong DOIs, 2 nonexistent drug IDs. Four counts reported as live were later admitted to be estimates. Correction round: 38 of 55 confirmed, 4 new failures. |
Error catalogue
Every documented error the process caught, by any system in any role, with where it was recorded.ChatGPT 10 · Claude 20 · Gemini 84
114 entries
| Round | System · role | Kind | What happened | Caught by | Outcome |
|---|---|---|---|---|---|
| R01 2026-08-29 | Gemini researcher | claim-support overreach | Curl et al. 2015 was cited for an 80–95% reduction in farmworkers' occupational pesticide exposure; the paper measures dietary organophosphate exposure in general-population adults. record | Two-check verification, claim-support step (Claude) | Excluded; the paper is used only for dietary exposure (R01-C031) |
| R01 2026-08-29 | Gemini researcher | claim-support overreach | Lori et al. 2017 was cited for 40–60% higher mycorrhizal (AMF) colonization; the paper reports microbial biomass and enzyme activity, not AMF colonization. record | Two-check verification, claim-support step (Claude) | Attribution removed; AMF direction later anchored by another source |
| R01 2026-08-29 | Gemini researcher | magnitude or scope drift | Several figures were shifted from their sources: Mäder 2002's 97% pesticide reduction given as 100%; Seufert 2012's 66 studies/316 comparisons given as 316 studies; the 21-year DOK paper described as 40+ years; a DOI variant with a typo. record | Editor precision check (Claude) | Corrected at ledger |
| R01 2026-08-29 | Gemini researcher | other | Five to six precise figures had no source (e.g. ecotoxicity 90–98% reduction, infiltration 2–4× faster, certification costs $1,000–5,000+). record | Editor precision check (Claude) | None entered the ledger without an independent anchor |
| R01 2026-08-29 | ChatGPT researcher | metadata error | The title of Magnano et al. 2024 was given as a paraphrase; the DOI was correct. record | Live verification (Claude) | Cited by its real title with an EU-scope qualifier |
| R01 2026-08-29 | Gemini researcher | metadata error | All four replacement citations returned in the correction round named real papers with the wrong journal and/or DOI. | Verification of the correction round (Claude) | Corrected entries registered (S-047, S-063–S-065); logged as a permanent caution |
| R02 2026-08-31 | Gemini researcher | metadata error | Chouvy & Afsahi 2014 was cited as 'Hashish Mauritanica', IJDP 25(1):6–10; the real tuple is 'Hashish revival in Morocco', IJDP 25(3):416–423. The claim it supported held. record | Citation-identity check (Claude) | Corrected tuple used; logged as a caution |
| R02 2026-08-31 | Gemini researcher | metadata error | Gabriel et al. 2018 (rodenticides in owls) was cited as PLoS ONE e0198033; the real venue is Avian Conservation and Ecology 13(1), art. 2. record | Citation-identity check (Claude) | Logged do-not-cite as cited; real record used |
| R02 2026-08-31 | Gemini researcher | nonexistent citation | 'Bodwitch et al. 2021, Land Use Policy 105497' could not be resolved to any record; the research program it described is real. record | Citation-identity check (Claude) | Logged do-not-cite; verified papers from the same program cited instead |
| R02 2026-08-31 | Gemini researcher | fabricated correction | In the correction round, 4 of 5 returned tuples failed identity checks: one wrong article number, one wrong DOI, and two titles that matched no record (one DOI resolved to a different article). record | Verification of the correction round (Claude) | Only verified tuples (Polson 2023, Dillis 2021, Bodwitch 2019, Gabriel 2012/2015/2018) entered the ledger |
| R02 2026-08-31 | Gemini researcher | other | Paraguay production figures (70–80% of regional supply; 6,000–8,000 ha) were presented as measurements; the traceable origin is enforcement-based extrapolation. record | Provenance trace (Claude) | Ledgered as unresolved, low-confidence estimate (R02-C014) |
| R03 2026-09-01 | Gemini researcher | nonexistent citation | Three DOIs were absent from the doi.org handle registry (a BMC-style DOI for the owl paper, a Taylor & Francis DOI for Caulkins & Reuter 2010, and a 'Craven et al. 2019' citation for a combustion-chemistry claim). record | Two-check verification (Claude) | Logged do-not-cite; real anchors identified |
| R03 2026-09-01 | Gemini researcher | wrong-object identifier | Four DOIs resolved to other works, including two near-transpositions (California Agriculture a0013 for a0014; Land Use Policy 106351 for 106531) and a citation ('Queirolo et al. 2019') that does not exist as cited. record | Two-check verification (Claude) | Corrected tuples recorded; failed DOIs never re-cited |
| R03 2026-09-01 | Gemini researcher | claim-support overreach | Schwabe 2023 was cited as demonstrating systematic lab shopping (it demonstrates potency-label inflation); two Gabriel papers were merged into one '79–85%' range; Caulkins & Reuter 2010 was stretched to a cannabis-eradication claim. record | Two-check verification, claim-support step (Claude) | Reworded; the fisher figures split by paper (79% and 85%) |
| R03 2026-09-01 | Gemini researcher | truncated or incomplete output | The exported report was truncated: 9 of 15 sources and 19 of 25 candidate claims were missing. record | Raw-input intake (Claude) | Re-export requested; restored sections received 2026-09-02 |
| R03 2026-09-01 | Gemini researcher | other | Seven magnitudes were given without sources; on checking, 4 were reconstructable, 2 partial and 1 unverifiable as stated. record | Magnitude disposition (Claude) | Adopted only with proper sources; the unverifiable one withdrawn |
| R03 2026-09-01 | ChatGPT researcher | magnitude or scope drift | Diversion prosecutions were placed in the Eastern District of Oklahoma (they are Western District), and 'most Canadian home grows within the 4-plant limit' overstated the data (42.3% exceeded it in 2018, 24.5% in 2020). record | Two-check verification (Claude) | Fixed at ledger; exceedance numbers carried |
| R03 2026-09-02 | Gemini researcher | fabricated correction | Correction-round citations combined three papers into one tuple, attached a new title to a verified DOI ('Hashish clubs…'), and replaced the correct author list of Gabriel 2018 with one from a different study; the report body still carried the failed DOIs. record | Verification of the correction round (Claude) | All logged do-not-cite; ledger cites only verified tuples |
| R03 2026-09-02 | Gemini researcher | magnitude or scope drift | Correction-round figures were overstated or misattributed: court-case wholesale prices not on record, legal-state price bands presented as prohibition-state prices, a 70–80% legal-capture figure (records show about 67–70%), and 'no satellite visibility' of illicit production. | Verification of the correction round (Claude) | Prices withdrawn; other claims deflated or reworded |
| R04 2026-09-06 | Gemini researcher | nonexistent citation | 'Zipper et al. 2019, ERL 14(9):094002' had a DOI returning 404, and no paper with that title was found by bibliographic search. record | Two-check verification (Claude) | Withdrawn in the correction round; logged do-not-cite |
| R04 2026-09-06 | Gemini researcher | wrong-object identifier | Three DOIs resolved to unrelated papers: Wang 2019 (leaf enclosure) to a PAH study, 'Rehman 2021' to a sugarcane-bagasse paper, Cuypers 2017 to an explosives-detection paper. record | Two-check verification (Claude) | Editor-resolved tuples recorded; failed DOIs logged do-not-cite |
| R04 2026-09-06 | Gemini researcher | metadata error | A real DOI (S-110) was given an invented title, volume and pages; two DOIs did not resolve for real papers (one with a mangled prefix); Dillis 2019 carried the 2020 paper's author list. record | Two-check verification (Claude) | Cited under real titles only |
| R04 2026-09-06 | Gemini researcher | magnitude or scope drift | Outdoor containerized cannabis was given as about 22.8 kg CO2e/kg, attributed to Desaulniers Brousseau 2024a; the paper reports 61.8 and 110.7 kg CO2e/kg, and arithmetic built on the 22.8 figure was off by a factor of about 3–5. record | Full-text claim-support check (Claude) | Ledger carries the paper's figures; Gemini accepted the correction |
| R04 2026-09-06 | Gemini researcher | claim-support overreach | California's waste rule (4 CCR §17223) was read as prohibiting on-farm composting and forcing landfilling; the text lists on-premises composting and tilling among permitted methods. record | Regulation-text check (Claude) | Claim withdrawn; ChatGPT's reading adopted |
| R04 2026-09-06 | Gemini researcher | magnitude or scope drift | Several figures differed from their sources: ozone 0.1–0.3 ppb (paper up to 0.34/0.67 ppb), a 21% yield deficit (17%), 70–80% condensate recovery (70%), $3B/yr (US$6B), 428 cultivators (901), 68 watersheds (60), and 'zero synthetic pesticides permitted'. record | Two-check verification (Claude) | Corrected values used; misstatements logged as cautions |
| R04 2026-09-06 | Gemini researcher | other | Nine magnitudes had no traceable source, including a statewide licensed-canopy total and land share that no agency publishes. record | Provenance search (Claude) | Provenance-or-withdraw requested; none ledgered without a source |
| R04 2026-09-06 | Gemini researcher | claim-support overreach | Myth verdicts were labelled 'Falsified' on four claims where the evidence supported only 'unsupported' or 'not established'. record | Calibration check against the brief (Claude) | Verdicts recalibrated at ledger |
| R04 2026-09-06 | Claude editor | editor error | The critique's corrected tuple for Cuypers 2017 listed 4 of the paper's 6 authors; Gemini copied it into its correction round. | Verifier check of the correction round (Claude, verify-D) | Full Crossref author list applied; rule adopted that editor-issued tuples come from the full registry record |
| R04 2026-09-06 | Gemini researcher | fabricated correction | The correction round introduced two new altered tuples on real DOIs: a changed subtitle and author initials for S-110, and an altered title continuation and author list for S-099. record | Verification of the correction round (Claude) | Logged do-not-cite; real titles used |
| R04 2026-09-06 | Gemini researcher | magnitude or scope drift | A terpene emission capacity of 0.5–2.5 µgC/g/h was kept through the correction round; the cited paper reports 4.9–8.7. | Verification of the correction round (Claude) | Paper's values recorded |
| R04 2026-09-06 | Gemini researcher | truncated or incomplete output | The 'corrected' document export still contained the failed DOIs, the 22.8 kg figure, the altered title and other withdrawn items. record | Verification of the correction round (Claude) | The pasted correction response kept as the record; the export preserved but not cited |
| R05 2026-08-25 | Gemini researcher | metadata error | A 2023 Molecules paper (Zandkarimi et al.) was cited with a different author list ('Zager, Lynn, Vergara et al.'); the journal, volume and page matched the real paper. record | Dual-run cross-examination (the two researchers gave different authors for the same paper), then live verification by Claude | Corrected to Zandkarimi et al.; altered author list logged in the do-not-cite log; Gemini retracted it |
| R05 2026-08-25 | Gemini researcher | nonexistent citation | 'Mettu et al. 2023, J. Cleaner Production 385:135688' could not be located by targeted search. record | Live citation verification (Claude) | Treated as nonexistent; Gemini fully retracted; claims resting on it lost that support |
| R05 2026-08-25 | Gemini researcher | wrong-object identifier | Westmoreland & Bugbee 2022 (Front. Plant Sci. 13:984034) was cited as a cannabis lighting review; that paper is about phosphorus nutrition. record | Live citation verification (Claude) | Logged do-not-cite; lighting figures left unanchored until a correct source was found |
| R05 2026-08-25 | Gemini researcher | claim-support overreach | Four production-share percentages, greenhouse-subtype carbon ranges, per-plant water figures, lighting-efficiency benchmarks and a price-premium range were given with generic or no retrievable sources; Gemini later described them as synthetic extrapolations. record | Cross-examination against the ChatGPT pass and live verification (Claude) | Not ledgered as stated; production share entered the ledger as a data gap |
| R05 2026-08-25 | Gemini researcher | magnitude or scope drift | Outdoor water use of 5.5–12.0 L/plant/day was attributed to Bauer 2015, which used a 22.7 L/plant/day working assumption drawn from a county ordinance. record | Cross-examination (ChatGPT's reading of Bauer) and verification (Claude) | Water claims ledgered only from the papers' own figures |
| R05 2026-08-25 | Gemini researcher | claim-support overreach | Myth checks were answered with confident verdicts in the opposite direction ('light-dep is the sweet spot: True'; 'outdoor passes testing at high rates') where the evidence supported only context-dependent or unknown. record | Cross-examination (Claude) | ChatGPT's calibrated wording adopted; PROTOCOL v1.2 added the rule that 'unsupported' and 'the opposite is true' are different findings |
| R05 2026-08-28 | Gemini researcher | fabricated correction | The replacement lighting citation supplied in the follow-up round ('Kusuma, Westmoreland & Bugbee 2020, PLOS ONE e0230755') also failed verification; the real paper is Kusuma, Pattison & Bugbee 2020 in Horticulture Research. record | Verification of the correction round (Claude) | Logged do-not-cite; led to the PROTOCOL v1.2 rule that corrections receive full verification |
| R05 2026-09-06 | Claude editor | magnitude or scope drift | Signed claim R05-C013 carried 'up to ~23% of seven-day low flow' from Bauer 2015 as the overall bound; 23% was the least-impacted watershed, and the others reached 34–165%, 36–173% and over 100%. record | Claude during R04 verification; confirmed by ChatGPT's independent second check | Ledger, synthesis and source entry patched 2026-09-06 with a rule-13 record; first completed rule-10 correction cycle |
| R05 2026-09-06 | Claude editor | metadata error | Source S-006 (Mills 2012, Energy Policy) carried the title of a different object, the 2011 laboratory working paper 'Up in smoke'. | Claude during R04 verification; flagged independently by ChatGPT | Title corrected in the source library with a rule-13 addendum |
| R05 2026-09-06 | Claude editor | metadata error | Two inherited source entries were wrong: S-008 carried article number e0120063 instead of e0120016, and S-009 merged Dillis 2019 (California Agriculture) with Dillis 2020 (J. Environ. Manage.). | Claude during R04 verification | Both corrected with rule-13 logging |
| R06 2026-09-07 | ChatGPT researcher | wrong-object identifier | A finding on single-element deficiencies was cited by author-year as 'Cockson et al.' with no identifier; the finding belongs to Llewellyn et al. 2023, and Cockson 2019 has no flower-yield data. record | Two-check verification (Claude) | Re-anchored to Llewellyn 2023 and limited to the two micronutrients tested; recorded as ChatGPT's first identity failure in six cycles |
| R06 2026-09-07 | ChatGPT researcher | claim-support overreach | Three wording overstatements: a fertigation effect on yield (the effect was on total THC yield only), an early Mg impairment threshold (vegetative effect was small), and a statement that Fiering 2026 does not trace metals to fertilizer (the authors propose it as a possible source). record | Two-check verification (Claude) | Reworded at ledger |
| R06 2026-09-07 | Gemini researcher | nonexistent citation | 'Zheng, Frizzone & Bugbee 2021, HortScience 56(8):920–928' does not exist; it was used as a co-anchor of a modeled claim. record | Two-check verification (Claude) | Withdrawn; the dependent claim (C013) retired |
| R06 2026-09-07 | Gemini researcher | wrong-object identifier | Five DOIs resolved to other papers (Caplan 2017b to a grape cell-culture paper, Bernstein 2019 to a biocomposite paper, Jiao to a cassia-oil paper, Jones 2005 to another soil paper, Raven & Loeppert to a book review). record | Two-check verification (Claude) | Corrected tuples supplied and re-verified in the correction round |
| R06 2026-09-07 | Gemini researcher | metadata error | Six partial identities: a DOI built from a page number (Alden & Faust 2025), a wrong venue and DOI (Punja 2021), a wrong DOI digit, an altered title, an added co-author not on the paper, and one unresolvable reference. record | Two-check verification (Claude) | Corrected in the correction round |
| R06 2026-09-07 | Gemini researcher | magnitude or scope drift | Study details were misreported: total N of 160 mg/L (paper 200), '≥90% nitrate' (80/20), a flushing panel 'n=36' (36 was a percentage), and a modeled +26.5% footprint difference whose arithmetic gives +20.9% from the yield ratio alone. record | Two-check verification and editor cross-reading (Claude) | Figures corrected; the modeled claim retired |
| R06 2026-09-07 | Gemini researcher | claim-support overreach | Figures and findings were attributed to sources that do not contain them: amino-acid uptake '<1–5% of N flux', rock-phosphate Cd ranges and translocation factors, outdoor living-soil biomass (Caplan 2017b is indoor coir), terpene conclusions (Bernstein 2019 has no organic arm or terpene data), a 7.6 L coir/peat setup (the study was deep-water culture). record | Two-check verification, claim-support step (Claude) | Numbers withdrawn; claims restricted to what sources support |
| R06 2026-09-07 | Gemini researcher | claim-support overreach | 'REFUTED' was applied to myths (e.g. white ash) where the evidence supported only 'unsupported'; one rested on tobacco and biomass combustion science. record | Calibration check against the brief (Claude) | Downgraded to UNSUPPORTED where positive opposite evidence was absent |
| R06 2026-09-07 | Gemini researcher | fabricated correction | The correction round introduced a new incorrect title for McKernan 2016, re-attributed the withdrawn terpene sentence to Saloner & Bernstein 2021 (which ran no organic arm), changed a trial's dates, and left withdrawn numbers in the report prose. record | Verification of the correction round (Claude, verify-D: accept 11, partial 11, reject 4) | Corrected export kept as a record, not used as a source |
| R07 2026-09-09 | Gemini researcher | truncated or incomplete output | The first document export of the pass was a 3 KB stub; the full report arrived later as a document file. | Raw-input intake (Claude) | Full ~130 KB report used |
| R07 2026-09-10 | Gemini researcher | nonexistent citation | 'Caplan, Stemeroff, Dixon & Zheng 2019, HortTechnology 29(4):469–477' does not exist; its DOI resolves to a tomato soil-disinfestation paper. record | Two-check verification, Crossref and OpenAlex (Claude) | Withdrawn in the correction round; real Caplan 2019 Acta Horticulturae paper used |
| R07 2026-09-10 | Gemini researcher | wrong-object identifier | The Knight et al. 2010 DOI resolved to a limb-asymmetry paper; the attached yield figure was not in the real paper's abstract. record | Two-check verification (Claude) | Re-tupled in the correction round; figure withdrawn |
| R07 2026-09-10 | Gemini researcher | metadata error | Three real papers carried altered tuples (invented co-authors for Velechovský 2024, invented volume and pages for Punja 2021, Schober 2023 credited to 'Ahmed'), and two corpus source IDs were attached to the wrong papers. record | Two-check verification (Claude) | Corrected in the correction round |
| R07 2026-09-10 | Gemini researcher | claim-support overreach | The Yep, Gale & Zheng 2020 experiment was described with design details not in the paper (60 L DWC tubs, different cultivars, a water temperature); the paper used 11 L peat pots plus a 3 L pot/DWC aquaponic arm. record | Cross-examination against the ChatGPT pass and full-text check (Claude) | ChatGPT's reading adopted; description logged do-not-cite |
| R07 2026-09-10 | Gemini researcher | other | Tables carried magnitudes with no source (kg of rockwool per plant, g N per day, 30–50% microbial-test failure, cycles per year, CEC 40–100); none of the 16 candidate claims was fully supported by its own citation. record | Two-check verification (Claude) | All withdrawn in the correction round |
| R07 2026-09-10 | Gemini researcher | magnitude or scope drift | Velechovský 2024 figures were misread: a +182% recirculation effect given against the wrong comparator, nutrient augmentation as +93/+43/+940% (paper +83/+39/+860%), and an incorrect planting density. record | Full-text check (Claude) | Verified full-text figures used |
| R07 2026-09-10 | Gemini researcher | claim-support overreach | Two papers' findings were reported in the reverse direction (Tebdoie 2025 substrate results; Yep 2020a salinity response), and multi-genotype trials were described as absent although two were in the record. record | Two-check verification and cross-examination (Claude) | Corrected in the correction round |
| R07 2026-09-10 | Gemini researcher | search-wrapped or unresolvable link | The correction round stated that all search-engine-wrapped DOI links had been replaced with bare DOIs; six were still wrapped. record | Verification of the correction round (Claude, verify-D item 10) | Item rejected; DOIs resolved directly |
| R07 2026-09-10 | Gemini researcher | fabricated correction | The correction round introduced new details not in the sources: 'β-pinene' (the source says pinene), an '11-week flowering cycle', and a changed substrate list for Schober 2023. | Verification of the correction round (Claude) | Strings logged do-not-cite |
| R07 2026-09-10 | ChatGPT researcher | claim-support overreach | Allred et al. 2025 was summarized with a uniform effect; the height effect was uniform but the yield effect was cultivar-dependent. record | Two-check verification (Claude) | Corrected at ledger |
| R07 2026-09-11 | Claude editor | editor error | Synthesis v0.1 rendered the Yep 2020 result (+42–116% for the synthetic-peat arm) as '42–116% less' and misstated other details (e.g. ACGIH described as federal); the same slips appeared in notes on signed ledger claims C001, C004, C026 and C028. | ChatGPT editorial pass on the synthesis | Synthesis v0.2; ledger notes patched under rule 13 (wording only) |
| R08 2026-08-31 | Gemini researcher | nonexistent citation | Two citations matched no record: 'Vergara et al. 2016, R. Soc. Open Sci. 160397' (DOI 404) and 'Fonteno 1993, HortScience 28(2):81'. record | Two-check verification (Claude) | Both logged do-not-cite; nearest real objects identified |
| R08 2026-08-31 | Gemini researcher | wrong-object identifier | Seven DOIs pointed to other papers or did not resolve (e.g. Lyu 2019 to a rice cadmium paper, Cranshaw 2020 to a corn tar-spot paper, McKernan 2021 to a corrosion paper, Bugbee 2002 to an unauthored item). record | Two-check verification (Claude) | Editor resolved the intended papers; corrected tuples re-verified |
| R08 2026-08-31 | Gemini researcher | metadata error | A real DOI (Seemakram 2022) was given an invented title, an industry flushing trial was credited to a nonexistent author 'Stemler', and a Viruses paper was credited to 'Bektaş et al.' instead of Atallah, Yassin & Verchot. record | Two-check verification (Claude) | Real titles and authors recorded; altered strings logged do-not-cite |
| R08 2026-08-31 | Gemini researcher | claim-support overreach | Shannon-diversity figures for living soil vs coir vs rockwool were attributed to Winston 2014 and Comeau 2020; neither paper compares substrates or contains those figures. record | Two-check verification, claim-support step (Claude) | Claim chain treated as unsourced; logged as a permanent caution |
| R08 2026-08-31 | Gemini researcher | claim-support overreach | Several real papers were cited for findings they do not contain: Treseder 2008 (AMF phosphorus thresholds), Bernstein 2019 (twice), Vanhove 2011 (a 77-grow survey), Burgel 2020 (mineral vs organic percentages); Bugbee 2002's abstract argues against the threshold it was cited for. record | Two-check verification, claim-support step (Claude) | Attributions removed; direction re-anchored to verified studies where possible |
| R08 2026-08-31 | Gemini researcher | magnitude or scope drift | Real sources carried altered magnitudes: viroid losses given as 30–50% (paper 12–42%), mineralization rates and timeframes off, root biomass reported as increased where the paper found it unaffected. | Two-check verification (Claude) | Cited at the papers' own numbers |
| R08 2026-08-31 | Gemini researcher | other | Many precise figures carried no citation (canopy shares, a +10–25% price premium, a full cost and labor table, 15–35% microbial-test failure rates, reset triggers). record | Cross-examination (Claude; ChatGPT had reported these domains as searched and empty) | Inadmissible as quantitative claims; correction round requested provenance or withdrawal |
| R08 2026-08-31 | Gemini researcher | truncated or incomplete output | The pass omitted three brief-required sections: candidate claims, search-stopping rationale and an explicit data-gap list. record | Structural compliance check (Claude) | Requested in the correction round |
| R08 2026-08-31 | ChatGPT researcher | claim-support overreach | Two details could not be confirmed from accessible text (a 55.5–62.3 g/plant yield range; '516 cases' in a review), and 'higher rates depressed yield' held for only one substrate. One reference omitted two co-authors. record | Two-check verification (Claude) | Softened at ledger |
| R08 2026-08-31 | Claude verifier | verifier-induced error | The verifier 'corrected' Gemini's correct figure (potting media up to ~75% of outdoor GWP) to 25–38%, a metal-depletion figure, then registered the same paper twice in the source library, creating an apparent second LCA. record | ChatGPT post-signoff cross-model audit (2026-09-02); confirmed by Claude's reciprocal check | C027, synthesis and source library corrected; duplicate S-132 retired; Gemini's error tally reduced by one; led to PROTOCOL v1.4 rules 10–13 |
| R08 2026-08-31 | Claude editor | editor error | The critique's correction of Gemini's bleach concentration ('5–10%, not 10–20%') was itself only partly right: the full text shows effective concentrations depend on the matrix. | Full-text check during correction-round verification (Claude) | Matrix-dependent wording recorded; logged as a caution |
| R08 2026-08-31 | Gemini researcher | fabricated correction | Correction-round citations included a DOI resolving to a fruit-chemistry review (for Massuela 2023), a DOI resolving to a buckwheat paper (for Carlile 2015) and an unregistered DOI (for Raviv 2002). record | Verification of the correction round (Claude) | All three logged do-not-cite; real tuples used |
| R08 2026-08-31 | Gemini researcher | claim-support overreach | In the correction round, a disinfectant claim was re-asserted that the full text refutes (2% Virkon S had no effect on viroid RNA), an organ-accumulation ordering contradicted its sources, and a 28-soil study was called a meta-analysis. record | Verification of the correction round, full-text reads (Claude) | Claims removed or reworded; logged as cautions |
| MED01 · P1 2026-09-23 | ChatGPT researcher | independence breach | The ChatGPT session had prior project context visible and, with the attachments not mounted, retrieved the frozen brief and protocol from the project's shared document store itself. record | Researcher's own disclosure, weighed by the verifier (Claude) | Convergence on standard landmarks discounted; P2 kickoff required fresh isolated sessions with local attachments |
| MED01 · P1 2026-09-23 | Gemini researcher | independence breach | The Gemini session had prior editorial-review context visible; Gemini stated it disregarded it. record | Researcher's own disclosure, weighed by the verifier (Claude) | Convergence discounted as above |
| MED01 · P1 2026-09-23 | ChatGPT researcher | wrong-object identifier | PMID 35817852, cited for a count of cannabinoids known by 1980, resolves to a 2022 Nature Metabolism review; one co-author name was also wrong. record | Verification run 1 (Claude) | Replaced in the correction round by Turner 1980; non-load-bearing |
| MED01 · P1 2026-09-23 | Gemini researcher | wrong-object identifier | Three identity failures and two metadata errors: Beal 1995 DOI returned 404 (paper real), the Tashkin 1973 DOI resolved to a different NEJM article with wrong pages, Gill 1970 had wrong pages, and the 1967 Mechoulam paper was conflated with the 1965 total synthesis. record | Verification run 1 (Claude) | Corrections requested; substance retained under resolved identifiers |
| MED01 · P1 2026-09-23 | Gemini researcher | other | A PubMed count (11,241 records, with decade figures) could not be reproduced by re-running the stated query, which returned 10,630. record | Verbatim query re-run (Claude) | Verifier count used; Gemini accepted |
| MED01 · P1 2026-09-23 | Gemini researcher | claim-support overreach | Magnitudes were attributed to sources that do not contain them: a 25–30% eye-pressure drop in 11 volunteers (the cited letter has no abstract or figures), codeine equivalence, '>75%' of research on harms, '>90%' and '24–48 h'. record | Abstract-level claim-support check (Claude) | Withdrawn or rejected in the correction round |
| MED01 · P1 2026-09-23 | Gemini researcher | magnitude or scope drift | Framing exceeded the studies: an oral-THC trial presented as a smoked-vs-oral comparison, a top-tier rating for dronabinol halting weight loss (weight difference not significant), present-day 'disproven' verdicts with no period source, and receptor evidence described as established in humans. record | Drift checks D1–D14 (Claude) | Reworded to what the studies did; Gemini conceded the route framing |
| MED01 · P1 2026-09-23 | Claude verifier | verifier-induced error | The verifier recorded that a per-patient breakdown (C019) was not in the abstract; it was. The abstract had been read truncated at about 900 characters. | ChatGPT §M10a cross-audit | Correction #2 to ledger, synthesis and both plain-English documents; rule against recording 'not reported' from a truncated read |
| MED01 · P1 2026-09-24 | Gemini researcher | other | A candidate claim cited 'EMBASE scans' that were not in the search log; Gemini later admitted EMBASE was not searched. record | Search-log audit (Claude), confirmed in the correction round | Claim narrowed |
| MED01 · P1 2026-09-24 | Gemini researcher | wrong-object identifier | The correction round introduced 5 new wrong-object PMIDs and 1 wrong-object DOI, each an adjacent record to the real paper. record | Rule-9 re-resolution of the correction round (Claude) | Correct records used; logged in the researcher-performance record |
| MED01 · P1 2026-09-25 | Claude verifier | verifier-induced error | A qualification in C001 stated that the MeSH heading Cannabinoids dates from 1971; the NLM record says 1977. | ChatGPT §M10a cross-audit | Correction #2: year changed to 1977 |
| MED01 · P1 2026-09-26 | Claude editor | editor error | The end-of-era statement cited C031/C032 for cognitive and psychomotor effects they do not support, and combined C017/C018 so that aerosolised-THC bronchodilation read as smoked-cannabis bronchodilation. | ChatGPT editorial passes before sign-off | Fixed in synthesis v0.2 / v1.0 |
| MED01 · P1 2026-09-26 | Claude editor | editor error | Both plain-English documents rendered 'independent risk factor' (C030) as 'the authors called cannabis a cause'. record | ChatGPT §M11 fidelity review | Plain-English v0.2: wording restored with the note that an observational study cannot by itself establish causation |
| MED01 · P1 2026-09-26 | Gemini auditor | non-verbatim quotation | In the cross-audit, six passages presented as verbatim synthesis text do not appear in the signed synthesis (e.g. invented timepoints and P value, an 'RR 2.4 overall', invented WHO conclusions, an in-vivo stroke model), and two historical commissions were substituted. record | Verifier diff of audit quotations against synthesis v1.0 (Claude) | Logged as a major finding on the auditor; standing rule that auditor quotations are copied and diffed |
| MED01 · P1 2026-09-26 | Gemini auditor | wrong-object identifier | The cross-audit re-used PMID 1097148 for Noyes 1975b, a wrong-object identifier already caught in the correction round, and confirmed the verifier's C019 error instead of catching it. record | Verifier check of the audit (Claude) | Recorded on the auditor; the C019 catch credited to ChatGPT |
| MED01 · P2 2026-10-06 | Gemini researcher | independence breach | The first Gemini Stage 1 run located the brief by searching the project's document mirror and saw the Probe Appendix header and questions Q1–Q5 before filing. record | Gemini's own disclosure | Danny ordered a rerun in a fresh session under a separate account; the first run was marked superseded and kept for the postmortem |
| MED01 · P2 2026-10-06 | Gemini researcher | wrong-object identifier | 27 of 62 cited PMIDs (44%) were wrong-object or nonexistent, each labelled 'Verified live (resolved title copied)'; all 27 intended papers were real and located by the verifier. record | Verification run 1 (Claude) | 25 fixed in the correction round; correct records registered; permanent caution logged |
| MED01 · P2 2026-10-06 | Gemini researcher | metadata error | Seven DOIs did not resolve and three resolved to other papers; two cited Sativex drug identification numbers do not exist in the Health Canada database; two bibliometric papers were credited to the wrong authors. record | Verification run 1 (Claude) | All 10 DOIs and the DIN fixed in the correction round |
| MED01 · P2 2026-10-06 | Gemini researcher | non-verbatim quotation | Five titles marked 'resolved title copied' differed from the resolved titles (reworded, truncated, or with an added subtitle), and every reference was marked 'Full text', including paywalled items. record | Verification run 1 (Claude) | Access declarations not used to lift evidence caps |
| MED01 · P2 2026-10-06 | Gemini researcher | other | Four canonical PubMed counts were reported as live index figures although Gemini stated E-utilities was blocked; none reproduced, and the correction round admitted they were estimated. record | Verifier live count run (Claude); confirmed in the correction round | Only verifier-reproduced counts used; logged as a fabricated-data finding (repeats P1) |
| MED01 · P2 2026-10-06 | Gemini researcher | claim-support overreach | Portenoy 2012 was reported as showing statistically significant pain reduction; the primary analysis was not significant (P=.59) and significance came only from secondary analyses. record | Abstract-level claim-support check (Claude) | Logged as a caution; not ledgered as stated |
| MED01 · P2 2026-10-06 | Gemini researcher | magnitude or scope drift | A finding from a 2021 Crohn's study ('without endoscopic mucosal healing') was attributed to the 2013 trial, and a 2018 Cochrane review was used as a period certainty rating. record | Hindsight rule check (Claude) | Not ledgered; logged as anachronism |
| MED01 · P2 2026-10-06 | Gemini researcher | claim-support overreach | Koppel 2014 was described as a different guideline with different ratings; a Whiting 2015 spasticity estimate was attached to the wrong scale; a 2006 Cesamet labeling supplement was presented as an FDA approval. record | Abstract and official-record checks (Claude) | Logged as cautions; correct characterisations ledgered |
| MED01 · P2 2026-10-06 | Gemini researcher | magnitude or scope drift | Numerical details differed from abstracts: CAMS confidence interval and funder, Ware 2010 confidence interval and dosing schedule, Abrams 2007 design ('crossover' for a parallel-group trial) and P value. record | Abstract-level claim-support check (Claude) | Abstract values ledgered |
| MED01 · P2 2026-10-06 | Gemini researcher | fabricated correction | The correction round introduced four new failures: an unrelated 2008 meta-analysis substituted for Rocha 2014, a wrong first name, wrong pages, and the Cesamet labeling supplement relabelled as an efficacy-supplement approval. record | Verification of the correction round (Claude) | Correct records used |
| MED01 · P2 2026-10-06 | Claude verifier | verifier-induced error | The publication-integrity check read only retraction flags, not PubMed erratum links, so the Whiting 2015 Ashworth figure (−0.36) was ledgered from the abstract although the journal had corrected it (−0.12, not significant). record | ChatGPT §M10a cross-audit (rated its most serious finding) | Corrections #13 and #15 (values verified from the corrected article via Danny's browser); rescan found errata on six other sources; new standing rules on erratum checks |
| MED01 · P2 2026-10-06 | Claude verifier | verifier-induced error | Two verifier rulings against Gemini were wrong or too broad: a remark that Tramèr 2001 counted comparisons, not trials (the full text says 30 randomised controlled trials), and a ruling that Kerbrat counts were unsupported (consistent with the regulatory record). record | ChatGPT §M10a cross-audit | Both rulings withdrawn (corrections #10, #11) |
| MED01 · P2 2026-10-09 | Gemini researcher | claim-support overreach | Two patents were mischaracterised: US 9,066,920 (claim 1 is partial seizures) and US 8,071,641 (a diabetes patent from a different assignee). | Official-record check (Claude) | Logged as cautions |
| MED01 · P2 2026-10-09 | ChatGPT researcher | claim-support overreach | US 9,474,726 was described as covering CBD for Dravet syndrome; claim 1 covers FIRES. | Official-record check (Claude) | ChatGPT accepted the correction; logged as a caution |
| MED01 · P2 2026-10-09 | Claude verifier | verifier-induced error | Trial NCT01898520 was listed as 'no publication identified' before the scheduled title search was run; it was published in 2020. The registry filter had also counted 12 non-therapeutic studies as patient trials. record | ChatGPT §M10a cross-audit | Correction #8: full registry re-screen (28 therapeutic trials, 17 matched, 11 unresolved) |
| MED01 · P2 2026-10-10 | Claude editor | editor error | Ledger wording errors: Wang 2008's 96.6% applied to 31 studies (it is events within 23 RCTs), Pletcher 2012 described by the FEV1/FVC ratio (the result concerned FEV1 and FVC separately), Wade 2004's primary outcome called a composite, and a patent described as 'for diabetes' rather than Type II diabetes. record | ChatGPT §M10a cross-audit; confirmed at source by the verifier | Corrections #5, #6, #7, #9 |
| MED01 · P2 2026-10-10 | Claude editor | editor error | The end-of-era statement placed cancer appetite/cachexia evidence under 'preliminary' although two large randomized trials existed. record | ChatGPT synthesis editorial pass | Correction #4: placement follows design, not result |
| MED01 · P2 2026-10-12 | Claude editor | editor error | Plain-English documents described Devinsky 2016 as '214 children' (ages 1–30), implied that dronabinol and ondansetron worked equally well where equivalence was not tested, and gave one citation bracket for four bullets. record | ChatGPT §M10a cross-audit | Correction #10 and traceability fix |
| MED01 · P2 2026-10-12 | Claude editor | editor error | ChatGPT's fidelity review of the P2 plain-English drafts returned 24 findings (15 major, 9 minor); an earlier review of the drafts had found 3 errors (merged Sativex indications, a registry denominator, immune wording). | ChatGPT §M11 fidelity review | 23 accepted and 1 partly accepted; plain-English v0.2, then v1.0 |
| MED01 · P2 2026-10-12 | ChatGPT auditor | truncated or incomplete output | The first attempt at the plain-English fidelity review performed no review. | Editor intake (Claude) | Review re-run in three batches |
| MED01 · P2 2026-10-13 | Claude editor | editor error | A scripted edit anchored on a heading shared by Part 1 and Part 2 duplicated half the ledger. | Verifier check before mirroring (Claude) | Rebuilt from the pre-correction commit; rule to re-count claims after every scripted edit |
| MED01 · P2 2026-10-13 | Gemini auditor | non-verbatim quotation | In the cross-audit, 22 of 26 sampled source quotations presented as copied were not in the cited abstracts, and 11 of 11 quoted MED01 passages do not exist in any MED01 document. record | Verifier diff of audit quotations (Claude) | Audit rejected under §M10a; replaced by Danny's spot check; rule that a Gemini audit needs an automated quotation diff |
| MED01 · P2 2026-10-13 | Gemini auditor | wrong-object identifier | The cross-audit gave four wrong PMIDs for sources already in the library, attributed a CYP2C19 mechanism to Gaston 2017 (not in its abstract), and reported no major errors in documents where confirmed defects existed. record | Verifier check of the audit (Claude) | Two independently correct verdicts recorded as agreement only |
| MED01 · P2 2026-10-14 | Claude editor | claim-support overreach | C052 said the authors reported nabilone effective for sleep quality; the abstract shows improvement on restfulness but not wakefulness. | Danny's source-level spot check | Correction #14 |