MED01-P2 — Verification of the two §M10a cross-audits (2026-10-13)
The auditors’ own quotations, identifiers and source claims were checked by the verifier, as §M10a requires. Tools: NCBI E-utilities (abstracts and CommentsCorrections), PMC article pages, Google Patents, AEMPS CIMA, the UK electronic medicines compendium (emc), and the MED01 git documents. The JAMA Network site blocked both the sandbox and the built-in browser with a bot check; it was not bypassed. Anything that rests only on JAMA pages is marked convergence, verifier read pending.
A. Gemini audit — not accepted as an independent audit
- Quotations from sources: 22 of 26 sampled are not in the cited abstract. They were presented as “copied directly”. Matching was done after normalising case and punctuation. Most mix real figures with invented wording. Examples:
- Wachtel 2002, “largely indistinguishable from pure oral THC”. The abstract says “similar subjective effects, with only minor differences”.
- Ilan 2005, Bhattacharyya 2010 and Gloss 2014: the quoted sentences are absent.
- Some quoted figures appear nowhere in the cited abstract:
- Treister-Goltzman: “13,058 … 1319”
- Ware 2010b: “−3.25 … p=0.74”
- Blake: “p=0.044, 0.018”
- Budney: “24–48 hours”
- Gaston: “81 patients”
- Andreae: “OR 3.22”
- Leweke: “n=42”
- Devinsky 2017: “93%”, plus the 36% / 19% / 28% / 16% / 12% breakdown
- Reiman: “97%”
- Some numbers are real but sit inside invented sentences. Mittleman’s 4.8 is real; the abstract says “elevated 4.8 times”, not “4.8-fold”. Corey-Bloom’s 5.28 and Gunn’s 2.02 are also real.
- Quotations from MED01 documents: 11 of 11 checked do not exist. These are the “Synthesis (§n)” and “PE Summary (§…)” passages used to rate downstream fidelity “Faithful”. Sections such as “The Regulatory Paradox”, “The Three Eras”, “Common Myths” and “The 2017 Consensus” are not in any MED01 document. The synthesis does not name Treister-Goltzman or quote Tashkin as “demonstrated … causes”. Every downstream-fidelity verdict in this audit is therefore unsupported.
- Identifiers: four wrong. All are PMIDs the auditor gave for sources already in the library:
- Walitt: 27431354 (correct 27428009)
- Tramèr: 11447006 (correct 11440936)
- Meiri: 17565492 (correct 17355735)
- Naftali 2013: 23648649 (correct 23648372)
- Substance. The audit reported “no critical or major errors”. The ChatGPT audit found, and the verifier confirmed below, at least seven real defects in the same documents, several of them in claims Gemini traced (C074, C077, C078). Gemini also asserted that “Gaston 2017 … demonstrated that CBD dramatically inhibits CYP2C19”. That mechanism is not in Gaston’s abstract; the clobazam-metabolite rise is Geffrey 2015.
- This is the third consecutive Gemini filing with fabricated verification content:
- the P1 cross-audit had six invented “verbatim” quotations;
- the P2 passes had 27 wrong-object PMIDs and fabricated counts;
- this audit is the third.
Ruling: the Gemini audit does not satisfy §M10a (quotation by copy, opened identifiers), so it cannot serve as the independent audit of the ChatGPT-origin components. Two of its verdicts are independently true:
- C065: US 9,474,726 claim 1 is FIRES, confirmed by the verifier on 2026-10-09.
- PC8 ruling: the reasoning matches the ledger.
They count only as agreement, not as audit evidence. Decision for Danny (see dispositions): rerun Gemini with the documents pasted in, or escalate the ChatGPT-origin components to a source-level spot check by Danny.
B. ChatGPT audit — findings checked against sources
| ChatGPT finding | Verifier check (2026-10-13) | Result |
|---|---|---|
| C048 Wade 2004 primary = VAS of “most troublesome symptom”, not a composite | PubMed abstract: “The primary outcome measure was a Visual Analogue Scale (VAS) score for each patient’s most troublesome symptom.” | Confirmed |
| C074 Pletcher: FEV1 and FVC, not the FEV1/FVC ratio | Abstract: “at low levels of exposure, FEV(1) increased by 13 mL/joint-year … and FVC by 20 mL/joint-year”. The full text discusses the ratio only in prior studies | Confirmed |
| C065 US 8,071,641 claim 1 = Type II diabetes | Google Patents claim 1: “A method of treating Type II diabetes in a subject…” | Confirmed |
| C077 Wang 96.6% = events within the 23 RCTs | Abstract: “In the 23 randomized controlled trials … A total of 4779 adverse events … Most (4615 [96.6%]) were not serious.” | Confirmed |
| C067 NCT01898520 was published (Fairhurst 2020) | PubMed 32342496, Fairhurst C et al., Dev Med Child Neurol 2020: 72 randomized; “no significant difference in the spasticity 0 to 10 NRS”; “three cases of hallucinations”. EU-CTR 2018 results: not reproduced (the search page did not render) | Confirmed (publication); EU-CTR = convergence |
| C064 Kerbrat regulatory cohort (IGAS: one death + four others) | NEJM abstract: 6 on 50 mg; 4 consented to the report; syndrome in 3 of 4; one brain dead. IGAS PDF returned 404 at the tried URL | Convergence, verifier read pending |
| C077/C078 Whiting AE OR 3.03, withdrawal OR 2.94, serious-AE OR 1.41 (JAMA Table 3) | JAMA blocked | Convergence (two models): Gemini gave 3.03 and 2.94 in its P2 pass and ChatGPT reports them from Table 3. Verifier read pending |
| C078 Whiting correction: Ashworth WMD −0.12 (−0.24 to 0.01; 5 trials), NRS −0.76 (−1.38 to −0.14; 3 trials) | PubMed CommentsCorrections for 26103030 lists four errata: “Incorrect Figure Label” (2015-08-04), “Incorrect Values Reported” (2015-08-25, 10.1001/jama.2015.9010), “Incorrect Nonproprietary Drug Name and Approved Use” (2015-12-01), “Incorrect Effect Estimate” (2016-04-12, 10.1001/jama.2016.3470). Erratum text blocked (JAMA) | Erratum existence confirmed; corrected values = convergence, verifier read pending. The ledger’s −0.36 figure is unsafe as it stands |
| C055 Devinsky 2016 enrolled ages 1–30 | Abstract: “patients (aged 1-30 years) with severe, intractable, childhood-onset, treatment-resistant epilepsy” | Confirmed |
| C053 Meiri: similar response, not equivalence | Abstract: “TR was similar with dronabinol (54%), ondansetron (58%), and com[bination]…” | Confirmed (wording point) |
| C053 Tramèr full text says “30 randomised controlled trials” | PMC34325 full text: “We analysed data from 30 randomised controlled trials published between 1975 and 1997” | Confirmed. The verifier’s earlier “comparisons ≠ trials” remark against Gemini was wrong and is withdrawn |
| Plain-English summary cross-references for cancer weight loss | Summary closing list: one citation bracket after four bullets | Confirmed (traceability) |
| H8 Meier “8 IQ points” | PMC3479587: “adolescent-onset cannabis users who used cannabis persistently up to age 38 y (n = 23) lost an average of 8 IQ points”; the most persistent users overall (n = 41) lost 5–6 points | Confirmed — HELD resolved (subgroup only) |
| H4 UK first authorisation 16 June 2010 | emc Sativex SmPC §9: “Date of first authorisation 16 June 2010” | Confirmed — HELD resolved |
| H4 Spain 20 July 2010 | AEMPS ficha técnica §9: “Fecha de la primera autorización: 20/julio/2010” | Confirmed — HELD resolved |
| H4 Australia ARTG 26 November 2012 | Not reproduced | Convergence |
| H1 Novotna −0.84 (NCBI Bookshelf secondary table) | Bookshelf blocked (reCAPTCHA) | Convergence (secondary source) |
| H2 FDA orphan designation, Dravet, 2013-11-14 | Not reproduced (OOPD query failed again) | Convergence |
| H3 NCT01040910: protocol primary = clinical response (Cochrane risk-of-bias note) | Not reproduced | Convergence |
| H7 Dutch pharmacy supply from 2003-09-01 | Not reproduced | Convergence |
| H10 ACP and AMA position text | Not reproduced | Convergence |
C. Verifier failures exposed by the audit
-
Publication-integrity check was incomplete. The critique-P2 §3c check looked only at PubMed publication-type flags (retraction, expression of concern) and Crossref updated-by. It did not read PubMed’s ErratumIn links. A rescan on 2026-10-13 of all 127 P2 PMIDs found errata or updates on seven sources:
- Whiting 2015: four errata
- Devinsky 2016: Lancet Neurol “Corrections”, April 2016
- Bachhuber 2014: JAMA Intern Med erratum, November 2014
- Van Gaal 2005 and Pi-Sunyer 2006: errata
- Wilson & Nicoll 2001: Nature erratum
- Marshall 2014: Cochrane update in 2019, retrospective
The content of each erratum must be checked against the ledgered figures. The Whiting errata already change one.
-
The registry title-search step was never run. Critique §6b scheduled “one more title search each” before the no-publication examples were disclosed. The ledger and plain-English documents named NCT01898520 before that step was done. A one-line PubMed search finds its 2020 publication.
-
Abstract-level checking does not catch publisher corrections. This is ChatGPT’s point, and it is correct.
-
One verifier ruling against Gemini was itself wrong (Tramèr wording). Another was too broad (Kerbrat counts, pending the IGAS read).