There’s a newer version of this page: open the latest version

Verified errors in high-impact papers: write-ups

2026-10-23 22:23 UTC. 134 findings, each documented from the paper's own text and numbers (and, for trials, the registry's version history), with a runnable script and figures. Wording is factual and makes no claim about intent; none of these has yet been raised with the authors or journals.

How each one was checked. Numerical errors were found by a cheap full-paper screen (Claude Haiku 5.5 or GPT-6 Luna), confirmed by two cheap verifiers and then by Opus 5.5 and GPT-6.1 Sol independently. Registry discrepancies were flagged by GPT-6 Luna and Claude Haiku 5.5 comparing each trial report with its ClinicalTrials.gov record and confirmed by Opus. For this page each was checked again by agents with Python and web access: the publisher's version of record and supplements; Crossref, Europe PMC, PubPeer and Retraction Watch for corrections; for trials, the registry's version history (what was registered before enrollment and at publication), the paper's role and any disclosure; and a blind re-check by an agent that never saw the earlier verdicts. A separate agent then fact-checked each write-up, re-running the script, verifying every quote and date, and inspecting every figure; write-ups it rejected were revised up to twice.

Severity (from a separate consequence assessment): 5 = a headline conclusion or decision-relevant estimate is wrong (for trials: the conclusion rests on an outcome that was not the registered primary); 4 = a key result is wrong but the main conclusion survives; 3 = a secondary result; 2 = a misstatement a careful reader would catch; 1 = cosmetic.

Index

Numerical and internal-consistency errors (123)

SeverityJournalFinding
5J Hepatol 2020SIR and SMR confidence intervals are 34–42 times too narrow for the observed counts
5JAMA Network Open 2018Reported risk ratios are ratios of mean 1/2 codes, understating relative risk reductions for perpetration 9- to 28-fold
5Nat Commun 2025Reported nationwide energy, cost and CO2 savings are 1,000 times what the paper's inputs give
4Arthritis Rheumatol 2019Abstract reports adverse-event rates of 17% and 10%; Results and Table 3 give 62% and 55%
4BMC Med 2023PIK3CA progression-free survival P = 0.016 cannot be derived from its reported HR and CI
4BMC Med 2023Abstract reports all pocket-depth measures significant for CVD mortality; Table 2 shows none are
4BMC Med 2025All reported HPV prevalence confidence intervals are about 20 to 140 times narrower than the counts allow
4BMC Med 2025Abstract and conclusions call the 57.4% concordant trials discordant; discordance was 42.6%
4BMJ 2017Abstract labels the age-and-sex adjusted all-cancer hazard ratio as the multivariable adjusted estimate
4BMJ Glob Health 2024Primary-outcome 95% confidence interval is about 80 times too narrow for the reported failure counts
4BMJ Glob Health 2025Reported complete-abortion risk ratio of 2.55 should be about 1.02 by the paper's own conversion
4BMJ Glob Health 2016Abstract's patient-handwashing confidence interval is impossible and conflicts with the paper's own non-significant clustered estimate
4BMJ Glob Health 2020Ebola review calls case-fatality change non-significant while reporting OR 1.6, 95% CI excluding 1, p<0.001
4BMJ Glob Health 2022Abstract reports p=0.002 for an odds ratio whose own 95% CI (0.86 to 7.36) includes 1
4BMJ Glob Health 2022Abstract reports whole-cohort estimates as sexual-minority results, making two non-significant subgroup findings appear significant
4Cell 2010Abstract names G3244A where the body names G3242A as the second termination-abolishing pathogenic mutation
4Cell Rep 2026Abstract says metformin upregulates intestinal G6PC; the paper's own Results report a decrease
4Cell Rep Med 2024Oral cancer test's accuracy table counts 101 subjects in a cohort the paper gives as 92
4Clin Cancer Res 2025Abstract reports all-grade adverse event rates as grade ≥3 treatment-related rates in a phase I trial
4Commun Biol 2025Table 1 crystallographic statistics do not match the AcrIIA19 structure described in the text and deposited
4Commun Biol 2019Sahel-wide foliage increases of 6% and 20% do not match the relative increases implied by the paper's own totals
4Diabetes Care 2009Table 3 extrapolates VA costs with the 75% population share instead of the stated 5%
4Diabetologia 2025Reported p values for HbA1c and other secondary outcomes contradict their own confidence intervals
4Eur Respir J 2023Abstract reports p<0.001 for the matched cystic fibrosis versus rheumatoid arthritis comparison; Table 4 gives p=0.055
4Eur Respir J 2015Abstract transposes two of four unilateral and bilateral effusion mortality figures given correctly in Results
4Hum Brain Mapp 2023Dorsal attention network mediation reports a negative indirect effect with an entirely positive confidence interval
4Hum Brain Mapp 2009Paired t-test p-values for one-year cortical thickness change do not match the reported t statistics
4Hum Brain Mapp 2022Table 4 brain-volume T values contradict their p values and the table's own means and SDs
4Int J Epidemiol 2018Abstract reports low birthweight rising from 9.0% to 10.1%; Table 1 shows 8.2% to 8.3%
4J Neurosci 2018Figure 6D calls a TRPV4 knockout effect significant at p = 0.06706, above the 0.05 threshold
4JAMA 2010Reported renal-cancer P=0.004 is P=0.24 under the paper's own stated Fisher test
4JAMA Netw Open 2024Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1
4JAMA Netw Open 2025Week-24 HDRS-17 difference reported as P < .001; its t129 = 0.71 gives P = .48
4JAMA Netw Open 2025Abstract reports overall-cohort length-of-stay estimates as the results for hypotensive patients
4JAMA Netw Open 2026Risk-difference confidence intervals for the primary outcome are about 23 to 31 times too narrow
4JAMA Netw Open 2023Reported HbA1c intervention effect of 0.02% lies outside its own 95% confidence interval of 0.03%–0.14%
4JAMA Network Open 2023Diabetes-only hyperglycemia primary outcome has denominators larger than the trial's possible diabetic population
4JAMA Network Open 2024Subgroup effects called significantly different although their 95% CIs overlap, contrary to the paper's criterion
4JAMA Network Open 2025Printed stroke confidence interval excludes its own estimate; Table 2 shows the effect is non-significant
4JAMA Network Open 2022Registry share of Asian or Pacific Islander patients is 2.98%, not 5.9%, reversing reported underrepresentation
4JAMA Psychiatry 2015Right anterior insula odds ratio of 0.96 lies outside its own reported 95% CI (0.01–0.75)
4Lancet Child Adolesc Health 2018Abstract cites non-significant interaction odds ratios as evidence that correct fluid significantly reduced mortality
4Lancet Global Health 2025Several adjusted odds ratios in the abstract have confidence intervals narrower than the data allow
4Lancet HIV 2025Abstract swaps the DALYs-averted estimates of the two dynamic choice HIV prevention policies
4Lancet HIV 202195% upper bound for zero neural tube defects among 384 dolutegravir-exposed outcomes is tenfold too small
4Lancet Neurology 2026Reported p<0.001 for mouse leak frequencies cannot be reproduced; the paper's counts give p=0.11–0.36
4Lancet Reg Health Am 2026The firearm death-in-ED estimate is reported as 88.82, 83.82, 77.63 and 379.00
4Lancet Reg Health Am 2026Abstract's adjusted odds ratios for education and prenatal care differ from the paper's own Results
4Lancet Reg Health Am 2026Abstract effect sizes for crowding, ageing and primary care coverage disagree with the paper's Table 2
4Lancet Reg Health Am 2022Abstract reports 15.4% survival before 24 weeks; the paper's own counts give 24.1%
4Lancet Reg Health Eur 2025Abstract reports low-birth-weight estimates, including a nominally significant ASD effect, as preterm-birth results
4Lancet Reg Health Eur 2025Reported overall 90-day mortality of 63.7% conflicts with the paper's own counts of 408 deaths
4Lancet Reg Health Eur 2025Abstract reports univariate hazard ratios as adjusted estimates and attaches them to the wrong treatments
4Nat Commun 2026Abstract reports a 59.7% catalyst heat-duty reduction; the body reports 35.2% at the optimum
4Nat Commun 2026The abstract reports a positive temperature association with H7 spread; the paper's own coefficients are negative
4Nat Commun 2017Table 1 P values do not follow from its own mouse counts under the stated tests
4Nat Commun 2018Abstract reports reduced PKN1 and PKN4, but the Results report increased PKN2 and no PKN4
4Nat Commun 2019Abstract says host genetics explains almost half of meningitis susceptibility; the paper estimates 29%
4Nat Commun 2023Headline protection p-value cannot be reproduced from the paper's counts using its stated test
4Nat Med 2023Abstract's verified-vaccination percentages and effect do not match its own counts or the Results
4Nat Med 2012Abstract says VEGFR1 is downregulated in hyperlipidemic mice; the Results and Discussion report an increase
4Neurology 2026Abstract's baseline cognitive-component estimates differ from the Results and Table 2, changing one component's significance
4Neurology 2026Abstract reports 6-month BMI difference versus prednisone as 12-month result, and reuses it for height
4Neurology 2026Abstract reports 73.5% of trials used biomarkers; the Results and phase counts give 56.3%
4Nucleic Acids Res 2015Abstract pairs Ala510 with dA and Asn513 with dC; the paper's Results and Conclusion pair them the other way round
4PLOS Medicine 2018Abstract's age pattern for all STIs except HSV-2 is contradicted by the paper's syphilis estimates
4PLoS Comput Biol 2018Abstract reverses the direction of the paper's exposed-versus-occluded skin connectivity result
4PLoS Comput Biol 2020Abstract says LabMol-149 has EC50 below 500 nM; Table 3 gives 1450 and 509 nM
4PLoS Comput Biol 2020Abstract names March and October as best twice-yearly spraying months; body, table and data give September–November
4PLoS Genet 2023Abstract reports SF2/SRSF1 decrease in astrocytes; the paper's data show it only in PNG and SPG
4PLoS Genet 2013Table 2 odds ratios for haplogroup B4 and other zero-cell rows cannot come from the counts
4PLoS Med 2019Abstract gives the earlier cohort's 70% accuracy as this trial's result; recomputed value is 63%
4PLoS Med 2019The abstract's univariate diabetes odds ratio is inverted, and its p-value disagrees with Fig 5
4PLoS Med 2022Abstract labels the 39.65 million prevented CVD events as deaths; Results report 13.86 million deaths
4PLoS Pathog 2023Abstract reverses the direction of the Alpha versus B.1.177 viral-burden difference reported in Table 1
4PLoS Pathog 2012The abstract and the body disagree on nitric oxide after arg1 knockdown; which wording is right depends on the comparison
4Proc Natl Acad Sci U S A 2023Abstract says vitamin D3 rose in Scribble knockout mice; Results and Fig. 3E show a fall
4Proc Natl Acad Sci U S A 2024Abstract's tissue-culture counts (1/6, 6/6) conflict with the paper's Results and design (2/8, 8/8)
4Proc Natl Acad Sci U S A 2026Title says the 2025–2030 guidelines raise water use; the paper's own results show lower water use
4Proc Natl Acad Sci U S A 2018The abstract names the inhibited RNA strand as negative, but the paper's own data show positive
4Proc Natl Acad Sci U S A 2018Significance statement says tracer chromosome goes "exclusively" with X; Results report 28 of 42 cells
4Proc Natl Acad Sci U S A 2022Figure 3 legend calls a transduction P value of 0.096 statistically significant
4Proc Natl Acad Sci U S A 2022Chi-square P = 0.0379 for casein B-cell responders does not follow from 19/45 vs 10/35
4Psychol Med 2024Abstract reports PGS-ADHD odds ratios above 1 as decreased odds; Results report them below 1
4Psychol Med 2026Cohort 2 is reported as 361,258 participants in Table 2 but 32,220 in the flowchart
4Psychol Med 2021The logistic-regression "odds ratios", including those in the abstract, appear to be log-odds coefficients
4Sci Adv 2023Reported positive predictive value of 55% conflicts with the paper's own sensitivity, specificity and prevalence
4Sci Adv 2024Reported CO2 storage capacities per km³ are about 920 times what the paper's inputs give
4Sci Adv 2021The abstract gives the top tolerated monkey dose in mg/kg; the body gives μg/kg
3Ann Rheum Dis 2017The abstract's anxiety odds ratio (5.80) lies outside its own 95% CI (1.19 to 4.22)
3BMC Med 2024Printed 95% CI for a mortality predictor implies P ≈ 0.98, not the reported 0.006
3Brain 2023Abstract reverses the direction of both age effects on cortical thickness reported in Table 3
3Brain 2018A Discussion passage reverses the lesion–donation directions stated in the Abstract and Results
3Cancer Res 2022Abstract reports 18 PDX models for the co-clinical trial; the paper's data contain 15
3Clin Cancer Res 2012Fisher exact P reported as 0.0005 for 1/6 versus 7/18 remissions; the counts give 0.62
3Diabetologia 2024Hypertension–knowledge p = 0.007 cannot be reproduced from Table 1; its summary statistics give p ≈ 0.59
3Diabetologia 2025Abstract calls a non-significant subgroup difference at seroconversion significant; Results and Table 3 report none
3Epidemiology 2016The 2013 estimate of MSM living with HIV lies above its own 90% upper limit
3Gastroenterology 2012Abstract reports a 31% fall in endotoxin activity on the prudent diet; Results report 38%
3Gastroenterology 2020Abstract reverses the direction of the paper's own DDIT4 and gastric cancer survival finding
3J Clin Oncol 2024Abstract hazard ratio for SLiM-CRAB status lies outside its own confidence interval and contradicts the Results
3JAMA Netw Open 2025Abstract reports two obesity-group odds ratios as the overweight and obesity estimates for EPT use
3JAMA Network Open 2021Abstract reports a diastolic blood pressure difference that the trial's own analyses do not show
3Kidney Int 2021Hospital-stay estimate of 8.4 days lies outside its own reported 95% CI of 4.4–6.7
3Lancet Infect Dis 2014Abstract reports 58 (53%) of 113 resistant sites; Results' own counts give 57 (51%) of 112
3Lancet Reg Health Am 2025Abstract reports lower Ct values in vaccinated participants; the paper's data and text show higher
3Mol Psychiatry 2016Abstract reverses the direction of the cerebellar IDH3A protein difference reported in Table 3B
3Nat Commun 2020Re-positive rate's 95% confidence interval excludes its own point estimate in abstract and Results
3Nat Metab 2024Abstract says serum-treated human islets came from T2DM donors; the donor table lists none
3PLoS Comput Biol 2025Abstract reports WGS spanned 49 (98%) of 48 P-VNTRs; the body gives 92% and 45
3PLoS Med 2024Abstract gives Padj = 0.010 for a primary macrolide comparison; Results give 0.19 (Table 2: 0.187)
3PLoS Pathog 2025The abstract says RNF213 loss lowers IFN-β-induced ISG expression, but Fig 2A shows no difference
3PLoS Pathog 2018The abstract reports increased heat sensitivity, but the paper's own data show increased heat resistance
3PNAS 2021Abstract reports phantom-sensation scores as phantom range of motion in degrees
3Proc Natl Acad Sci U S A 2018Abstract reverses the direction of the white-light calcium responses shown in Results and Fig. 4B
3Sci Adv 2020Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts
3Sci Adv 2021Abstract says the mAb reduces neutrophil apoptosis; the body and Fig. 3 show an increase
3Sci Adv 2023Abstract and Introduction state the reverse of the paper's own Schwann cell depletion result
3eLife 2024Binomial p for 12 of 17 complexes is 0.072, not the reported 0.0245
2Brain 2025Abstract reports the deductive-reasoning test as F(2,387), an error df larger than the whole sample
2JAMA Netw Open 2026Abstract calls a MELD–referral odds ratio of 1.13 "lower referral odds"
2Nat Med 2025Abstract reports P < 0.0001 for 3/170 vs 3/172 serious adverse events; the table gives P > 0.999
2eLife 2023Abstract reports 389 animals; the Methods list 116, and 389 sums per-context subject counts

Undisclosed differences from the trial registration (11)

SeverityJournalFinding
5JAMA Netw Open 2024Registered primary outcome, door-to-balloon time, was replaced after completion; main text omits this
4Cell Rep Med 2026Paper reports dose-limiting toxicity as primary endpoint; the only registered primary outcome is response rate
4Gut 2018Registered 6-week re-intervention count reported instead as a 6-month total procedure count, without disclosure
4JAMA Intern Med 2014Trial report names an unregistered primary outcome and omits registered primary HIV risk behavior
4N Engl J Med 2014Registered primary outcome is overall survival; the final report uses melanoma-specific survival without comment
4PLoS Med 2014HTC uptake, the trial's first primary outcome, appears in no version of its registration
4PLoS Med 2021The paper calls store sales primary; the registered primary, women's diet quality, is reported as secondary
3Circulation 2021Paper says trial was registered prospectively; registry record was first submitted 33 months after study start
3Lancet Reg Health Am 2025Trial report omits one of two registered primary outcomes and does not disclose the omission
3Lancet Reg Health Am 2022HIV care trial report redefines registered retention outcome and omits hair-sample adherence without disclosure
2Circulation 2017Trial registered five months into enrollment and safety primary added later; paper says "prospectively registered"

2 verified finding(s) are held back because their write-up did not pass the fact-check after two revisions: PMC6878715, PMC12757054.

SIR and SMR confidence intervals are 34–42 times too narrow for the observed counts

J Hepatol (2020) · doi:10.1016/j.jhep.2020.07.040 · PMC7395653

Summary

The paper compares 111 COVID-19 cases and 20 deaths among 13,255 Spanish liver transplant (LT) recipients with the age- and gender-matched general population. It reports a standardised incidence ratio (SIR, ×100) of 191.22 (95% CI 190.28–192.16) and a standardised mortality ratio (SMR, ×100) of 95.55 (95% CI 94.25–96.85), with intervals of the same width for the over-60 subgroup. The Methods say the intervals treat the observed count as Poisson. With 111 cases and 20 deaths that method gives 157.3–230.3 for the SIR and 58.4–147.6 for the SMR. Every other standard method computed from the paper's own strata gives a similarly wide interval. The published widths instead correspond to Poisson counts of about 153,000–159,000 and 20,750 events. Supplementary Table S2 shows where the SMR width comes from: a projected count of 20,741 deaths, obtained by applying the LT stratum case-fatality to the general population's cases. The SIR still exceeds 100 once corrected. The SMR interval, however, includes 100. The paper's repeated statement that LT recipients' mortality is lower than in the matched general population, and its suggestion that immunosuppression may be protective, therefore lack statistical support in its own data.

What the paper reports

Patients and methods, "Data sources, calculation of standardised ratios, and statistical analysis":

Expected incidence and outcomes in LT patients were obtained from the numbers observed in the general population, after balancing for age and gender. Ninety-five percent confidence intervals (95% CI) were calculated under the assumption that the number of cases observed followed a Poisson distribution.

Abstract, Results:

A total of 111 liver transplant patients were diagnosed with COVID-19 (SIR = 191.2 [95% CI 190.3–192.2]).

The mortality rate was 18%, which was lower than in the matched general population (SMR = 95.5 [95% CI 94.2–96.8]).

Results, "Standardised incidence of COVID-19 in LT patients":

After adjusting for age and gender, the number of observed cases almost doubled the expected number of cases (Table S1) thus resulting in a standardised incidence ratio of 191.22 (95% CI 190.28–192.16). In the subset of patients older than 60 years, the standardised incidence ratio was even higher (206.48; 95% CI 205.45–207.52).

Results, "Standardised mortality ratio of COVID-19 in LT patients":

After adjusting for age and gender, the number of observed deaths among LT patients was slightly lower than the expected number of deaths (Table S2), thus resulting in a standardised mortality ratio of 95.55 (95% CI 94.25–96.85).

Adjusted mortality rates in patients older than 60 were similar in LT recipients and in the general population (standardised mortality ratio = 100.29; 95% CI 98.93–101.66).

Supplementary Tables S1 and S2 (mmc1.pdf) print the following at the foot of each table:

Observed 280,501; Expected 146,690 … 191.22 (95%CI 190.28-192.16)

Observed 20,741; Expected 21,707 … 95.55 (95%CI 94.25-96.85)

The conclusions that rest on the SMR interval. From the Abstract, Conclusions:

Being chronically immunosuppressed, liver transplant patients have an increased risk of acquiring COVID-19 but their mortality rates are lower than the matched general population.

From the Lay summary:

In liver transplant patients, chronic immunosuppression increases the risk of acquiring COVID-19 but it could reduce disease severity.

From the Highlights:

Mortality rates are lower than those observed in the matched general population.

From the Discussion:

we demonstrated that standardised mortality rates were actually lower in LT patients as compared with the general population

From the Discussion, concluding paragraph:

Mortality rates were lower than those observed in the age- and gender-matched general population, thereby suggesting that chronic immunosuppression could exert a certain protective effect against the most severe forms of COVID-19.

What the paper's own numbers show

Method. All ratios are written ×100, as in the paper. I recomputed each interval in three ways.

  1. The stated method. I took the observed count O (111 cases, 92 cases over 60 from Table S1, and 20 deaths, all over 60). The expected count E is implied by the reported ratio, E = O / (ratio/100). I then computed the exact (Garwood) Poisson 95% interval for O and divided it by E. Byar's approximation and the log-normal approximation agree to within about 3 units.
  2. Direct standardisation from the strata. I entered the 12 age × gender strata of Tables S1–S2 and checked how the point estimates were built. The published ratios are reproduced by direct standardisation: LT stratum rates are applied to the general-population structure, and the resulting projected count (the tables' "Observed" 280,501 and 20,741) is divided by the general-population count. For that estimator, the variance comes from the LT stratum counts, Σ(wᵢ²·xᵢ). I give log-scale intervals, plus Fay–Feuer gamma intervals as a sensitivity check.
  3. Indirect standardisation from the same strata, i.e. the conventional SIR/SMR (O/E, with E taken from the general-population stratum rates), with exact Poisson intervals.

Finally, I asked which Poisson count would produce each published width: N = (1.96 × ratio / half-width)².

Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Crude LT incidence Results 837.41 per 10⁵ (111 of 13,255) 837.42 per 10⁵
SIR point estimate Abstract; Results; Table S1 191.22 ("Observed 280,501; Expected 146,690") Direct standardisation from the strata: projected 280,556 cases / 146,690 = 191.26
SIR, 95% CI Abstract; Results; Table S1 190.28–192.16 (half-width 0.49% of the estimate) Stated method (O = 111, E = 58.05): 157.3–230.3, 39 times as wide. Direct standardisation: 142.0–257.7 (Fay–Feuer 138.5–271.9). Indirect: 177.9 (146.3–214.2)
SIR >60, point and 95% CI Results 206.48 (205.45–207.52) Stated method (O = 92, E = 44.56): 166.5–253.2, 42 times as wide. Direct: 206.61 (160.1–266.6; Fay–Feuer 157.3–269.7). Indirect: 194.9 (157.1–239.0)
SMR point estimate Abstract; Results; Table S2 95.55 ("Observed 20,741; Expected 21,707") Projected deaths 20,740.75 / 21,707 = 95.55
SMR, 95% CI Abstract; Results; Table S2 94.25–96.85 Stated method (O = 20, E = 20.93): 58.4–147.6, 34 times as wide, includes 100. Direct: 57.8–158.0 (Fay–Feuer 53.7–310.3). Indirect: 97.3 (59.5–150.3)
SMR >60, point and 95% CI Results 100.29 (98.93–101.66) 20,740.75 / 20,681 = 100.29. Stated method (O = 20, E = 19.94): 61.3–154.9. Direct: 60.7–165.8. Indirect: 99.3 (60.6–153.3)
Poisson count implied by each published width derived — SIR 158,967; SIR >60 152,887; SMR 20,753; SMR >60 20,737 (the observed counts are 111, 92, 20, 20)
Published SMR intervals, reproduced Table S2 94.25–96.85; 98.93–101.66 Poisson on the projected 20,741 deaths: 94.25–96.85 and 98.93–101.65
Published SIR >60 interval, reproduced Results 205.45–207.52 Poisson on the projected >60 count (152,770): 205.44–207.52
Published overall SIR interval Table S1 190.28–192.16 Not reproduced exactly. Poisson on 280,501 gives 190.51–191.93; on the 146,690 general-population cases, 190.24–192.20

Three of the four published intervals are reproduced, to within rounding, by treating a projected population-scale count as if it were the observed Poisson count. The fourth (overall SIR) has the same order of width. Its exact source could not be identified, but it corresponds to roughly 159,000 events. In every case the actual information is 111 cases or 20 deaths.

Two smaller inconsistencies in the supplement, neither of which changes any conclusion:

Figure

Reported and recomputed 95% confidence intervals for the SIR and SMR, overall and for patients over 60, on a log scale with a reference line at 100

Figure 1. Reported and recomputed 95% confidence intervals for the four standardised ratios (×100), on one log-scale axis. The thin vertical line marks a ratio of 100, meaning no difference from the matched general population. Orange shows the values printed in the paper; their intervals are narrower than the marker. Filled blue shows the paper's stated method: an exact Poisson interval for the observed count (111 cases, 92 cases over 60, 20 deaths), scaled by the expected count implied by the reported ratio. Open blue squares show the interval for the directly standardised ratio, computed from the 12 strata in Supplementary Tables S1–S2 with the variance of the LT stratum counts. Both recomputations of the SIR stay above 100. Both recomputations of the SMR, overall and over 60, include 100.

Relative half-width of a 95% Poisson interval against the count N, showing the reported intervals at about 20,750 and 153,000 to 159,000 events and the observed counts at 20, 92 and 111

Figure 2. Where the published widths come from. The gray line is the relation between a Poisson count N and the relative half-width of its 95% interval, 1.96/√N. Each reported interval (orange) is placed at the count its width implies: about 20,750 events for both SMR intervals and 152,887–158,967 events for the SIR intervals. Those values match the projected counts printed or implied in Tables S1–S2 (20,741 deaths; 152,770 cases over 60). The counts actually observed (blue) are 111 and 92 cases and 20 deaths. For these, the relative half-width is about 18–20% for the SIRs and about 44% for the SMR.

Why it matters

The incidence finding survives correction. The corrected SIR is about 1.9, with a 95% CI of about 1.57–2.30 under the stated method and about 1.42–2.58 for the directly standardised estimator. That still shows a higher incidence among LT recipients. The paper overstates its precision by a factor of about 40, but the direction and significance of the result do not change.

The mortality finding does not survive. Under every method computed here, the SMR interval runs from about 54–60 to 148–310, so it is consistent with mortality anywhere from roughly 40% lower to substantially higher than in the matched general population. The over-60 SMR, which the paper itself describes as "similar", has an interval of about 61–155. In short, 20 deaths against about 21 expected cannot distinguish LT recipients from the matched population.

The statement that mortality is "lower than in the matched general population" appears in five places: the Abstract Results, the Abstract Conclusions, a Highlight, the Discussion ("we demonstrated that standardised mortality rates were actually lower") and the concluding paragraph. The concluding paragraph uses it to suggest that "chronic immunosuppression could exert a certain protective effect", and the Lay summary says that immunosuppression "could reduce disease severity". These statements rest on an interval (94.25–96.85) that excludes 100 only because its variance comes from a projected count of 20,741 rather than from the 20 deaths observed. The paper is a widely cited early report on COVID-19 in transplant recipients (Europe PMC lists 260 citing records), so the unsupported claim about mortality and immunosuppression could have reached clinical readers.

Explanations considered

None of these explanations accounts for the numbers.

Status

As of 2026-10-23:

Sources: the PMC article (PMC7395653); the Europe PMC full-text XML and supplementary files for PMC7395653; Crossref metadata for doi:10.1016/j.jhep.2020.07.040; the Europe PMC citations list for PMID 32750442.

Suggested correction

  1. Recompute the four intervals from the observed counts, as the Methods describe: SIR 191.22 (95% CI about 157–230); SIR >60 206.48 (about 167–253); SMR 95.55 (about 58–148); SMR >60 100.29 (about 61–155). An alternative is to report the directly standardised ratios with intervals based on the LT stratum counts (about 142–258, 160–267, 58–158 and 61–166).
  2. State that LT recipients' COVID-19 mortality did not differ significantly from that of the age- and gender-matched general population. Revise the Abstract Results and Conclusions, the second Highlight, the Lay summary and the Discussion to match, and remove or clearly label as speculative the inference that chronic immunosuppression may be protective.
  3. Describe the standardisation actually used (direct, with LT stratum rates applied to the general-population structure) and relabel the "Observed" and "Expected" entries in Tables S1–S2 as projected and reference counts.
  4. Reconcile the Table S1 population total (strata sum 48,640,775 against 47,026,208 printed) and the Table S2 denominator (21,707 against 21,717 deaths).
Recomputation script (python recompute.py.txt)
# Recomputation for PMC7395653 (J Hepatol 2020, doi:10.1016/j.jhep.2020.07.040)
# Run:  python recompute.py.txt      (needs numpy, scipy, matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png)
# into the folder that holds this script.
#
# Inputs are copied from the paper:
#   - Results text: 111 LT cases, SIR 191.22 (190.28-192.16); >60 SIR 206.48 (205.45-207.52);
#     20 deaths, SMR 95.55 (94.25-96.85); >60 SMR 100.29 (98.93-101.66).
#   - Supplementary Tables S1 and S2 (mmc1.pdf): 12 age x gender strata.
# Ratios are written x100, as in the paper (191.22 means 1.9122).

import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs
strata = ["0-19 M", "0-19 W", "20-39 M", "20-39 W", "40-49 M", "40-49 W",
          "50-59 M", "50-59 W", "60-69 M", "60-69 W", "70+ M", "70+ W"]
# Table S1, general population (GP): population N and COVID-19 cases C
N_gp = np.array([4772317, 4481988, 5540648, 5474878, 3943490, 3869686,
                 3457353, 3516656, 2543236, 2738641, 3357068, 4944814])
C_gp = np.array([602, 625, 8497, 13307, 10105, 12173,
                 12818, 14620, 13398, 11387, 25693, 23465])
# Table S1, liver transplant (LT) population: patients n and COVID-19 cases c
n_lt = np.array([313, 285, 310, 280, 479, 333, 2373, 693, 3601, 1029, 2432, 1127])
c_lt = np.array([1, 0, 0, 1, 3, 4, 8, 2, 40, 13, 27, 12])
# Table S2: deaths among GP cases and among LT cases
D_gp = np.array([3, 5, 69, 55, 153, 84, 461, 206, 1355, 569, 10864, 7893])
D_lt = np.array([0, 0, 0, 0, 0, 0, 0, 0, 5, 3, 7, 5])
old = np.array([s.startswith(("60", "70")) for s in strata])

# Reported (point, lower, upper) as printed in Results
REP = {
    "SIR overall": (191.22, 190.28, 192.16),
    "SIR >60":     (206.48, 205.45, 207.52),
    "SMR overall": (95.55, 94.25, 96.85),
    "SMR >60":     (100.29, 98.93, 101.66),
}
# Abstract (rounded) versions
REP_ABS = {"SIR overall": (191.2, 190.3, 192.2), "SMR overall": (95.5, 94.2, 96.8)}


def garwood(o, a=0.05):
    """Exact Poisson CI for an observed count o."""
    lo = stats.chi2.ppf(a / 2, 2 * o) / 2 if o > 0 else 0.0
    hi = stats.chi2.ppf(1 - a / 2, 2 * (o + 1)) / 2
    return lo, hi


def byar(o):
    lo = o * (1 - 1 / (9 * o) - Z / (3 * np.sqrt(o))) ** 3
    hi = (o + 1) * (1 - 1 / (9 * (o + 1)) + Z / (3 * np.sqrt(o + 1))) ** 3
    return lo, hi


def fay_feuer(w, x, wmax):
    """Fay & Feuer (1997) gamma CI for a weighted sum of Poisson counts sum(w*x)."""
    y, v = np.sum(w * x), np.sum(w ** 2 * x)
    lo = stats.gamma.ppf(0.025, y ** 2 / v, scale=v / y)
    hi = stats.gamma.ppf(0.975, (y + wmax) ** 2 / (v + wmax ** 2),
                         scale=(v + wmax ** 2) / (y + wmax))
    return lo, hi


def log_delta(est, rel_se):
    return est * np.exp(-Z * rel_se), est * np.exp(Z * rel_se)


def h(t):
    print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)


# ---------------------------------------------------------------- 0. table checks
h("0. Totals of Supplementary Tables S1-S2")
print(f"GP cases (sum of strata)      {C_gp.sum():,}   (printed overall 146,690)")
print(f"GP population (sum of strata) {N_gp.sum():,}   (printed overall 47,026,208)")
print(f"LT patients / LT cases        {n_lt.sum():,} / {c_lt.sum()}   (printed 13,255 / 111)")
print(f"GP deaths / LT deaths         {D_gp.sum():,} / {D_lt.sum()}   (printed 21,717 / 20)")
print(f"LT cases >60: {c_lt[old].sum()};  LT deaths >60: {D_lt[old].sum()};  "
      f"GP cases >60: {C_gp[old].sum():,};  GP deaths >60: {D_gp[old].sum():,}")
print(f"Crude LT incidence  111/13,255 = {111 / 13255 * 1e5:.2f} per 10^5 (printed 837.41)")
print(f"Crude LT mortality  20/111     = {100 * 20 / 111:.2f}%  (printed 18%, 18.01)")

# ---------------------------------------------------------------- 1. how the ratios were built
h("1. Reproducing the published point estimates from the strata")
proj_cases = np.sum(N_gp * c_lt / n_lt)                 # LT stratum rates applied to GP population
proj_cases_old = np.sum((N_gp * c_lt / n_lt)[old])
cfr_lt = np.divide(D_lt, c_lt, out=np.zeros(12), where=c_lt > 0)
proj_deaths = np.sum(C_gp * cfr_lt)                    # LT stratum case fatality applied to GP cases
print(f"Projected cases = sum(N_gp * c/n)       = {proj_cases:,.1f}  (Table S1 prints 'Observed 280,501')")
print(f"  ratio to 146,690 GP cases              = {100 * proj_cases / C_gp.sum():.2f}  (reported 191.22)")
print(f"  >60: projected {proj_cases_old:,.1f} / {C_gp[old].sum():,} = "
      f"{100 * proj_cases_old / C_gp[old].sum():.2f}  (reported 206.48)")
print(f"Projected deaths = sum(C_gp * d/c)      = {proj_deaths:,.2f}  (Table S2 prints 'Observed 20,741')")
print(f"  ratio to 21,707 ('Expected' in S2)     = {100 * proj_deaths / 21707:.2f}  (reported 95.55)")
print(f"  ratio to 21,717 (GP deaths in S2/text) = {100 * proj_deaths / 21717:.2f}")
print(f"  >60: {proj_deaths:,.2f} / {D_gp[old].sum():,} = {100 * proj_deaths / D_gp[old].sum():.2f}  (reported 100.29)")
print("=> the published ratios are directly standardised: LT stratum rates applied to the")
print("   GP structure. The numerator 'Observed' is a projected count, not an observed one.")

# ---------------------------------------------------------------- 2. stated method
h("2. The paper's stated method: CI from a Poisson observed count")
O = {"SIR overall": 111, "SIR >60": int(c_lt[old].sum()),
     "SMR overall": 20, "SMR >60": int(D_lt[old].sum())}
stated = {}
for k, (r, lo, hi) in REP.items():
    o = O[k]
    E = o / (r / 100)
    gl, gh = garwood(o)
    bl, bh = byar(o)
    stated[k] = (100 * gl / E, 100 * gh / E)
    hw = (hi - lo) / 2
    exact_hw = (stated[k][1] - stated[k][0]) / 2
    print(f"{k}: O = {o}, E implied by reported ratio = {E:.2f}")
    print(f"   reported {r} ({lo}-{hi}); half-width {hw:.2f} = {100 * hw / r:.2f}% of estimate")
    print(f"   exact Poisson O CI {gl:.2f}-{gh:.2f} -> ratio CI {stated[k][0]:.1f}-{stated[k][1]:.1f}"
          f"   (Byar {100 * bl / E:.1f}-{100 * bh / E:.1f}; log-normal "
          f"{r * np.exp(-Z / np.sqrt(o)):.1f}-{r * np.exp(Z / np.sqrt(o)):.1f})")
    print(f"   exact CI is {(stated[k][1] - stated[k][0]) / (hi - lo):.0f} times as wide as the reported CI")
for k, (r, lo, hi) in REP_ABS.items():
    print(f"Abstract {k}: {r} ({lo}-{hi})")

# ---------------------------------------------------------------- 3. strata-based alternatives
h("3. Alternatives computed from the strata (both standardisation directions)")
# 3a indirect standardisation (conventional SIR/SMR)
E_ind = np.sum(n_lt * C_gp / N_gp)
E_ind_old = np.sum((n_lt * C_gp / N_gp)[old])
cfr_gp = D_gp / C_gp
Ed_ind = np.sum(c_lt * cfr_gp)
Ed_ind_old = np.sum((c_lt * cfr_gp)[old])
ind = {}
for k, o, E in [("SIR overall", 111, E_ind), ("SIR >60", O["SIR >60"], E_ind_old),
                ("SMR overall", 20, Ed_ind), ("SMR >60", 20, Ed_ind_old)]:
    gl, gh = garwood(o)
    ind[k] = (100 * o / E, 100 * gl / E, 100 * gh / E)
    print(f"Indirect {k}: O = {o}, E = {E:.2f}, ratio {ind[k][0]:.1f} (exact 95% CI {ind[k][1]:.1f}-{ind[k][2]:.1f})")

# 3b direct standardisation with the variance of the LT stratum counts
dirci = {}
specs = [
    ("SIR overall", N_gp / n_lt / C_gp.sum(), c_lt, np.ones(12, bool)),
    ("SIR >60", N_gp / n_lt / C_gp[old].sum(), c_lt, old),
    ("SMR overall", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / 21707, D_lt, c_lt > 0),
    ("SMR >60", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / D_gp[old].sum(), D_lt, (c_lt > 0) & old),
]
for k, w, x, use in specs:
    w = np.where(use, w, 0.0)
    est = np.sum(w * x)
    rel = np.sqrt(np.sum(w ** 2 * x)) / est
    dl, dh = log_delta(100 * est, rel)
    fl, fh = fay_feuer(w, x, w[use].max())
    dirci[k] = (100 * est, dl, dh)
    print(f"Direct {k}: ratio {100 * est:.2f}, relative SE {rel:.3f}; log-scale 95% CI {dl:.1f}-{dh:.1f};"
          f" Fay-Feuer gamma 95% CI {100 * fl:.1f}-{100 * fh:.1f}")

# ---------------------------------------------------------------- 4. where the width comes from
h("4. Count implied by each reported interval width (Poisson, N = (z*ratio/half-width)^2)")
implied = {}
for k, (r, lo, hi) in REP.items():
    hw = (hi - lo) / 2
    implied[k] = (Z * r / hw) ** 2
    print(f"{k}: implied N = {implied[k]:,.0f}   (observed count {O[k]})")
    zeq = hw / (r / np.sqrt(O[k]))            # z that the observed-count SE would need
    print(f"   relative half-width for the observed count: {100 * Z / np.sqrt(O[k]):.1f}%;"
          f" the reported width equals z = {zeq:.3f}, i.e. a {100 * (2 * stats.norm.cdf(zeq) - 1):.1f}% confidence level")
for lab, cnt, r in [("SIR overall, Poisson on 280,501 (S1 'Observed')", 280501, 191.22),
                    ("SIR overall, Poisson on 146,690 (GP cases)", 146690, 191.22),
                    ("SIR >60, Poisson on projected >60 count", proj_cases_old, 206.48),
                    ("SMR overall, Poisson on 20,741 (S2 'Observed')", 20741, 95.55),
                    ("SMR >60, Poisson on 20,741", 20741, 100.29)]:
    se = r / np.sqrt(cnt)
    print(f"{lab}: normal {r - Z * se:.2f}-{r + Z * se:.2f}")

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "axes.edgecolor": INK2, "svg.fonttype": "none"})


def style(ax):
    ax.set_facecolor(BG)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.grid(axis="x", which="major", color=GRID, lw=0.8)
    ax.set_axisbelow(True)


# Figure 1: reported vs recomputed intervals on one log axis
groups = [("SIR, all LT patients", "SIR overall", "111 cases"),
          ("SIR, age >60", "SIR >60", "92 cases"),
          ("SMR, all LT patients", "SMR overall", "20 deaths"),
          ("SMR, age >60", "SMR >60", "20 deaths")]
fig, ax = plt.subplots(figsize=(7.2, 6.0), facecolor=BG)
style(ax)
y, yt, ylab = 0, [], []
for title, k, cnt in groups:
    r, lo, hi = REP[k]
    rows = [("Reported", r, lo, hi, ORANGE, "o", True),
            ("Exact Poisson, " + cnt, r, *stated[k], BLUE, "o", True),
            ("Strata, direct std.", *dirci[k], BLUE, "s", False)]
    ax.text(30.5, y - 0.62, title, fontsize=11, fontweight="bold", color=INK, va="center")
    for lab, p, a, b, col, mk, filled in rows:
        ax.plot([a, b], [y, y], color=col, lw=2, solid_capstyle="butt")
        ax.plot([a, a], [y - 0.12, y + 0.12], color=col, lw=2)
        ax.plot([b, b], [y - 0.12, y + 0.12], color=col, lw=2)
        ax.plot(p, y, marker=mk, ms=8, color=col, mfc=col if filled else BG, mew=2, zorder=3)
        fmt = "{:.2f}" if lab == "Reported" else "{:.1f}"
        ax.text(b * 1.04, y, f"{fmt.format(p)} ({fmt.format(a)}–{fmt.format(b)})",
                va="center", fontsize=10, color=INK2)
        yt.append(y)
        ylab.append(lab)
        y += 0.55
    y += 0.75
ax.axvline(100, color=INK2, lw=0.9, zorder=1)
ax.text(100, -1.05, "ratio = 100\n(no difference)", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(30, 720)
ax.set_xticks([40, 60, 100, 150, 200, 300])
ax.set_xticklabels(["40", "60", "100", "150", "200", "300"])
ax.minorticks_off()
ax.set_yticks(yt)
ax.set_yticklabels(ylab, fontsize=10)
for t, lab in zip(ax.get_yticklabels(), ylab):
    t.set_color(INK)
ax.set_ylim(y - 0.5, -1.25)
ax.set_xlabel("Standardised ratio ×100 with 95% CI (log scale)")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
           plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
                      label="Recomputed: stated method, exact Poisson on the observed count"),
           plt.Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
                      label="Recomputed: Tables S1–S2 strata, direct standardisation variance")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.42, -0.11), frameon=False,
          fontsize=10, ncol=1)
fig.tight_layout()
for ext in ("svg", "png"):
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)

# Figure 2: relative half-width vs the Poisson count it corresponds to
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
xs = np.logspace(1, 6, 200)
ax.plot(xs, 100 * Z / np.sqrt(xs), color=INK2, lw=1.2, zorder=1)
ax.text(1500, 100 * Z / np.sqrt(1500) * 1.5, "Poisson: half-width = 1.96/√N", color=INK2, fontsize=10, ha="left")
pts_rep = [("SIR overall", "SIR, all"), ("SIR >60", "SIR, >60"), ("SMR overall", "SMR, all and >60")]
for k, lab in pts_rep:
    r, lo, hi = REP[k]
    ax.plot(implied[k], 100 * (hi - lo) / 2 / r, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(implied["SMR >60"], 100 * (REP["SMR >60"][2] - REP["SMR >60"][1]) / 2 / REP["SMR >60"][0],
        "o", ms=9, color=ORANGE, zorder=3)
ax.annotate(f"Reported SIR intervals: ≈ {implied['SIR >60']:,.0f}–{implied['SIR overall']:,.0f} events\n"
            f"(projected >60 count {proj_cases_old:,.0f};\n Table S1 'Observed' 280,501)",
            (implied["SIR overall"], 100 * 0.94 / 191.22), xytext=(250, 0.115), fontsize=10, color=INK,
            arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.annotate(f"Reported SMR intervals: ≈ {implied['SMR >60']:,.0f}–{implied['SMR overall']:,.0f} events\n"
            f"(Table S2 'Observed' 20,741)",
            (implied["SMR overall"], 100 * 1.30 / 95.55), xytext=(250, 0.55), fontsize=10, color=INK,
            arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
for o in (111, 92, 20):
    ax.plot(o, 100 * Z / np.sqrt(o), "o", ms=9, color=BLUE, zorder=3)
ax.text(150, 100 * Z / np.sqrt(111) * 1.05, "111 cases (all) and 92 (>60)", color=INK, fontsize=10)
ax.text(27, 100 * Z / np.sqrt(20) * 1.05, "20 deaths", color=INK, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(10, 1e6)
ax.set_ylim(0.1, 100)
ax.set_yticks([0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50])
ax.set_yticklabels(["0.1%", "0.2%", "0.5%", "1%", "2%", "5%", "10%", "20%", "50%"])
ax.set_xticks([10, 100, 1e3, 1e4, 1e5, 1e6])
ax.set_xticklabels(["10", "100", "1,000", "10,000", "100,000", "1,000,000"])
ax.minorticks_off()
ax.set_xlabel("Poisson count N that the interval corresponds to (log scale)")
ax.set_ylabel("95% CI half-width, % of estimate")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0,
                      label="Reported intervals, at the count their width implies"),
           plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0,
                      label="Recomputed: width for the observed counts")]
ax.legend(handles=handles, loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
    fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nfigures written to", HERE)

Reported risk ratios are ratios of mean 1/2 codes, understating relative risk reductions for perpetration 9- to 28-fold

JAMA Network Open (2018) · doi:10.1001/jamanetworkopen.2018.1213 · PMC6324287

Summary

This secondary analysis of a cluster randomized trial reports that a school-based HIV/STD risk-reduction intervention reduced self-reported perpetration of forced sex. The reported "risk ratios" for perpetration are 0.978, 0.964, 0.959, 0.967 and 0.964 at 3–54 months, and the paper sums up the effect as "about a 3% reduction in risk". The paper's own counts give risk ratios of 0.39 to 0.67, which are relative reductions of 33% to 61%. Every reported value instead matches (1 + p₁)/(1 + p₀), the ratio of group means of an outcome coded 1 (no) and 2 (yes), which is the coding the Methods describe. After approximate adjustment for the arms' different sex mix, all 10 cumulative-incidence ratios in Table 3 are reproduced to three decimals. The direction of the effect and the significance tests are not affected. What is wrong is the published effect size, its confidence intervals, the sex-comparison ratios in Table 4 and the "3%" statement, and these are the numbers a reader, meta-analyst or guideline would take from the paper.

What the paper reports

Methods, "Outcomes":

Binary variables were used to assess history of ever perpetrating and experiencing forced sexual intercourse, with responses coded 1 for respondents not reporting ever experiencing the event and 2 for those reporting ever experiencing the event.

Methods, "Statistical Analysis" (the superscript is a reference number):

The efficacy of the HIV/STD intervention compared with the control intervention on the forced sex outcomes was tested using generalized estimating equation Poisson regression models, adjusting for gender and students clustered within schools.

Risk ratios42 and corresponding 95% confidence intervals are reported.

Abstract, "Results" (repeated word for word in the Results text):

Fewer intervention than control participants reported forced sex perpetration postintervention compared with the control group at 3 months (9 of 561 [2%] vs 20 of 491 [4%]; risk ratio [RR], 0.978; 95% CI, 0.959-0.997), 6 months (17 of 561 [3%] vs 35 of 491 [7%]; RR, 0.964; 95% CI, 0.941-0.988), 12 months (21 of 561 [4%] vs 42 of 491 [9%]; RR, 0.959; 95% CI, 0.934-0.985), 42 months (41 of 561 [7%] vs 56 of 491 [11%]; RR, 0.967; 95% CI, 0.937-0.998), and 54 months (52 of 561 [9%] vs 68 of 491 [14%]; RR, 0.964; 95% CI, 0.932-0.997).

Table 3 (columns: perpetrating RR (95% CI) | P | experiencing RR (95% CI) | P), first and last rows:

3 | 0.978 (0.959-0.997) | .02 | 0.983 (0.963-1.004) | .11

3-54 | 0.990 (0.982-0.999) | .02 | 0.992 (0.982-1.001) | .09

Table 4 (first five columns: follow-up | boys | girls | "Boys vs Girls, RR (95% CI)" | P), perpetration at 12 months:

12 | 58/495 (12) | 5/557 (1) | 0.904 (0.881-0.928) | <.001

Results text:

Table 4 shows reports of perpetrating and experiencing forced sex were significantly higher among boys compared with girls at each of the postintervention assessments

Among boys, the effect on perpetration of forced sex was significant at 3-month follow-up (RR, 0.952; 95% CI, 0.917-0.988), 6-month follow-up (RR, 0.934; 95% CI, 0.892-0.979), and 12-month follow-up (RR, 0.924; 95% CI, 0.879-0.972)

Discussion, "Limitations":

The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk.

What the paper's own numbers show

Method. With a log-link Poisson model, the exponentiated arm coefficient estimates the ratio of the outcome's means in the two arms. For a 0/1 outcome that ratio is the risk ratio p₁/p₀. For the 1/2 coding the Methods describe it is (1 + p₁)/(1 + p₀), which stays close to 1 whatever the risks are. For each reported ratio I computed, from the counts printed next to it (Table 2 cumulative and time-specific incidence; Table 4 counts by sex): the risk ratio with an unadjusted Wald CI; the ratio of 1/2-code means with an unadjusted CI; and, as alternative readings, the inverted "no event" ratio (1 − p₀)/(1 − p₁), 1 + risk difference, and the odds ratio. The paper's model is adjusted for sex, and the arms differ in sex mix (Table 1: 306 of 561 vs 251 of 491 girls), while boys report forced sex several times as often as girls (Table 4). I therefore also applied an approximate sex standardization: each arm's expected value under no intervention effect, using the pooled sex-specific risks from Table 4, divides out the confounding. All CIs computed here ignore clustering. They show the scale of each quantity and do not replace the trial's own clustered analysis. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Perpetration, 3 mo (9/561 vs 20/491) Abstract; Results; Table 3 RR 0.978 (0.959–0.997) risk ratio 0.394 (0.18–0.86); ratio of 1/2-code means 0.976 (0.957–0.996); sex-standardized 0.978
Perpetration, 6 mo (17/561 vs 35/491) Abstract; Results; Table 3 RR 0.964 (0.941–0.988) risk ratio 0.425 (0.24–0.75); code-mean ratio 0.962 (0.938–0.986); sex-standardized 0.964
Perpetration, 12 mo (21/561 vs 42/491) Abstract; Results; Table 3 RR 0.959 (0.934–0.985) risk ratio 0.438 (0.26–0.73); code-mean ratio 0.956 (0.930–0.982); sex-standardized 0.959
Perpetration, 42 mo (41/561 vs 56/491) Abstract; Results; Table 3 RR 0.967 (0.937–0.998) risk ratio 0.641 (0.44–0.94); code-mean ratio 0.963 (0.933–0.995); sex-standardized 0.967
Perpetration, 54 mo (52/561 vs 68/491) Abstract; Results; Table 3 RR 0.964 (0.932–0.997) risk ratio 0.669 (0.48–0.94); code-mean ratio 0.960 (0.927–0.994); sex-standardized 0.964
Experience, 3/6/12/42/54 mo Table 3 0.983 / 0.977 / 0.966 / 0.963 / 0.972 risk ratios 0.544 / 0.586 / 0.561 / 0.655 / 0.771; code-mean ratios 0.981 / 0.974 / 0.962 / 0.959 / 0.967; sex-standardized 0.983 / 0.977 / 0.966 / 0.963 / 0.972
Time-specific incidence pooled over visits, perpetration (52/2717 vs 68/2302) Table 3, row 3-54 0.990 risk ratio 0.648; code-mean ratio 0.990
Time-specific incidence pooled over visits, experience (72/2680 vs 82/2287) Table 3, row 3-54 0.992 risk ratio 0.749; code-mean ratio 0.991
Width of reported CIs Table 3 SE(log) 0.0099–0.0186 matches the unadjusted SE of the code-mean ratio (ratio 0.96–0.99 at all 10 points); the unadjusted SE of the log risk ratio is 8 to 40 times larger
Boys vs girls, perpetration 3 mo (26/495 vs 3/557) Table 4 0.956 boys/girls risk ratio 9.75; girls/boys code-mean ratio 0.955
Boys vs girls, perpetration 54 mo (95/495 vs 25/557) Table 4 0.878 boys/girls risk ratio 4.28; girls/boys code-mean ratio 0.877
Boys vs girls, experience 12 mo (72/494 vs 5/556) Table 4 0.882 boys/girls risk ratio 16.21; girls/boys code-mean ratio 0.881
The 12 Table 4 "Boys vs Girls" values that have counts beside them Table 4 0.875–0.994 girls/boys code-mean ratio within 0.0015 of each reported value; boys/girls risk ratios 2.80–18.01. The two pooled 3–54 month values in this column (0.968, 0.965) have no counts beside them and are also below 1
Size of effect Discussion, Limitations "about a 3% reduction in risk" mean of (1 − reported perpetration RR) = 3.4%; relative reductions from the counts 61%, 57%, 56%, 36%, 33% (perpetration) and 46%, 41%, 44%, 34%, 23% (experience)
Significance Table 3 P = .02, .004, .002, .04, .03 (perpetration) unadjusted two-proportion tests P = 0.015, 0.002, 0.001, 0.022, 0.020; the reported P values are somewhat larger, as clustering would make them

Across the 24 reported ratios with counts beside them (Table 3 cumulative and pooled rows, Table 4 sex comparison), the crude ratio of 1/2-code means differs from the reported value by at most 0.0047 (RMS 0.0024). The small remaining gap in Table 3 is in the direction sex adjustment predicts: the intervention arm has more girls, so the crude comparison slightly favors it. The approximate sex standardization closes the gap at all 10 cumulative time points (largest difference 0.0004).

Figure

Reported risk ratios compared with risk ratios computed from the paper's counts, both outcomes, all five follow-ups

Figure 1. Intervention vs control on one log-scale axis, with the line at 1 marking no effect. Orange circles: the "risk ratios" and 95% CIs reported in Table 3 (and, for perpetration, in the Abstract and Results). The reported CIs are so narrow that they barely extend past the markers. Blue diamonds: the risk ratio p₁/p₀ computed from the cumulative-incidence counts printed beside them (Table 2 and Abstract), with an unadjusted 95% CI that ignores clustering, so the true intervals would be somewhat wider. The reported values sit at 0.96–0.98 at every time point. The counts give 0.39–0.67 for perpetration and 0.54–0.77 for experience. The direction agrees, but on the relative-reduction scale the size of the effect differs by a factor of 9 to 28 for perpetration (8 to 27 for experience).

Which quantity the reported ratios track

Figure 2. For each of the 24 reported ratios that have counts beside them, the difference between a candidate quantity computed from those counts and the reported value. The orange line at 0 is the reported value. Filled blue circles: ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀) (for Table 4, girls over boys). Crosses: the same ratio after approximate sex standardization (Table 3 cumulative rows only, because the model was adjusted for sex). Hollow triangles: inverted "no event" ratio. Hollow squares: 1 + risk difference. The risk ratio and the odds ratio are off this scale (in the Table 3 cumulative rows the risk ratio is 0.20–0.58 below the reported values and the odds ratio 0.23–0.59 below). The two alternatives agree with the reported values only when events are rare, and they drift away as prevalence rises. The ratio of 1/2-code means stays within 0.005 everywhere, and within 0.0004 once sex is accounted for.

Why it matters

Explanations considered

Status

Suggested correction

  1. Refit the GEE models with the outcomes coded 0/1, for example as a log-link Poisson or log-binomial model with robust standard errors, adjusted for sex and clustering as described. Replace every risk ratio and 95% CI in the Abstract, the Results text (including the boys-only estimates) and Table 3, including the pooled 3–54 month rows.
  2. Recompute Table 4 on the 0/1 scale, state the direction of the comparison (boys vs girls) so that it matches the values, and re-estimate the intervention × sex interaction ratios.
  3. Revise "The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk" to reflect the corrected estimates. The crude risk ratios from the published counts suggest relative reductions of roughly a third to more than a half for perpetration.
Recomputation script (python recompute.py.txt)
"""Recompute the intervention-effect "risk ratios" of PMC6324287
(JAMA Netw Open 2018, doi:10.1001/jamanetworkopen.2018.1213).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

All inputs are copied from the paper as archived in PMC:
  Table 1: arm sizes and number of girls per arm
  Table 2: cumulative and time-specific incidence counts by arm
  Table 3: reported RRs (95% CI) and P values for the intervention effect
  Table 4: incidence counts by sex and reported "Boys vs Girls" RRs
  Methods: outcomes "coded 1 for respondents not reporting ever experiencing the
           event and 2 for those reporting ever experiencing the event", analysed with
           GEE Poisson regression adjusted for gender and clustering.

If a log-link Poisson model is fitted to a variable Y coded 1/2 instead of 0/1, the
exponentiated coefficient estimates E[Y | arm 1] / E[Y | arm 0] = (1 + p1) / (1 + p0),
not the risk ratio p1 / p0.

All confidence intervals computed here are unadjusted (individuals treated as
independent, no clustering); they are for comparing scales, not substitutes for the
trial's own clustered analysis.
"""
import math
import os

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ------------------------------------------------------------------ inputs (from the paper)
MONTHS = [3, 6, 12, 42, 54]
N_INT, N_CTL = 561, 491                    # Table 1
GIRLS = {"int": 306, "ctl": 251}           # Table 1, "Female, No. (%)"
BOYS = {"int": N_INT - GIRLS["int"], "ctl": N_CTL - GIRLS["ctl"]}

# Table 2 cumulative incidence (a/n1 intervention, b/n0 control) and Table 3 reported RR, CI, P
PERP = {3: (9, 561, 20, 491, 0.978, 0.959, 0.997, .02),
        6: (17, 561, 35, 491, 0.964, 0.941, 0.988, .004),
        12: (21, 561, 42, 491, 0.959, 0.934, 0.985, .002),
        42: (41, 561, 56, 491, 0.967, 0.937, 0.998, .04),
        54: (52, 561, 68, 491, 0.964, 0.932, 0.997, .03)}
EXPE = {3: (13, 559, 21, 491, 0.983, 0.963, 1.004, .11),
        6: (22, 559, 33, 491, 0.977, 0.953, 1.002, .07),
        12: (30, 559, 47, 491, 0.966, 0.939, 0.994, .02),
        42: (50, 559, 67, 491, 0.963, 0.931, 0.996, .03),
        54: (72, 559, 82, 491, 0.972, 0.937, 1.008, .12)}
# Table 2 time-specific incidence (intervention list, control list) and Table 3 row "3-54"
TS = {"Perpetration": ([(9, 561), (8, 552), (4, 544), (20, 540), (11, 520)],
                       [(20, 491), (15, 471), (7, 456), (14, 449), (12, 435)], 0.990),
      "Experience": ([(13, 559), (9, 546), (8, 537), (20, 529), (22, 509)],
                     [(21, 491), (12, 470), (14, 458), (20, 444), (15, 424)], 0.992)}
# Table 4: boys a/n1, girls b/n0, reported "Boys vs Girls" RR
T4 = {"Perpetration": {0: (4, 499, 1, 558, 0.994), 3: (26, 495, 3, 557, 0.956),
                       6: (47, 495, 5, 557, 0.923), 12: (58, 495, 5, 557, 0.904),
                       42: (78, 495, 19, 557, 0.894), 54: (95, 495, 25, 557, 0.878)},
      "Experience": {0: (5, 499, 2, 558, 0.994), 3: (32, 494, 2, 556, 0.943),
                     6: (51, 494, 4, 556, 0.914), 12: (72, 494, 5, 556, 0.882),
                     42: (91, 494, 26, 556, 0.885), 54: (113, 494, 41, 556, 0.875)}}


# ------------------------------------------------------------------ helpers
def rr(a, n1, b, n0):
    return (a / n1) / (b / n0)


def rr_ci(a, n1, b, n0):
    r = rr(a, n1, b, n0)
    se = math.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
    return r, r * math.exp(-Z * se), r * math.exp(Z * se), se


def code_ratio(a, n1, b, n0):
    """Ratio of mean 1/2 codes: (1 + p1) / (1 + p0)."""
    return (1 + a / n1) / (1 + b / n0)


def code_ratio_ci(a, n1, b, n0):
    p1, p0 = a / n1, b / n0
    m1, m0 = 1 + p1, 1 + p0
    se = math.sqrt(p1 * (1 - p1) / (n1 * m1 ** 2) + p0 * (1 - p0) / (n0 * m0 ** 2))
    r = m1 / m0
    return r, r * math.exp(-Z * se), r * math.exp(Z * se), se


def noevent_inverted(a, n1, b, n0):
    """Control 'no event' proportion / intervention 'no event' proportion."""
    return ((n0 - b) / n0) / ((n1 - a) / n1)


def one_plus_rd(a, n1, b, n0):
    return 1 + a / n1 - b / n0


def odds_ratio(a, n1, b, n0):
    return (a * (n0 - b)) / (b * (n1 - a))


def two_prop_p(a, n1, b, n0):
    p = (a + b) / (n1 + n0)
    se = math.sqrt(p * (1 - p) * (1 / n1 + 1 / n0))
    zz = (a / n1 - b / n0) / se
    return 2 * stats.norm.sf(abs(zz))


def sex_confounding(t, outcome, kind):
    """Approximate sex standardization. Expected arm-level quantity if each arm had the
    pooled sex-specific risks of Table 4 and no intervention effect; the crude ratio
    divided by this factor approximates a sex-adjusted ratio. kind='code' or 'binary'."""
    a, nb, b, ng, _ = T4[outcome][t]
    pb, pg = a / nb, b / ng
    e_int = (BOYS["int"] * pb + GIRLS["int"] * pg) / N_INT
    e_ctl = (BOYS["ctl"] * pb + GIRLS["ctl"] * pg) / N_CTL
    if kind == "code":
        return (1 + e_int) / (1 + e_ctl)
    return e_int / e_ctl


def hr(title):
    print("\n" + "=" * 100 + "\n" + title + "\n" + "=" * 100)


# ------------------------------------------------------------------ 1. Table 3 cumulative incidence
hr("1. Table 3 (and Abstract/Results): reported RR vs quantities computed from Table 2 counts")
rows_fig1 = []
rows_fig2 = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    print(f"\n{outcome}")
    print(" mo | counts        | reported RR (95% CI)  | ratio of 1/2-code means (unadj. CI) "
          "| sex-std code ratio | risk ratio p1/p0 (unadj. CI) | sex-std RR | rel. reduction | abs. diff (pp)")
    for t, (a, n1, b, n0, R, L, U, P) in d.items():
        cr, cl, cu, cse = code_ratio_ci(a, n1, b, n0)
        r, rl, ru, rse = rr_ci(a, n1, b, n0)
        cr_std = cr / sex_confounding(t, outcome, "code")
        r_std = r / sex_confounding(t, outcome, "binary")
        print(f" {t:>2} | {a:>2}/{n1} v {b:>2}/{n0} | {R:.3f} ({L:.3f}-{U:.3f})   "
              f"| {cr:.3f} ({cl:.3f}-{cu:.3f})                 | {cr_std:.3f}              "
              f"| {r:.3f} ({rl:.2f}-{ru:.2f})             | {r_std:.3f}      "
              f"| {100 * (1 - r):.0f}%            | {100 * (a / n1 - b / n0):+.1f}")
        rows_fig1.append((outcome, t, R, L, U, r, rl, ru))
        rows_fig2.append((f"{outcome[:4]}. {t} mo", R, cr, noevent_inverted(a, n1, b, n0),
                          one_plus_rd(a, n1, b, n0), "T3"))

hr("2. Precision: width of reported CI vs unadjusted CI on each scale (log scale half-width / 1.96)")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    for t, (a, n1, b, n0, R, L, U, P) in d.items():
        rep_se = math.log(U / L) / (2 * Z)
        _, _, _, cse = code_ratio_ci(a, n1, b, n0)
        _, _, _, rse = rr_ci(a, n1, b, n0)
        print(f" {outcome[:4]} {t:>2} mo: reported SE(log) {rep_se:.4f}; unadjusted SE(log) code-mean "
              f"ratio {cse:.4f} (ratio {rep_se / cse:.2f}); unadjusted SE(log RR) {rse:.3f} "
              f"(ratio of RR SE to reported SE: {rse / rep_se:.0f}x)")

hr("3. Significance: reported P vs unadjusted two-proportion test from the same counts")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    for t, (a, n1, b, n0, R, L, U, P) in d.items():
        print(f" {outcome[:4]} {t:>2} mo: reported P {P}; unadjusted two-proportion z-test P "
              f"{two_prop_p(a, n1, b, n0):.4f}")

# ------------------------------------------------------------------ 2. Table 3 time-specific row
hr("4. Table 3 row '3-54' (time-specific incidence pooled over the 5 visits)")
for outcome, (iv, cv, R) in TS.items():
    a = sum(x for x, _ in iv); n1 = sum(y for _, y in iv)
    b = sum(x for x, _ in cv); n0 = sum(y for _, y in cv)
    print(f" {outcome}: {a}/{n1} vs {b}/{n0}; reported {R:.3f}; ratio of code means "
          f"{code_ratio(a, n1, b, n0):.3f}; risk ratio {rr(a, n1, b, n0):.3f}")
    rows_fig2.append((f"{outcome[:4]}. 3-54 pooled", R, code_ratio(a, n1, b, n0),
                      noevent_inverted(a, n1, b, n0), one_plus_rd(a, n1, b, n0), "T3"))

# ------------------------------------------------------------------ 3. Table 4 sex comparison
hr("5. Table 4 'Boys vs Girls' RR vs quantities computed from the Table 4 counts")
for outcome, d in T4.items():
    print(f"\n{outcome}")
    for t, (a, n1, b, n0, R) in d.items():
        print(f" {t:>2} mo: boys {a}/{n1} ({100 * a / n1:.1f}%) girls {b}/{n0} ({100 * b / n0:.1f}%); "
              f"reported {R:.3f}; girls/boys code-mean ratio {code_ratio(b, n0, a, n1):.3f}; "
              f"boys/girls code-mean ratio {code_ratio(a, n1, b, n0):.3f}; "
              f"boys/girls risk ratio {rr(a, n1, b, n0):.2f}")
        rows_fig2.append((f"{outcome[:4]}. " + ("baseline" if t == 0 else f"{t} mo"), R, code_ratio(b, n0, a, n1),
                          noevent_inverted(b, n0, a, n1), one_plus_rd(b, n0, a, n1), "T4"))

# Table 4 also prints pooled 3-54 month "Boys vs Girls" values, with no counts beside them
T4_POOLED = {"Perpetration": 0.968, "Experience": 0.965}
print("\n Table 4 rows '3-54' (no counts printed, so not recomputable): "
      + ", ".join(f"{k} {v:.3f}" for k, v in T4_POOLED.items()))
allt4 = [v[4] for d in T4.values() for v in d.values()] + list(T4_POOLED.values())
print(f" Table 4 'Boys vs Girls' column: {len(allt4)} values in all, "
      f"{sum(len(d) for d in T4.values())} with counts; range {min(allt4):.3f}-{max(allt4):.3f}; "
      f"all below 1: {all(v < 1 for v in allt4)}")
t4dev = [code_ratio(b, n0, a, n1) - R for d in T4.values() for (a, n1, b, n0, R) in d.values()]
t4rr = [rr(a, n1, b, n0) for d in T4.values() for (a, n1, b, n0, R) in d.values()]
print(f" the 12 with counts: max |girls/boys code-mean ratio - reported| {max(abs(v) for v in t4dev):.4f}; "
      f"boys/girls risk ratios {min(t4rr):.2f}-{max(t4rr):.2f}")

# ------------------------------------------------------------------ 4. candidate definitions
hr("6. Which quantity do the 24 reported values track? (candidate minus reported)")
cands = {"ratio of mean 1/2 codes": 2, "inverted 'no event' ratio": 3, "1 + risk difference": 4}
for name, k in cands.items():
    dev = np.array([row[k] - row[1] for row in rows_fig2])
    print(f" {name:<26}: mean dev {dev.mean():+.4f}, max |dev| {np.abs(dev).max():.4f}, "
          f"RMS {math.sqrt((dev ** 2).mean()):.4f}")
dev_rr = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    for t, (a, n1, b, n0, *_r) in d.items():
        dev_rr.append(rr(a, n1, b, n0) - _r[0])
for tab in ("T3", "T4"):
    sub = [r for r in rows_fig2 if r[5] == tab]
    print(f" {tab} ({len(sub)} values): max |dev| code-mean {max(abs(r[2] - r[1]) for r in sub):.4f}, "
          f"no-event {max(abs(r[3] - r[1]) for r in sub):.4f}, 1+RD {max(abs(r[4] - r[1]) for r in sub):.4f}")
dev_std = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    for t, (a, n1, b, n0, R, *_r) in d.items():
        dev_std.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
print(f" sex-standardized ratio of 1/2-code means (Table 3 cumulative, 10 values): max |dev| "
      f"{max(abs(v) for v in dev_std):.4f}; equal to reported at 3 decimals in "
      f"{sum(abs(v) < 0.0005 for v in dev_std)} of 10")
print(f" risk ratio p1/p0 (Table 3 cumulative only): deviations {min(dev_rr):+.3f} to {max(dev_rr):+.3f}")
print(" odds ratios, Table 3 cumulative: " + ", ".join(
    f"{outcome[:4]} {t}: {odds_ratio(*v[:4]):.3f}" for outcome, d in (("Perpetration", PERP),
                                                                       ("Experience", EXPE))
    for t, v in d.items()))
dev_or = [odds_ratio(*v[:4]) - v[4] for d in (PERP, EXPE) for v in d.values()]
print(f" odds ratio (Table 3 cumulative only): deviations {min(dev_or):+.4f} to {max(dev_or):+.4f}")
dev_code_cum = [r[2] - r[1] for r in rows_fig2[:10]]
print(f" crude ratio of 1/2-code means (Table 3 cumulative, 10 values): deviations "
      f"{min(dev_code_cum):+.4f} to {max(dev_code_cum):+.4f}")
noev_t4 = [r[3] - r[1] for r in rows_fig2 if r[5] == "T4"]
print(f" inverted 'no event' ratio, Table 4: lowest deviation {min(noev_t4):+.4f} "
      f"(figure 2 y-axis lower limit -0.046)")

# ------------------------------------------------------------------ 5. magnitude statement
hr("7. Discussion: 'about a 3% reduction in risk'")
rep = np.array([v[4] for v in PERP.values()])
print(f" mean(1 - reported perpetration RR) = {100 * np.mean(1 - rep):.1f}%")
rrs = np.array([rr(*v[:4]) for v in PERP.values()])
print(" perpetration relative reductions from counts: "
      + ", ".join(f"{100 * (1 - x):.0f}%" for x in rrs))
rre = np.array([rr(*v[:4]) for v in EXPE.values()])
print(" experience relative reductions from counts:   "
      + ", ".join(f"{100 * (1 - x):.0f}%" for x in rre))
print(" ratio of relative reduction from counts to (1 - reported RR), perpetration: "
      + ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rrs, rep)))
repe = np.array([v[4] for v in EXPE.values()])
print(" ratio of relative reduction from counts to (1 - reported RR), experience:   "
      + ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rre, repe)))
fold = [(1 - x) / (1 - R) for x, R in zip(np.r_[rrs, rre], np.r_[rep, repe])]
print(f" range of that ratio over all 10 points in Figure 1: {min(fold):.1f} to {max(fold):.1f}")
print(" perpetration absolute differences (pp):       "
      + ", ".join(f"{100 * (v[0] / v[1] - v[2] / v[3]):.1f}" for v in PERP.values()))

hr("8. Can adjustment for sex move the risk ratio to ~0.97? Worst case: all events in one sex")
for t, (a, n1, b, n0, R, *_r) in PERP.items():
    lo = (a / BOYS["int"]) / (b / BOYS["ctl"])
    hi = (a / GIRLS["int"]) / (b / GIRLS["ctl"])
    print(f" perpetration {t:>2} mo: within-boys RR if all events were boys {lo:.3f}; "
          f"within-girls RR if all events were girls {hi:.3f}; reported {R:.3f}")


# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
                     "savefig.facecolor": BG, "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
    plt.close(fig)


# Figure 1: reported RR vs risk ratio from the counts, log scale, one axis
fig, ax = plt.subplots(figsize=(7.2, 5.4))
labels, ys = [], []
y = 0
for i, (outcome, t, R, L, U, r, rl, ru) in enumerate(rows_fig1):
    if i == 5:
        y += 0.8
    yy = -y
    ax.plot([L, U], [yy + 0.17] * 2, color=ORANGE, lw=2, solid_capstyle="round")
    ax.plot(R, yy + 0.17, "o", color=ORANGE, ms=8, mec=BG, mew=1.2, zorder=3)
    ax.plot([rl, ru], [yy - 0.17] * 2, color=BLUE, lw=2, solid_capstyle="round")
    ax.plot(r, yy - 0.17, "D", color=BLUE, ms=7.5, mec=BG, mew=1.2, zorder=3)
    ax.text(U * 1.025, yy + 0.17, f"{R:.3f}", va="center", ha="left", fontsize=10, color=INK2)
    ax.text(rl / 1.04, yy - 0.17, f"{r:.2f}", va="center", ha="right", fontsize=10, color=INK2)
    labels.append(f"{t} mo")
    ys.append(yy)
    y += 1
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.1, 1.6)
ticks = [0.1, 0.2, 0.3, 0.5, 0.7, 1.0, 1.5]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{x:g}" for x in ticks])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_ylim(ys[-1] - 0.7, 0.9)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.103, ys[0] + 0.55, "Perpetrating forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(0.103, ys[5] + 0.55, "Experiencing forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(1.02, 0.75, "no effect", fontsize=10, color=INK2, va="center")
ax.set_xlabel("Intervention vs control, ratio (log scale)")
h1 = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
             label="Reported \"risk ratio\" and 95% CI (Table 3)")[0]
h2 = ax.plot([], [], "D-", color=BLUE, ms=7.5, lw=2,
             label="Risk ratio from the paper's counts (Table 2), unadjusted 95% CI")[0]
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.11), ncol=1,
          frameon=False, fontsize=10)
save(fig, "fig1")

# Figure 2: which quantity the reported values track
fig, ax = plt.subplots(figsize=(7.2, 4.6))
x = np.arange(len(rows_fig2))
dev_code = np.array([r[2] - r[1] for r in rows_fig2])
dev_noev = np.array([r[3] - r[1] for r in rows_fig2])
dev_rd = np.array([r[4] - r[1] for r in rows_fig2])
ax.axhline(0, color=ORANGE, lw=2, zorder=1)
ax.plot(x, dev_rd, "s", ms=8, mfc=BG, mec=BLUE, mew=1.6, label="1 + risk difference")
ax.plot(x, dev_noev, "^", ms=8.5, mfc=BG, mec=BLUE, mew=1.6,
        label="Inverted \"no event\" ratio, (1 − p₀)/(1 − p₁)")
ax.plot(x, dev_code, "o", ms=8, color=BLUE, mec=BG, mew=1.0,
        label="Ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀)")
std_x, std_dev = [], []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
    for t, (a, n1, b, n0, R, *_r) in d.items():
        std_x.append(len(std_x))  # same order as the first 10 rows of rows_fig2
        std_dev.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
ax.plot(std_x, std_dev, "X", ms=9, color=BLUE, mec=BG, mew=0.8, zorder=4,
        label="Ratio of mean 1/2 codes, approx. sex-standardized (Table 3 cumulative rows)")
ax.text(len(x) - 0.5, 0.0015, "reported value (difference = 0)", color=INK, fontsize=10,
        ha="right", va="bottom")
n3 = sum(1 for r in rows_fig2 if r[5] == "T3")
ax.axvline(n3 - 0.5, color=INK2, lw=0.8, ls=(0, (2, 2)))
ax.text((n3 - 1) / 2, 0.012, "Table 3: intervention vs control", ha="center", fontsize=10, color=INK2)
ax.text(n3 + (len(x) - n3 - 1) / 2, 0.012, "Table 4: sex comparison", ha="center",
        fontsize=10, color=INK2)
ax.set_xticks(x)
ax.set_xticklabels([r[0] for r in rows_fig2], rotation=60, ha="right", fontsize=10)
ax.set_ylabel("Candidate minus reported value")
ax.set_ylim(-0.046, 0.016)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.legend(loc="upper left", bbox_to_anchor=(-0.02, -0.33), frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)

Reported nationwide energy, cost and CO2 savings are 1,000 times what the paper's inputs give

Nat Commun (2025) · doi:10.1038/s41467-025-64962-2 · PMC12575868

Summary

The paper simulates an electrochromic smart window in office buildings in all 50 U.S. states and reports an average HVAC saving of 26.72 MJ/m² against low-e glass. It then applies that saving to the 97 billion ft² of U.S. commercial floor space and reports, in the Abstract, Results and Discussion, a nationwide saving of 66.87 billion MWh, worth $2.67–7.35 trillion and avoiding 0.87–66.94 billion t of CO2. The paper's own two inputs give 26.72 MJ/m² × 9.01 × 10⁹ m² = 2.41 × 10¹¹ MJ, which is 66.9 million MWh (66.9 TWh). The reported figure is 999.8 times larger, and the cost and CO2 totals, which Supplementary Eqs. S10–S11 compute from it, are 1,000 times too large as well: they should be about $2.7–8.4 billion and 0.87–67 million t. These totals are the abstract's headline impact claim. The reported energy figure is about 16 times all U.S. utility-scale electricity generation, so no reading of the inputs makes it possible.

What the paper reports

Abstract:

Simulations indicate that a large-scale deployment across the U.S. can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons.

With its rapid switching, long-term durability, and scalability, this device presents significant economic and environmental advantages for real-world applications.

Results, section "HVAC energy-saving capabilities as building smart windows", paragraph 4 (the "54" after "2018" is the citation to reference 54, the EIA Commercial Buildings Energy Consumption Survey):

As shown in Table S7 (Supplementary Information), the average energy savings across U.S. states is calculated to be 26.72 MJ m-2. The U.S. Energy Information Administration (EIA) reported that the total commercial building area in the U.S. was about 97 billion square feet in 201854, allowing for a calculation of total energy savings across the country to be as high as 66.87 billion MWh.

The economic costs of producing this large amount of electricity using nuclear, coal, gas, photovoltaic, onshore, and offshore wind power generation systems are estimated to range from $2.67 trillion to $7.35 trillion (Fig. 7d and Table S8, Eq. S10, Supplementary Information).

Even clean energy sources, like nuclear, photovoltaic, and wind, produce 0.87 billion tons of CO2, while coal, a highly polluting energy source, has an astounding 66.94 billion tons of CO2 emissions.

It holds significant economic and environmental value, aiding in energy conservation, emission reduction, and achieving net-zero energy buildings globally.

Discussion, paragraph 1:

The smart window consistently displayed higher energy efficiency across all states in the U.S., with an average value of 26.72 MJ m⁻². If implemented nationwide in office buildings, it is estimated to save 66.87 billion MWh of electricity. Depending on the power generation methods, the savings would translate into electricity costs ranging from $2.67 trillion to $7.35 trillion and a reduction in carbon emissions of between 0.87 billion and 66.94 billion tons.

Methods, building energy simulation:

A year-long simulation of monthly building energy consumption was performed for Phoenix, Arizona, along with an annual analysis of building energy consumption across various climate zones in all 50 U.S. states to investigate regional impacts.

Supplementary Information (MOESM1), Table S8 and Eqs. S10–S11. The equations are C = LCOE × E / 10¹² (trillion USD) and T = f × E / 10¹² (billion t CO2), "where … LCOE = levelized cost of electricity (USD/MWh), f = CO2 emission factor (kg CO2/MWh), and E = electricity consumption (MWh)". Table S8 lists the following LCOE (USD/MWh), cost (trillion USD), emission factor (kg/MWh) and CO2 (billion t): Nuclear 110 / 7.35 / 13 / 0.87; Coal 105 / 7.02 / 1001 / 66.94; Gas CCGT 60 / 4.01 / 486 / 32.50; Solar PV 55 / 3.68 / 43 / 2.88; Wind onshore 40 / 2.67 / 13 / 0.87; Wind offshore 125 / 8.36 / 13 / 0.87. The source data file (MOESM3, sheet "Fig. 7d") gives the same costs and CO2 totals.

All quotes from the article were checked verbatim against the PMC-archived text. The supplementary passages were read in the publisher's MOESM1 file.

What the paper's own numbers show

Method. I took the 50 per-state savings from Supplementary Table S7 and checked their mean and that each equals the low-e value minus the device value. I converted the stated floor area with the exact factor 1 ft² = 0.09290304 m², multiplied, and converted with 1 MWh = 3,600 MJ. I then applied the paper's own Eqs. S10–S11 and the Table S8 LCOE and emission factors to the recomputed energy. For scale, the recomputed and reported totals are compared with two EIA figures: U.S. utility-scale generation in 2023 and total energy use in U.S. commercial buildings in 2018. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
Mean annual HVAC saving, 50 states Results; Discussion; Table S7 26.72 MJ/m² 26.716 MJ/m² (mean of the 50 rows; each row = low-e − device)
U.S. commercial floor area Results (ref. 54) 97 billion ft² 97 × 10⁹ × 0.09290304 = 9.0116 × 10⁹ m²
Nationwide annual saving, energy — — 26.72 × 9.0116 × 10⁹ = 2.408 × 10¹¹ MJ
Nationwide annual saving, electricity Abstract; Results; Discussion 66.87 billion MWh (6.687 × 10¹⁰ MWh) 6.689 × 10⁷ MWh = 66.89 million MWh (66.9 TWh)
Ratio reported / recomputed — — 999.8
Same total in kWh — — 66.89 billion kWh
Per-area saving implied by 66.87 billion MWh derived — 26,714 MJ/m², which is 489 times the largest single-state saving (54.60 MJ/m², Hawaii) and 12.8 times the largest total annual HVAC use in Table S7 (2,080.05 MJ/m², Alaska, low-e glass)
Cost of the saved electricity (Eq. S10) Abstract; Results; Discussion; Table S8 $2.67–7.35 trillion $2.68–7.36 billion for the technologies in the stated range; $8.36 billion for offshore wind
CO2 associated with the saved electricity (Eq. S11) Abstract; Results; Discussion; Table S8 0.87–66.94 billion t 0.87–66.95 million t
Unit values implied by the text ranges derived $2.67–7.35 trillion; 0.87–66.94 billion t $39.9–109.9 per MWh and 13.0–1,001 kg CO2/MWh, which match the Table S8 LCOE and emission factors. The cost and CO2 totals are therefore Eq. S10–S11 applied to 66.87 billion MWh, and carry the same factor of 1,000
Offshore wind cost vs stated range (secondary) Table S8 vs Results text Text: "$2.67 trillion to $7.35 trillion", including offshore wind Table S8 lists offshore wind at 8.36 (125 USD/MWh), above the stated upper end
Solar PV CO2 vs text (secondary) Table S8 vs Results text Text: "nuclear, photovoltaic, and wind, produce 0.87 billion tons" Table S8 gives 2.88 for solar PV (43 kg/MWh)
Pre-revision values (peer review file, MOESM2, Table R1) — 24.439 MJ/m² → "61.1"; 26.72 MJ/m² → "66.87", in a row labelled "(MWh)" 61.18 and 66.89 million MWh; the same ratio of about 1,000 (998.8 and 999.8) if the listed values are read as billion MWh, as in the paper
Reported total vs U.S. utility-scale generation, 2023 (EIA: 4,178 TWh) external 66,870 TWh Reported = 16.0 times all U.S. generation; recomputed = 1.60% of it
Reported total vs energy used in U.S. commercial buildings, 2018 (EIA CBECS: 6.8 quadrillion Btu = 1,993 TWh) external 66,870 TWh Reported = 33.6 times all commercial-building energy; recomputed = 3.36% of it

The recomputed electricity figure (66.89 million MWh) and the reported one (66.87 billion MWh) agree in their leading digits and differ only in the multiplier. The small difference in the last digit is consistent with rounding of the mean saving or of the conversion factor (with the unrounded Table S7 mean, the recomputed value is 66.88 million MWh). Applying Eqs. S10–S11 to the reported E reproduces every Table S8 entry to within 0.01, so the supplement is internally consistent with the inflated E. The discrepancy enters at the conversion of the energy product to MWh.

Figure

Calculation chain from the paper's inputs to the nationwide totals, with the step where the reported figure diverges

Figure 1. The arithmetic chain behind the nationwide totals, using only numbers stated in the paper and its supplement. The mean per-area saving (Table S7) and the commercial floor area (ref. 54, converted to m²) give 2.408 × 10¹¹ MJ per year. The highlighted, dashed step is the conversion to MWh. At 3,600 MJ per MWh the total is 66.9 million MWh (blue, left). The paper reports 66.87 billion MWh (orange, right), which equals the same total expressed in kWh. The bottom row applies the paper's Eqs. S10–S11 and Table S8 factors. With the recomputed energy, the costs are $2.68–7.36 billion ($8.36 billion for offshore wind) and the CO2 is 0.87–66.95 million t. The paper reports the same digits in trillions of dollars and billions of tonnes.

Nationwide annual energy on one log axis: reported, recomputed, and two U.S. totals for scale

Figure 2. Annual energy on a single logarithmic axis, in TWh (1 TWh = 1 million MWh). The orange dot is the paper's reported nationwide saving (66,870 TWh). The blue dot is the value recomputed from the paper's own inputs (66.9 TWh). The ratio between them is 999.8. The gray squares are external reference points from the U.S. EIA: all U.S. utility-scale electricity generation in 2023 (4,178 TWh) and all energy used in U.S. commercial buildings in 2018 (6.8 quadrillion Btu, 1,993 TWh). The recomputed saving is 3.4% of commercial-building energy use. The reported saving is 33.6 times that use and 16 times the country's entire electricity output.

Why it matters

The nationwide totals are the only quantitative statement of the device's real-world impact in the Abstract. The abstract's closing claim of "significant economic and environmental advantages" rests largely on them, and the Results ("substantial economic and environmental benefits") and Discussion repeat them. As printed, they overstate the energy, money and CO2 saved by three orders of magnitude. A reader of the abstract would take away savings of trillions of dollars and tens of billions of tonnes of CO2 per year. These figures are larger than the entire U.S. electricity system, and the CO2 figure is larger than recent estimates of total global annual emissions. The values the paper's inputs actually support are billions of dollars and tens of millions of tonnes. That is still a meaningful result, but it is a different claim. The correct values appear nowhere in the paper or its supplement, so a reader can recover them only by redoing the calculation. Table S8, Fig. 7d and the Fig. 7d source data carry the inflated values, so anyone reusing the supplement would inherit the error.

The following are not affected: the device chemistry and electrochromic measurements (switching times of 1.6 s and 0.8 s, 64.23% optical modulation, capacity, energy recovery of 44.4% and 31.0%, cycling stability), the outdoor temperature test, the SHGC values, and the per-area EnergyPlus results (Fig. 7a–c, Tables S6–S7, including the 26.72 MJ/m² average). The direction of the comparison with low-e glass is unchanged.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Abstract. Replace "can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons" with:

can save 66.9 million MWh (66.9 TWh) per year, amounting to up to $7.36 billion, and reduce CO2 emissions by up to 66.95 million tons.

Results, paragraph 4. Replace "as high as 66.87 billion MWh" with "as high as 66.9 million MWh (66.9 TWh)". Replace "from $2.67 trillion to $7.35 trillion" with "from $2.68 billion to $8.36 billion". This upper end includes offshore wind, which the sentence lists; alternatively, offshore wind can be removed from the list and the range given as $2.68–7.36 billion. Replace "0.87 billion tons" and "66.94 billion tons" with "0.87 million tons" and "66.95 million tons". For solar PV, the Table S8 factor gives 2.88 million tons, so "photovoltaic" should be dropped from the 0.87 group or given its own value.

Discussion, paragraph 1. Make the same replacements: 66.9 million MWh; $2.68–8.36 billion (or $2.68–7.36 billion); 0.87–66.95 million tons.

Supplementary Table S8 and Fig. 7d (and source data sheet "Fig. 7d"). Relabel the cost column "billions, USD" and the CO2 column "million tons" (the digits are correct for E = 66.9 million MWh to within rounding). Alternatively, keep the trillion/billion headings and divide every entry by 1,000. In Eqs. S10–S11, the stated units then become C in billion USD = LCOE × E / 10⁹ and T in million t = f × E / 10⁹, with E = 6.69 × 10⁷ MWh.

Recomputation script (python recompute.py.txt)
"""Recompute the nationwide energy, cost and CO2 totals in PMC12575868
(Nat Commun 2025; doi:10.1038/s41467-025-64962-2) from the paper's own stated inputs,
and redraw fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy and matplotlib)

Inputs are copied from the published article (Abstract, Results "HVAC energy-saving
capabilities as building smart windows", Discussion) and from Supplementary Information
Tables S7 and S8 and Eqs. S10-S11 (MOESM1_ESM.docx). Two external reference values, used
only for the plausibility comparison, are from the U.S. EIA and are labelled as such.
"""
import os
import numpy as np

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs from the paper
# Supplementary Table S7: annual HVAC energy, MJ per m2 (EESD, low-e glass, saving), 50 states
S7 = {
    "Alabama": (274.50, 308.85, 34.35), "Alaska": (2074.00, 2080.05, 6.05),
    "Arizona": (346.10, 389.50, 43.40), "Arkansas": (361.15, 395.80, 34.65),
    "California": (70.80, 92.85, 22.05), "Colorado": (244.25, 267.50, 23.25),
    "Connecticut": (367.25, 384.80, 17.55), "Delaware": (281.10, 303.15, 22.05),
    "Florida": (448.30, 501.50, 53.20), "Georgia": (250.80, 280.20, 29.40),
    "Hawaii": (398.85, 453.45, 54.60), "Idaho": (228.95, 252.00, 23.05),
    "Illinois": (411.15, 434.10, 22.95), "Indiana": (373.80, 392.10, 18.30),
    "Iowa": (502.75, 529.10, 26.35), "Kansas": (387.00, 420.15, 33.15),
    "Kentucky": (304.25, 329.65, 25.40), "Louisiana": (370.35, 412.60, 42.25),
    "Maine": (394.90, 410.05, 15.15), "Maryland": (292.85, 318.60, 25.75),
    "Massachusetts": (260.90, 278.60, 17.70), "Michigan": (387.80, 409.40, 21.60),
    "Minnesota": (595.05, 615.90, 20.85), "Mississippi": (330.80, 368.10, 37.30),
    "Missouri": (413.10, 444.00, 30.90), "Montana": (293.85, 308.80, 14.95),
    "Nebraska": (304.70, 328.10, 23.40), "Nevada": (304.20, 351.50, 47.30),
    "New Hampshire": (413.55, 431.10, 17.55), "New Jersey": (244.40, 268.90, 24.50),
    "New Mexico": (183.40, 212.00, 28.60), "New York": (258.55, 284.50, 25.95),
    "North Carolina": (252.50, 283.75, 31.25), "North Dakota": (812.25, 828.90, 16.65),
    "Ohio": (335.35, 359.35, 24.00), "Oklahoma": (360.75, 398.90, 38.15),
    "Oregon": (150.15, 167.40, 17.25), "Pennsylvania": (289.55, 309.30, 19.75),
    "Rhode Island": (185.30, 203.15, 17.85), "South Carolina": (323.55, 363.95, 40.40),
    "South Dakota": (623.05, 645.75, 22.70), "Tennessee": (308.10, 345.40, 37.30),
    "Texas": (358.35, 400.40, 42.05), "Utah": (215.55, 240.00, 24.45),
    "Vermont": (469.00, 486.40, 17.40), "Virginia": (283.25, 315.90, 32.65),
    "Washington": (124.90, 140.75, 15.85), "West Virginia": (307.15, 328.20, 21.05),
    "Wisconsin": (467.85, 487.15, 19.30), "Wyoming": (381.80, 396.05, 14.25),
}
SAVING_REPORTED = 26.72        # MJ/m2, Results and Discussion
AREA_FT2 = 97e9                # "about 97 billion square feet in 2018" (EIA CBECS, ref 54)
E_REPORTED_MWH = 66.87e9       # "66.87 billion MWh" (Abstract, Results, Discussion)
# Supplementary Table S8: LCOE (USD/MWh), cost (trillion USD), f (kg CO2/MWh), CO2 (billion t)
S8 = {
    "Nuclear": (110, 7.35, 13, 0.87),
    "Coal": (105, 7.02, 1001, 66.94),
    "Gas CCGT": (60, 4.01, 486, 32.50),
    "Solar PV": (55, 3.68, 43, 2.88),
    "Wind onshore": (40, 2.67, 13, 0.87),
    "Wind offshore": (125, 8.36, 13, 0.87),
}
TEXT_COST_RANGE = (2.67, 7.35)       # trillion USD, Abstract/Results/Discussion
TEXT_CO2_RANGE = (0.87, 66.94)       # billion t, Results/Discussion
# Peer review file (MOESM2), pre-revision vs revised values (Table R1)
PRF = {"before": (24.439, 61.1), "after": (26.72, 66.87)}  # (MJ/m2, "Total energy savings ... (MWh)")

# ---------------------------------------------------------------- unit constants
FT2_TO_M2 = 0.09290304        # exact
MJ_PER_MWH = 3600.0           # exact
MJ_PER_KWH = 3.6              # exact
J_PER_BTU = 1055.05585        # International Table Btu

# ---------------------------------------------------------------- external references (plausibility only)
US_GEN_2023_TWH = 4178.0      # EIA FAQ id=427: "about 4,178 billion kilowatthours" utility-scale, 2023 (preliminary)
CBECS_2018_QUADS = 6.8        # EIA CBECS 2018: commercial buildings "consumed 6.8 quadrillion British thermal units"

print("=== 1. Mean saving across 50 states (Table S7) ===")
names = list(S7)
eesd = np.array([S7[k][0] for k in names])
lowe = np.array([S7[k][1] for k in names])
sav = np.array([S7[k][2] for k in names])
print(f"n = {len(sav)}; mean saving = {sav.mean():.4f} MJ/m2 (reported {SAVING_REPORTED})")
print(f"every saving = low-e minus EESD: {np.allclose(lowe - eesd, sav, atol=0.006)}")
print(f"savings range {sav.min():.2f}-{sav.max():.2f} MJ/m2; "
      f"total HVAC range: EESD {eesd.min():.2f}-{eesd.max():.2f}, low-e {lowe.min():.2f}-{lowe.max():.2f} MJ/m2")

print("\n=== 2. Nationwide energy saving from the paper's inputs ===")
area_m2 = AREA_FT2 * FT2_TO_M2
E_MJ = SAVING_REPORTED * area_m2
E_MWH = E_MJ / MJ_PER_MWH
E_KWH = E_MJ / MJ_PER_KWH
print(f"area = 97e9 ft2 x {FT2_TO_M2} = {area_m2:.4e} m2 ({area_m2/1e9:.2f} billion m2)")
print(f"E = {SAVING_REPORTED} MJ/m2 x {area_m2:.4e} m2 = {E_MJ:.4e} MJ")
print(f"  = {E_MWH:.4e} MWh = {E_MWH/1e6:.2f} million MWh = {E_MWH/1e6:.2f} TWh")
print(f"  = {E_KWH:.4e} kWh = {E_KWH/1e9:.2f} billion kWh")
E_MWH_mean = sav.mean() * area_m2 / MJ_PER_MWH
print(f"with the unrounded Table S7 mean ({sav.mean():.4f}): {E_MWH_mean/1e6:.2f} million MWh")
ratio = E_REPORTED_MWH / E_MWH
print(f"reported 66.87e9 MWh / recomputed = {ratio:.1f}")
implied = E_REPORTED_MWH * MJ_PER_MWH / area_m2
print(f"per-area saving implied by 66.87e9 MWh = {implied:,.0f} MJ/m2 ({implied/1000:.2f} GJ/m2)")
print(f"  = {implied/lowe.max():.1f} x the largest total annual HVAC use in Table S7 "
      f"({lowe.max():.2f} MJ/m2, low-e glass, Alaska)")
print(f"  = {implied/sav.max():.0f} x the largest single-state saving ({sav.max():.2f} MJ/m2)")
print(f"if 97e9 were read as m2 rather than ft2: {SAVING_REPORTED*97e9/MJ_PER_MWH/1e6:.0f} million MWh "
      f"(factor {SAVING_REPORTED*97e9/MJ_PER_MWH/E_MWH:.2f}, not 1000)")
print(f"years of savings needed to reach 66.87e9 MWh: {ratio:.0f}")

print("\n=== 3. Peer review file (pre-revision values) ===")
for k, (s, rep) in PRF.items():
    e = s * area_m2 / MJ_PER_MWH
    print(f"{k:6s}: {s} MJ/m2 -> {e/1e6:.2f} million MWh; table lists {rep} -> ratio "
          f"{rep*1e9/e:.1f} if read as billion MWh")

print("\n=== 4. Cost and CO2 (Table S8; Eq. S10 C = LCOE*E/1e12, Eq. S11 T = f*E/1e12) ===")
print(f"{'Technology':14s} {'LCOE':>5s} {'S8 cost':>8s} {'LCOE*Erep':>10s} {'cost w/ E':>10s} "
      f"{'f':>5s} {'S8 CO2':>7s} {'f*Erep':>7s} {'CO2 w/ E':>9s}")
print(f"{'':14s} {'$/MWh':>5s} {'T$':>8s} {'T$':>10s} {'B$':>10s} {'kg/MWh':>5s} {'Bt':>7s} {'Bt':>7s} {'Mt':>9s}")
costs_true, co2_true = {}, {}
for k, (l, c, f, t) in S8.items():
    costs_true[k] = l * E_MWH / 1e9          # billion USD
    co2_true[k] = f * E_MWH / 1e9            # kg -> million t
    print(f"{k:14s} {l:5d} {c:8.2f} {l*E_REPORTED_MWH/1e12:10.2f} {costs_true[k]:10.2f} "
          f"{f:5d} {t:7.2f} {f*E_REPORTED_MWH/1e12:7.2f} {co2_true[k]:9.2f}")
ct = sorted(costs_true.values())
print(f"recomputed cost range: ${ct[0]:.2f}-{ct[-1]:.2f} billion "
      f"(without offshore wind: ${ct[0]:.2f}-{sorted(ct)[-2]:.2f} billion); reported "
      f"${TEXT_COST_RANGE[0]}-{TEXT_COST_RANGE[1]} trillion")
co = sorted(co2_true.values())
print(f"recomputed CO2 range: {co[0]:.2f}-{co[-1]:.2f} million t; reported "
      f"{TEXT_CO2_RANGE[0]}-{TEXT_CO2_RANGE[1]} billion t")
print(f"implied $/MWh from the text range: {TEXT_COST_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
      f"{TEXT_COST_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"implied kg CO2/MWh from the text range: {TEXT_CO2_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
      f"{TEXT_CO2_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"Table S8 offshore wind cost {S8['Wind offshore'][1]} T$ lies outside the text range "
      f"{TEXT_COST_RANGE}: {S8['Wind offshore'][1] > TEXT_COST_RANGE[1]}")
print(f"Table S8 solar PV CO2 {S8['Solar PV'][3]} Bt vs text 'nuclear, photovoltaic, and wind ... 0.87'")

print("\n=== 5. Plausibility against U.S. totals (EIA) ===")
cbecs_twh = CBECS_2018_QUADS * 1e15 * J_PER_BTU / 3.6e15
rep_twh, true_twh = E_REPORTED_MWH / 1e6, E_MWH / 1e6
print(f"reported {rep_twh:,.0f} TWh; recomputed {true_twh:.1f} TWh")
print(f"US utility-scale generation 2023 = {US_GEN_2023_TWH:,.0f} TWh -> reported is "
      f"{rep_twh/US_GEN_2023_TWH:.1f}x; recomputed is {100*true_twh/US_GEN_2023_TWH:.2f}%")
print(f"US commercial-building energy 2018 = {CBECS_2018_QUADS} quads = {cbecs_twh:,.0f} TWh -> reported is "
      f"{rep_twh/cbecs_twh:.1f}x; recomputed is {100*true_twh/cbecs_twh:.2f}%")

# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
                     "svg.fonttype": "none", "font.family": "DejaVu Sans"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
    fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
    plt.close(fig)


def box(ax, x, y, w, h, title, body, edge=SEC, fill="white", tcol=TXT, ls="-", lw=1.2):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                fc=fill, ec=edge, lw=lw, ls=ls))
    ax.text(x + w / 2, y + h - 1.6, title, ha="center", va="top", fontsize=10, color=tcol, weight="bold")
    ax.text(x + w / 2, y + h - 5.2, body, ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)


def arrow(ax, p, q, col=SEC):
    ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=col,
                                 shrinkA=0, shrinkB=0))


def sci(x):
    e = int(np.floor(np.log10(x)))
    return f"{x/10**e:.3f} × 10$^{{{e}}}$"


# ---------------------------------------------------------------- fig1: calculation chain
fig = plt.figure(figsize=(7.2, 6.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 100); ax.axis("off")
ax.set_facecolor(BG)

box(ax, 2, 85, 43, 12, "Mean saving per m² (Table S7)",
    f"mean of 50 states = {sav.mean():.3f} MJ/m²\n(paper: {SAVING_REPORTED} MJ/m²)")
box(ax, 55, 85, 43, 12, "U.S. commercial floor area (ref. 54)",
    f"97 × 10$^{{9}}$ ft² × {FT2_TO_M2:.4f}\n= {area_m2/1e9:.2f} × 10$^{{9}}$ m²")
arrow(ax, (23.5, 84.2), (40, 77.6)); arrow(ax, (76.5, 84.2), (60, 77.6))
box(ax, 22, 65, 56, 12, "Annual saving in energy units",
    f"{SAVING_REPORTED} MJ/m² × {area_m2/1e9:.2f} × 10$^{{9}}$ m² = {sci(E_MJ)} MJ")
arrow(ax, (50, 64.2), (50, 58.4))
# highlighted divergence step
ax.add_patch(FancyBboxPatch((6, 45), 88, 13, boxstyle="round,pad=0.4,rounding_size=1.2",
                            fc="#fdf0ea", ec=ORANGE, lw=1.6, ls=(0, (4, 2))))
ax.text(50, 56.4, "Step where the figures diverge: MJ → MWh", ha="center", va="top",
        fontsize=10.5, weight="bold", color=TXT)
ax.text(50, 52.4, f"1 MWh = 3,600 MJ, so {sci(E_MJ)} MJ = {sci(E_MWH)} MWh.\n"
        f"The paper's figure equals the same total in kWh ({E_KWH/1e9:.2f} billion kWh), stated as MWh.",
        ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)
arrow(ax, (28, 44.2), (26, 37.6), BLUE); arrow(ax, (72, 44.2), (74, 37.6), ORANGE)
box(ax, 2, 25, 46, 12.5, "Recomputed from the paper's inputs",
    f"{E_MWH/1e6:.1f} million MWh\n({E_MWH/1e6:.1f} TWh per year)", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 25, 46, 12.5, "Reported (Abstract, Results, Discussion)",
    f"66.87 billion MWh\n(ratio to recomputed: {ratio:.1f})", edge=ORANGE, tcol=ORANGE, lw=2)
arrow(ax, (25, 24.2), (25, 18.2), BLUE); arrow(ax, (75, 24.2), (75, 18.2), ORANGE)
box(ax, 2, 3, 46, 15, "Eqs. S10–S11 with this E",
    f"cost ${ct[0]:.2f}–{sorted(ct)[-2]:.2f} billion\n(offshore wind ${ct[-1]:.2f} billion)\n"
    f"CO$_2$ {co[0]:.2f}–{co[-1]:.2f} million t", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 3, 46, 15, "Reported (Table S8 and text)",
    f"cost ${TEXT_COST_RANGE[0]}–{TEXT_COST_RANGE[1]} trillion\n(Table S8 offshore wind 8.36 trillion)\n"
    f"CO$_2$ {TEXT_CO2_RANGE[0]}–{TEXT_CO2_RANGE[1]} billion t", edge=ORANGE, tcol=ORANGE, lw=2)
save(fig, "fig1")

# ---------------------------------------------------------------- fig2: scale on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
rows = [
    ("Paper: nationwide saving", rep_twh, ORANGE, "o", f"{rep_twh:,.0f} TWh (66.87 billion MWh)"),
    ("Recomputed from the\npaper's own inputs", true_twh, BLUE, "o", f"{true_twh:.1f} TWh (66.9 million MWh)"),
    ("All U.S. utility-scale\nelectricity, 2023 (EIA)", US_GEN_2023_TWH, SEC, "s", f"{US_GEN_2023_TWH:,.0f} TWh"),
    ("All energy used in U.S.\ncommercial buildings,\n2018 (EIA CBECS)", cbecs_twh, SEC, "s",
     f"{cbecs_twh:,.0f} TWh (6.8 quadrillion Btu)"),
]
ys = np.arange(len(rows))[::-1]
for (lab, v, c, m, txt), y in zip(rows, ys):
    ax.plot([v], [y], marker=m, ms=10, color=c, mec=c, ls="none", zorder=3)
    if v > 2e4:
        ax.text(v / 1.35, y, txt, ha="right", va="center", fontsize=10, color=c)
    else:
        ax.text(v * 1.35, y, txt, ha="left", va="center", fontsize=10, color=c)
ax.set_xscale("log")
ax.set_xlim(10, 3e5)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=TXT)
ax.set_xlabel("Annual energy, TWh (log scale; 1 TWh = 1 million MWh)", fontsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(true_twh, ys[1] + 0.5), xytext=(rep_twh, ys[1] + 0.5),
            arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2))
ax.text(np.sqrt(rep_twh * true_twh), ys[1] + 0.42, f"ratio {ratio:.1f}", ha="center", va="top",
        fontsize=10, color=SEC)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")

Abstract reports adverse-event rates of 17% and 10%; Results and Table 3 give 62% and 55%

Arthritis Rheumatol (2019) · doi:10.1002/art.41012 · PMC6900077

Summary

The abstract of this phase IIb/III trial of fasinumab in osteoarthritis pain states that treatment-emergent adverse event (TEAE) rates "were 17% with fasinumab and 10% with placebo". The Results text and Table 3 give the TEAE incidence for the 16-week treatment period as 62% and 55%, and Table 3's counts reproduce those figures exactly (209/337 = 62.0%; 45/82 = 54.9%). No period, organ class, population or definition in the paper yields 17% and 10% together, while the discontinuation figures in the same abstract sentence do match the body. The abstract's only overall tolerability figure therefore understates the absolute adverse-event burden by about 45 percentage points in each arm and overstates the relative excess (1.7-fold implied vs 1.13-fold in the data).

What the paper reports

Abstract, Results:

Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment.

Patients and Methods, Safety end points:

Safety was evaluated based on the frequency of treatment-emergent adverse events (TEAEs), AEs of special interest (adjudicated arthropathy and sympathetic nervous system dysfunction), and laboratory tests.

Results, Safety, paragraph 2:

During the 16-week treatment period, the incidence of TEAEs was 62% with fasinumab and 55% with placebo (Table 3). Nervous system and musculoskeletal symptoms were more frequent following treatment with fasinumab (17% and 19%, respectively) than with placebo (9% and 17%, respectively).

Results, Safety, paragraph 3:

The incidence of serious TEAEs during treatment was low (2% with placebo versus 1% with fasinumab).

A low proportion of patients discontinued therapy due to TEAEs (4% [n = 14] in the fasinumab group and 1% [n = 1] in the placebo group).

Results, Safety, paragraph 4:

During the 20-week follow-up, the incidence of TEAEs was higher in the combined fasinumab group than in the placebo group (48% versus 38%) (Table 3)

Table 3 ("TEAEs reported in >3% of patients during the treatment and follow-up periods, by system organ class (safety analysis set)"; columns: placebo n = 82 | 1 mg n = 85 | 3 mg n = 84 | 6 mg n = 85 | 9 mg n = 83 | combined n = 337), treatment period and follow-up period summary rows:

≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)

≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)

Results, Disposition of the patients:

During the study, 419 patients received ≥1 dose of study medication (1 patient randomized to receive placebo and 1 randomized to receive 9 mg fasinumab discontinued before drug administration).

Discussion, paragraph 5:

Fasinumab was generally well tolerated in the present study. Although the rate of TEAEs was higher with fasinumab than with placebo, there were few discontinuations attributable to TEAEs.

What the paper's own numbers show

Method. I took the counts and group sizes from Table 3 (safety analysis set), checked that the four dose columns add up to the combined column in every row of Table 3 (summary, organ-class and preferred-term rows, 42 in all), and recomputed every printed percentage in all six columns. I compared the abstract's 17%/10% with the treatment-period and follow-up-period rows and with every system-organ-class row in Table 3. I also listed which integer counts would print as 17% of 337 and 10% of 82, and checked the full-analysis-set denominators (338 and 83). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Any TEAE, fasinumab, treatment period Abstract, Results 17% 209/337 = 62.0%
Any TEAE, placebo, treatment period Abstract, Results 10% 45/82 = 54.9%
Any TEAE, fasinumab vs placebo Results, Safety ¶2 62% vs 55% 62.0% vs 54.9% (agrees)
≥1 TEAE, combined column Table 3 209 (62.0) 54 + 52 + 55 + 48 = 209 of 85 + 84 + 85 + 83 = 337; 62.0% (agrees)
Any AE, follow-up period Results, Safety ¶4; Table 3 48% vs 38%; 160 (47.5) vs 31 (37.8) 160/337 = 47.5%; 31/82 = 37.8% (agrees; not 17%/10%)
Discontinuation due to TEAE Abstract; Results, Safety ¶3 4% vs 1% (n = 14, n = 1) 14/337 = 4.2%; 1/82 = 1.2% (agrees)
Nervous system disorders, treatment period Results, Safety ¶2; Table 3 17% vs 9% 57/337 = 16.9%; 7/82 = 8.5%
Vascular disorders, treatment period Results, Safety ¶3; Table 3 6% vs 10% 19/337 = 5.6%; 8/82 = 9.8%
Serious TEAEs, treatment period Results, Safety ¶3 1% vs 2% no counts given; not 17%/10%
Counts that would print as 17% and 10% derived — 56–58 of 337 fasinumab patients and exactly 8 of 82 placebo patients
Table 3 rows giving 17% and 10% together Table 3 (all 42 rows, both periods) — none
Any TEAE with full-analysis-set denominators Disposition (338, 83) — 209/338 = 61.8%; 45/83 = 54.2%
Abstract minus Table 3, treatment period derived — −45.0 pp (fasinumab), −44.9 pp (placebo)
Table 3 rate ÷ abstract rate derived — 3.65 (fasinumab), 5.49 (placebo)
Risk ratio, fasinumab vs placebo derived 17/10 = 1.70 implied (209/337)/(45/82) = 1.130
Absolute difference derived 7 pp implied 7.1 pp
Fisher exact test, 209/337 vs 45/82 derived — OR 1.34, two-sided p = 0.26

Table 3 is internally consistent: in all 42 rows (summary, organ-class and preferred-term) the dose columns sum to the combined column, and all 252 printed percentages, in every dose column, match their counts. The Results text agrees with Table 3. Only the abstract's TEAE rates differ.

Figure

The abstract's TEAE rates set beside the Results text and Table 3

Figure 1. The conflicting statements, side by side. Left: the abstract's sentence, with the TEAE rates it prints in orange. Right: the Results (Safety, paragraph 2) sentence on the same quantity and the Table 3 "≥1 TEAE" counts, with the percentages recomputed from those counts in blue (209/337 = 62.0%, 45/82 = 54.9%). Both describe any TEAE in the 16-week treatment period. The discontinuation figures in the same abstract sentence (4% and 1%) agree with the body (14/337 = 4.2%, 1/82 = 1.2%).

Every Table 3 row plotted against the abstract's pair

Figure 2. Every summary row and system-organ-class row of Table 3, plotted as the placebo rate (x) against the combined fasinumab rate (y), both recomputed from the counts (blue; filled for the any-adverse-event rows, hollow for organ classes; circles for the treatment period, squares for follow-up; preferred-term rows are omitted, as none exceeds 11% in either arm). The orange cross is the abstract's pair, 17% vs 10%. No row lies at that point: the nervous-system row matches the fasinumab figure but not the placebo one (16.9% vs 8.5%), and the vascular row matches the placebo figure but not the fasinumab one (5.6% vs 9.8%). The gray diagonal marks equal rates in the two arms.

Why it matters

The abstract sentence is the abstract's only statement of overall tolerability, and readers who use the abstract alone (indexing services, evidence summaries, secondary coverage) receive the wrong figure.

  1. Absolute burden. The abstract implies that about 1 in 6 fasinumab-treated patients and 1 in 10 placebo patients had a TEAE. Table 3 shows about 3 in 5 and 1 in 2 (62.0% and 54.9%). The abstract understates both arms by about 45 percentage points.
  2. Relative excess. The abstract's figures imply a 1.7-fold risk with fasinumab (a 70% relative increase). The data give a 1.13-fold risk (OR 1.34; Fisher exact p = 0.26). The absolute difference is about 7 points either way (7 pp as stated, 7.1 pp recomputed). So the abstract makes the overall burden look smaller and the difference between arms look proportionally larger than the data show.
  3. What is unaffected. The direction is the same in both versions: TEAEs were more frequent with fasinumab. The discontinuation figures (4% vs 1%), the arthropathy figures, the serious-TEAE statement in the body and all efficacy results are not affected. The body's statements that fasinumab was "generally well tolerated" and that "the rate of TEAEs was higher with fasinumab than with placebo" are worded consistently with Table 3. The abstract's Conclusion ("The observed benefit-to-risk relationship favors further clinical development to explore the lowest doses") rests mainly on the efficacy and arthropathy results, which are correct, but a reader weighing that conclusion from the abstract alone would be weighing it against an adverse-event rate of 17% rather than 62%.

Explanations considered

Status

Suggested correction

  1. In the Abstract (Results), replace "Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo" with "Treatment-emergent adverse event rates during the 16-week treatment period were 62% with fasinumab and 55% with placebo", matching the Results text and Table 3 (209/337 and 45/82). The discontinuation figures in the same sentence (4% and 1%) are correct and can stay.
  2. If the 17% and 10% were intended to describe a different quantity, name it in the abstract and report it, with counts, in the Results or Table 3, since no such quantity currently appears in the paper.
  3. Ask indexing services (PubMed, Europe PMC) to carry the corrected abstract.
Recomputation script (python recompute.py.txt)
"""Recompute the adverse-event rates of PMC6900077
(Arthritis Rheumatol 2019, doi:10.1002/art.41012).

Run:  python recompute.py.txt                 (numbers + fig1/fig2)
      python recompute.py.txt --candidates    (also the five review candidates)
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
  fig1.svg/png, fig2.svg/png      (the figures used in writeup.md)
  candidates/cand1..cand5.svg/png (five candidate figure designs for internal review; written
                                  only with --candidates; not part of the journal package)

Inputs are copied from the paper as archived in PMC:
  Table 3 (safety analysis set): every row (summary, system organ class and preferred term),
    all six columns, counts and printed percentages, as TABLE3_TEXT below
  Results, Safety paragraph 3: discontinuations due to TEAEs (n = 14 fasinumab, n = 1 placebo)
  Results, Disposition: randomized n = 338 fasinumab, 83 placebo (full analysis set)
  Abstract, Results: "Treatment-emergent adverse event rates were 17% with fasinumab and
  10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment."
"""
import os
import re
import sys

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402
from scipy.stats import fisher_exact  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
WRITE_CANDIDATES = "--candidates" in sys.argv[1:]

# ------------------------------------------------------------------ inputs (from the paper)
N_PBO, N_FAS = 82, 337                                # Table 3 header, safety analysis set
DOSES = {"1 mg": 85, "3 mg": 84, "6 mg": 85, "9 mg": 83}
N_RAND_PBO, N_RAND_FAS = 83, 338                      # Results, Disposition (randomized)
ABSTRACT = {"teae_fas": 17, "teae_pbo": 10, "disc_fas": 4, "disc_pbo": 1}   # Abstract, %
TEXT = {"teae_fas": 62, "teae_pbo": 55, "fu_fas": 48, "fu_pbo": 38}          # Results text, %
DISC = {"fas": 14, "pbo": 1}                                                 # Results, Safety paragraph 3
SERIOUS_TEXT = {"fas": 1, "pbo": 2}                   # Results, Safety paragraph 3, % (no counts given)

# Table 3, every row as printed: period (T = treatment, F = follow-up) | row | placebo |
# 1 mg | 3 mg | 6 mg | 9 mg | combined; each cell "n (%)", or "0".
TABLE3_TEXT = """\
T | ≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)
T | Infections and infestations | 13 (15.9) | 18 (21.2) | 17 (20.2) | 16 (18.8) | 21 (25.3) | 72 (21.4)
T | Upper respiratory tract infection | 1 (1.2) | 5 (5.9) | 3 (3.6) | 3 (3.5) | 7 (8.4) | 18 (5.3)
T | Urinary tract infection | 3 (3.7) | 5 (5.9) | 1 (1.2) | 4 (4.7) | 3 (3.6) | 13 (3.9)
T | Sinusitis | 3 (3.7) | 1 (1.2) | 2 (2.4) | 2 (2.4) | 4 (4.8) | 9 (2.7)
T | Musculoskeletal and connective tissue disorders | 14 (17.1) | 16 (18.8) | 15 (17.9) | 18 (21.2) | 14 (16.9) | 63 (18.7)
T | Arthralgia | 2 (2.4) | 9 (10.6) | 5 (6.0) | 8 (9.4) | 5 (6.0) | 27 (8.0)
T | Back pain | 2 (2.4) | 4 (4.7) | 1 (1.2) | 2 (2.4) | 3 (3.6) | 10 (3.0)
T | Joint swelling | 0 | 1 (1.2) | 6 (7.1) | 3 (3.5) | 0 | 10 (3.0)
T | Pain in extremity | 3 (3.7) | 3 (3.5) | 1 (1.2) | 2 (2.4) | 4 (4.8) | 10 (3.0)
T | Musculoskeletal pain | 4 (4.9) | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (1.2)
T | Myalgia | 0 | 1 (1.2) | 0 | 2 (2.4) | 0 | 3 (0.9)
T | Osteoarthritis | 0 | 0 | 0 | 0 | 3 (3.6) | 3 (0.9)
T | Nervous system disorders | 7 (8.5) | 15 (17.6) | 14 (16.7) | 15 (17.6) | 13 (15.7) | 57 (16.9)
T | Headache | 5 (6.1) | 7 (8.2) | 2 (2.4) | 4 (4.7) | 4 (4.8) | 17 (5.0)
T | Paresthesia | 0 | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
T | Dizziness | 2 (2.4) | 3 (3.5) | 2 (2.4) | 3 (3.5) | 1 (1.2) | 9 (2.7)
T | Hypoesthesia | 1 (1.2) | 2 (2.4) | 3 (3.6) | 2 (2.4) | 1 (1.2) | 8 (2.4)
T | Gastrointestinal disorders | 7 (8.5) | 9 (10.6) | 12 (14.3) | 9 (10.6) | 5 (6.0) | 35 (10.4)
T | Nausea | 3 (3.7) | 6 (7.1) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 10 (3.0)
T | Diarrhea | 3 (3.7) | 1 (1.2) | 1 (1.2) | 4 (4.7) | 2 (2.4) | 8 (2.4)
T | Dry mouth | 1 (1.2) | 3 (3.5) | 4 (4.8) | 1 (1.2) | 0 | 8 (2.4)
T | Vomiting | 0 | 1 (1.2) | 3 (3.6) | 1 (1.2) | 2 (2.4) | 7 (2.1)
T | Vascular disorders | 8 (9.8) | 6 (7.1) | 4 (4.8) | 5 (5.9) | 4 (4.8) | 19 (5.6)
T | Orthostatic hypotension | 3 (3.7) | 3 (3.5) | 3 (3.6) | 3 (3.5) | 2 (2.4) | 11 (3.3)
T | Hypertension | 4 (4.9) | 1 (1.2) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 5 (1.5)
T | Skin and subcutaneous tissue disorders | 1 (1.2) | 6 (7.1) | 4 (4.8) | 2 (2.4) | 5 (6.0) | 17 (5.0)
T | Rash | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (4.8) | 8 (2.4)
F | ≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)
F | Musculoskeletal and connective tissue disorders | 10 (12.2) | 15 (17.6) | 20 (23.8) | 25 (29.4) | 23 (27.7) | 83 (24.6)
F | Arthralgia | 5 (6.1) | 3 (3.5) | 11 (13.1) | 12 (14.1) | 9 (10.8) | 35 (10.4)
F | Rapidly progressive OA | 0 | 2 (2.4) | 2 (2.4) | 5 (5.9) | 7 (8.4) | 16 (4.7)
F | OA | 0 | 3 (3.5) | 4 (4.8) | 3 (3.5) | 4 (4.8) | 14 (4.2)
F | Musculoskeletal pain | 2 (2.4) | 2 (2.4) | 5 (6.0) | 4 (4.7) | 1 (1.2) | 12 (3.6)
F | Joint swelling | 1 (1.2) | 1 (1.2) | 2 (2.4) | 4 (4.7) | 1 (1.2) | 8 (2.4)
F | Pain in extremity | 1 (1.2) | 1 (1.2) | 1 (1.2) | 3 (3.5) | 2 (2.4) | 7 (2.1)
F | Infections and infestations | 7 (8.5) | 11 (12.9) | 15 (17.9) | 8 (9.4) | 12 (14.5) | 46 (13.6)
F | Upper respiratory tract infection | 0 | 2 (2.4) | 7 (8.3) | 3 (3.5) | 0 | 12 (3.6)
F | Urinary tract infection | 2 (2.4) | 3 (3.5) | 2 (2.4) | 0 | 1 (1.2) | 6 (1.8)
F | Bronchitis | 1 (1.2) | 0 | 1 (1.2) | 0 | 3 (3.6) | 4 (1.2)
F | Vascular disorders | 4 (4.9) | 2 (2.4) | 5 (6.0) | 3 (3.5) | 5 (6.0) | 15 (4.5)
F | Orthostatic hypotension | 4 (4.9) | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
"""


def parse_table3(text):
    out = []
    for line in text.strip().splitlines():
        c = [x.strip() for x in line.split("|")]
        cells = []
        for x in c[2:]:
            m = re.fullmatch(r"(\d+)(?: \(([\d.]+)\))?", x)
            n = int(m.group(1))
            cells.append((n, float(m.group(2)) if m.group(2) else (0.0 if n == 0 else None)))
        out.append((c[0], c[1], cells))
    return out


TABLE3 = parse_table3(TABLE3_TEXT)
COL_N = [N_PBO] + list(DOSES.values()) + [N_FAS]
COL_NAMES = ["placebo", "1 mg", "3 mg", "6 mg", "9 mg", "combined"]

# Summary and system-organ-class rows used in the figures (subset of TABLE3, cross-checked below)
# (placebo n, [1, 3, 6, 9 mg n], combined n, printed placebo %, printed combined %)
TREAT = {
    ">=1 TEAE": (45, [54, 52, 55, 48], 209, 54.9, 62.0),
    "Infections and infestations": (13, [18, 17, 16, 21], 72, 15.9, 21.4),
    "Musculoskeletal and connective tissue": (14, [16, 15, 18, 14], 63, 17.1, 18.7),
    "Nervous system": (7, [15, 14, 15, 13], 57, 8.5, 16.9),
    "Gastrointestinal": (7, [9, 12, 9, 5], 35, 8.5, 10.4),
    "Vascular": (8, [6, 4, 5, 4], 19, 9.8, 5.6),
    "Skin and subcutaneous tissue": (1, [6, 4, 2, 5], 17, 1.2, 5.0),
}
FOLLOW = {
    ">=1 posttreatment AE": (31, [36, 38, 42, 44], 160, 37.8, 47.5),
    "Musculoskeletal and connective tissue": (10, [15, 20, 25, 23], 83, 12.2, 24.6),
    "Infections and infestations": (7, [11, 15, 8, 12], 46, 8.5, 13.6),
    "Vascular": (4, [2, 5, 3, 5], 15, 4.9, 4.5),
}


def pct(a, b):
    return 100.0 * a / b


def r1(x):
    return round(x + 1e-9, 1)


# ------------------------------------------------------------------ 1. table arithmetic
print("=== 1. Table 3 arithmetic (safety analysis set) ===")
assert sum(DOSES.values()) == N_FAS
print(f"dose-group sizes {list(DOSES.values())} sum to {sum(DOSES.values())} = combined n {N_FAS}")

# 1a. every Table 3 row, every column
n_rows = len(TABLE3)
n_cells = sum(len(c) for _, _, c in TABLE3)
sum_bad = [(per, k) for per, k, c in TABLE3 if sum(n for n, _ in c[1:5]) != c[5][0]]
pct_bad = [(per, k, COL_NAMES[j], n, p, r1(pct(n, COL_N[j])))
           for per, k, c in TABLE3 for j, (n, p) in enumerate(c) if p is None or r1(pct(n, COL_N[j])) != p]
print(f"all Table 3 rows: {n_rows} rows ({sum(1 for r in TABLE3 if r[0] == 'T')} treatment, "
      f"{sum(1 for r in TABLE3 if r[0] == 'F')} follow-up), {n_cells} cells checked")
print(f"  rows where 1 + 3 + 6 + 9 mg counts != combined count: {sum_bad if sum_bad else 'none'}")
print(f"  cells (any column) where printed % != count/n rounded to 0.1: {pct_bad if pct_bad else 'none'}")
for label, rows in (("treatment period", TREAT), ("follow-up period", FOLLOW)):
    print(f"\n[{label}]")
    print(f"  {'row':40s} {'pbo n':>5s} {'pbo %':>6s} {'fas n':>5s} {'fas %':>6s}  doses sum  printed % match")
    for k, (p, ds, c, pp, pc) in rows.items():
        ok_sum = sum(ds) == c
        ok_pct = r1(pct(p, N_PBO)) == pp and r1(pct(c, N_FAS)) == pc
        print(f"  {k:40s} {p:5d} {pct(p, N_PBO):6.1f} {c:5d} {pct(c, N_FAS):6.1f}  {str(ok_sum):9s} {ok_pct}")

# cross-check the figure subset against the full table
lookup = {(per, k): c for per, k, c in TABLE3}
for per, rows in (("T", TREAT), ("F", FOLLOW)):
    for k, (p, ds, c, pp, pc) in rows.items():
        full = [key for key in lookup if key[0] == per and key[1].replace("≥", ">=").startswith(k)]
        assert len(full) == 1, (per, k, full)
        cells = lookup[full[0]]
        assert [n for n, _ in cells] == [p] + ds + [c] and cells[0][1] == pp and cells[5][1] == pc, k
print("\nsummary/organ-class subset used for the figures matches the full table")

teae_f, teae_p = pct(209, N_FAS), pct(45, N_PBO)
fu_f, fu_p = pct(160, N_FAS), pct(31, N_PBO)
print("\n=== 2. Any TEAE: abstract vs body ===")
print(f"treatment period, fasinumab: 209/337 = {teae_f:.1f}%  (Results text {TEXT['teae_fas']}%, abstract {ABSTRACT['teae_fas']}%)")
print(f"treatment period, placebo:   45/82  = {teae_p:.1f}%  (Results text {TEXT['teae_pbo']}%, abstract {ABSTRACT['teae_pbo']}%)")
print(f"follow-up period, fasinumab: 160/337 = {fu_f:.1f}%  (Results text {TEXT['fu_fas']}%)")
print(f"follow-up period, placebo:   31/82  = {fu_p:.1f}%  (Results text {TEXT['fu_pbo']}%)")
print(f"gap, abstract vs Table 3 (treatment): fasinumab {teae_f - 17:.1f} pp, placebo {teae_p - 10:.1f} pp")
print(f"Table 3 / abstract: fasinumab {teae_f / 17:.2f}x, placebo {teae_p / 10:.2f}x")
print(f"abstract implies risk ratio 17/10 = {17 / 10:.2f}; Table 3 gives ({209}/{N_FAS})/({45}/{N_PBO}) = {teae_f / teae_p:.3f}")
print(f"absolute difference: abstract {17 - 10} pp; Table 3 {teae_f - teae_p:.1f} pp")
odds, p = fisher_exact([[209, N_FAS - 209], [45, N_PBO - 45]])
print(f"Fisher exact test, 209/337 vs 45/82: OR = {odds:.2f}, two-sided p = {p:.2f}")
print(f"full analysis set denominators: 209/338 = {pct(209, N_RAND_FAS):.1f}%, 45/83 = {pct(45, N_RAND_PBO):.1f}%")

print("\n=== 3. Discontinuations due to TEAEs (same abstract sentence) ===")
print(f"fasinumab 14/337 = {pct(14, N_FAS):.1f}% (abstract 4%); placebo 1/82 = {pct(1, N_PBO):.1f}% (abstract 1%)")

print("\n=== 4. Which counts would print as 17% and 10%? ===")
f17 = [n for n in range(N_FAS + 1) if round(pct(n, N_FAS)) == 17]
p10 = [n for n in range(N_PBO + 1) if round(pct(n, N_PBO)) == 10]
print(f"fasinumab counts rounding to 17% of 337: {f17[0]}-{f17[-1]} "
      f"({pct(f17[0], N_FAS):.1f}%-{pct(f17[-1], N_FAS):.1f}%)")
print(f"placebo counts rounding to 10% of 82: {p10} ({pct(p10[0], N_PBO):.1f}%)")
allrows = [("treatment", k, v) for k, v in TREAT.items()] + [("follow-up", k, v) for k, v in FOLLOW.items()]
hits = [(per, k) for per, k, (p_, _, c, _, _) in allrows
        if round(pct(c, N_FAS)) == 17 and round(pct(p_, N_PBO)) == 10]
print(f"Table 3 summary/organ-class rows with fasinumab 17% AND placebo 10%: {hits if hits else 'none'}")
hits_all = [(per, k) for per, k, c in TABLE3
            if round(pct(c[5][0], N_FAS)) == 17 and round(pct(c[0][0], N_PBO)) == 10]
print(f"any Table 3 row (incl. preferred terms) with fasinumab 17% AND placebo 10%: "
      f"{hits_all if hits_all else 'none'}")
for per, k, (p_, _, c, _, _) in allrows:
    if round(pct(c, N_FAS)) == 17 or round(pct(p_, N_PBO)) == 10:
        print(f"  partial match ({per}) {k}: fasinumab {pct(c, N_FAS):.1f}%, placebo {pct(p_, N_PBO):.1f}%")
print(f"serious TEAEs during treatment (Results text): fasinumab {SERIOUS_TEXT['fas']}%, placebo {SERIOUS_TEXT['pbo']}%")

# ------------------------------------------------------------------ figure style
BG, TXT, SEC, ORANGE, BLUE = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6"
GRID, PANEL = "#e4e3df", "#f3f2ef"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
    "axes.edgecolor": "#bdbcb8", "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def clean(ax, grid_axis="x"):
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.grid(axis=grid_axis, color=GRID, linewidth=0.8)
    ax.set_axisbelow(True)


def panel(ax, x, y, w, h, title, lines):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc=PANEL, ec="#cfcdc8", lw=1))
    ax.text(x + 0.02, y + h - 0.035, title, ha="left", va="top", fontsize=11, weight="bold")
    yy = y + h - 0.13
    for ln, col, size in lines:
        ax.text(x + 0.02, yy, ln, ha="left", va="top", fontsize=size, color=col)
        yy -= 0.068 * (ln.count("\n") + 1) + 0.02


# ---- candidate 1: two text panels joined by an annotated connector (abstract vs body)
def cand_text_panels():
    fig = plt.figure(figsize=(W, 3.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    panel(ax, 0.02, 0.17, 0.40, 0.78, "Abstract, Results", [
        ("“Treatment-emergent adverse\nevent rates were", TXT, 10.5),
        ("17% with fasinumab and\n10% with placebo”", ORANGE, 11.5),
        ("Same sentence: discontinuations\n4% and 1%", SEC, 10),
    ])
    panel(ax, 0.58, 0.17, 0.40, 0.78, "Results, Safety ¶2 and Table 3", [
        ("“the incidence of TEAEs was\n62% with fasinumab and\n55% with placebo (Table 3)”", TXT, 10.5),
        ("Table 3, ≥1 TEAE:\n209/337 = 62.0% (fasinumab)\n45/82 = 54.9% (placebo)", BLUE, 11),
        ("Discontinuations 14/337 = 4.2%\nand 1/82 = 1.2%", SEC, 10),
    ])
    ax.annotate("", xy=(0.575, 0.62), xytext=(0.425, 0.62),
                arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
    ax.text(0.5, 0.66, "same quantity,\nsame period", ha="center", va="bottom", fontsize=10, color=SEC)
    ax.text(0.5, 0.58, "17 vs 62\n10 vs 55", ha="center", va="top", fontsize=11, weight="bold")
    ax.annotate("", xy=(0.575, 0.31), xytext=(0.425, 0.31),
                arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2, ls="--"))
    ax.text(0.5, 0.33, "agree", ha="center", va="bottom", fontsize=10, color=SEC)
    ax.text(0.02, 0.07, "Orange: as printed in the abstract.  Blue: recomputed from Table 3 counts\n"
            "(any TEAE, 16-week treatment period, safety analysis set).", ha="left", va="center",
            fontsize=10, color=SEC)
    return fig


# ---- candidate 2: dot plot, abstract vs recomputed, on one 0-100% axis
def cand_dot_plot():
    fig, ax = plt.subplots(figsize=(W, 3.3))
    rows = [("Any TEAE, fasinumab", 17, teae_f), ("Any TEAE, placebo", 10, teae_p),
            ("Discontinued for TEAE, fasinumab", 4, pct(14, N_FAS)),
            ("Discontinued for TEAE, placebo", 1, pct(1, N_PBO))]
    for i, (lab, rep, rec) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.plot([rep, rec], [y, y], color="#bdbcb8", lw=2, zorder=1)
        ax.scatter(rep, y, s=210, fc="none", ec=ORANGE, lw=2.5, zorder=4,
                   label="Abstract (as printed)" if i == 0 else None)
        ax.scatter(rec, y, s=90, color=BLUE, zorder=3, ec=BG, lw=1.5,
                   label="Table 3 / Results counts, recomputed" if i == 0 else None)
        if abs(rep - rec) > 5:
            ax.text(rep, y + 0.22, f"{rep}%", ha="center", va="bottom", fontsize=10, color=TXT)
            ax.text(rec, y + 0.22, f"{rec:.1f}%", ha="center", va="bottom", fontsize=10, color=TXT)
        else:
            ax.text(max(rep, rec) + 4, y, f"{rep}% vs {rec:.1f}% (agree)", ha="left", va="center",
                    fontsize=10, color=SEC)
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], color=TXT)
    ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.25)
    ax.set_xlabel("Patients (%), 16-week treatment period, safety analysis set")
    clean(ax)
    fig.legend(*ax.get_legend_handles_labels(), loc="upper center", ncol=2, frameon=False, fontsize=10)
    fig.tight_layout(rect=(0, 0, 1, 0.9))
    return fig


# ---- candidate 3: every Table 3 row as a point; the abstract's pair falls on none
def cand_search_scatter():
    fig, ax = plt.subplots(figsize=(W, 5.4))
    for per, rows, mk in (("treatment", TREAT, "o"), ("follow-up", FOLLOW, "s")):
        for k, (p_, _, c, _, _) in rows.items():
            overall = k.startswith(">=1")
            ax.scatter(pct(p_, N_PBO), pct(c, N_FAS), s=70 if overall else 55, marker=mk,
                       fc=BLUE if overall else BG, ec=BLUE, lw=2, zorder=3)
    ax.scatter(10, 17, s=90, marker="X", color=ORANGE, zorder=4, ec=BG, lw=1)
    callouts = [("Nervous system", "nervous system, treatment:\n", (1.0, 34), (-0.2, 0.9)),
                ("Vascular", "vascular, treatment: ", (24, 5), (0.6, 0.0))]
    for k, name, (tx, ty), (dx, dy) in callouts:
        p_, _, c, _, _ = TREAT[k]
        x0, y0 = pct(p_, N_PBO), pct(c, N_FAS)
        ax.annotate(f"{name}{y0:.1f}% vs {x0:.1f}%", xy=(x0 + dx, y0 + dy), xytext=(tx, ty),
                    fontsize=10, color=SEC, va="center",
                    arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=4))
    ax.text(teae_p - 1.5, teae_f, f"any TEAE, treatment\n{teae_f:.1f}% vs {teae_p:.1f}%",
            fontsize=10, color=TXT, va="center", ha="right")
    ax.text(fu_p - 1.5, fu_f, f"any AE, follow-up\n{fu_f:.1f}% vs {fu_p:.1f}%",
            fontsize=10, color=TXT, va="center", ha="right")
    ax.annotate("Abstract: 17% vs 10%\n(no Table 3 row is at this point)",
                xy=(10.6, 17.6), xytext=(22, 31), fontsize=10, color=TXT, va="center",
                bbox=dict(fc=BG, ec="none", pad=1.5),
                arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=2))
    ax.plot([0, 70], [0, 70], color="#bdbcb8", lw=1, zorder=0)
    ax.text(60, 55, "equal rates", fontsize=10, color=SEC)
    ax.set_xlim(0, 70); ax.set_ylim(0, 70)
    ax.set_xlabel("Placebo, % of 82 patients"); ax.set_ylabel("Fasinumab combined, % of 337 patients")
    clean(ax, "both")
    from matplotlib.lines import Line2D
    h = [Line2D([], [], marker="X", ls="", color=ORANGE, ms=10, label="Abstract (as printed)"),
         Line2D([], [], marker="o", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any TEAE, treatment (recomputed)"),
         Line2D([], [], marker="s", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any AE, follow-up (recomputed)"),
         Line2D([], [], marker="o", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, treatment"),
         Line2D([], [], marker="s", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, follow-up")]
    fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.005), ncol=2, frameon=False,
               fontsize=10, columnspacing=2.0, handletextpad=0.4)
    fig.subplots_adjust(left=0.10, right=0.97, top=0.97, bottom=0.28)
    return fig


# ---- candidate 4: bars from zero, abstract vs treatment period vs follow-up
def cand_bars():
    fig, ax = plt.subplots(figsize=(W, 3.4))
    groups = [("Fasinumab (n = 337)", 17, teae_f, fu_f), ("Placebo (n = 82)", 10, teae_p, fu_p)]
    h = 0.24
    for i, (g, rep, tr, fu) in enumerate(groups):
        y = 1 - i
        ax.barh(y + h, rep, height=h - 0.03, color=ORANGE, label="Abstract: any TEAE (as printed)" if i == 0 else None)
        ax.barh(y, tr, height=h - 0.03, color=BLUE, label="Table 3: ≥1 TEAE, treatment period" if i == 0 else None)
        ax.barh(y - h, fu, height=h - 0.03, color=BG, ec=BLUE, lw=1.5, hatch="///",
                label="Table 3: ≥1 AE, follow-up period" if i == 0 else None)
        for yy, v, fmt in ((y + h, rep, f"{rep}%"), (y, tr, f"{tr:.1f}%"), (y - h, fu, f"{fu:.1f}%")):
            ax.text(v + 1, yy, fmt, va="center", fontsize=10, color=TXT)
    ax.set_yticks([1, 0]); ax.set_yticklabels([g[0] for g in groups], color=TXT)
    ax.set_xlim(0, 100); ax.set_ylim(-0.55, 1.75)
    ax.set_xlabel("Patients with at least one adverse event (%)")
    clean(ax)
    ax.legend(loc="upper right", frameon=False, fontsize=10)
    fig.tight_layout()
    return fig


# ---- candidate 5: calculation flow from Table 3 counts to the printed figures
def cand_flow():
    fig = plt.figure(figsize=(W, 3.3))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    xs = [0.01, 0.29, 0.54, 0.79]; ws = [0.21, 0.17, 0.17, 0.20]
    heads = ["Table 3 counts", "Recomputed", "Results text", "Abstract"]
    for x, w, t in zip(xs, ws, heads):
        ax.text(x + w / 2, 0.93, t, ha="center", va="center", fontsize=11, weight="bold")
    rows = [(0.62, ["54+52+55+48\n= 209 of 337", f"209/337\n= {teae_f:.1f}%",
                    "“62% with\nfasinumab”", "“17% with\nfasinumab”"]),
            (0.27, ["45 of 82\n(placebo)", f"45/82\n= {teae_p:.1f}%", "“55% with\nplacebo”",
                    "“10% with\nplacebo”"])]
    for yc, cells in rows:
        for j, (x, w, s) in enumerate(zip(xs, ws, cells)):
            ec = ORANGE if j == 3 else (BLUE if j == 1 else "#cfcdc8")
            ax.add_patch(FancyBboxPatch((x, yc - 0.13), w, 0.26, boxstyle="round,pad=0,rounding_size=0.012",
                                        fc=PANEL, ec=ec, lw=2 if j in (1, 3) else 1))
            ax.text(x + w / 2, yc, s, ha="center", va="center", fontsize=10.5,
                    color=TXT)
        for j in range(3):
            x0 = xs[j] + ws[j] + 0.005; x1 = xs[j + 1] - 0.005
            ok = j < 2
            ax.annotate("", xy=(x1, yc), xytext=(x0, yc),
                        arrowprops=dict(arrowstyle="->", color=SEC if ok else ORANGE, lw=2,
                                        ls="-" if ok else "--"))
            ax.text((x0 + x1) / 2, yc + 0.04, "=" if ok else "\u2260", ha="center", va="bottom",
                    fontsize=14, color=SEC if ok else TXT, weight="bold")
    ax.text(0.01, 0.05, "Blue outline: recomputed from the paper's counts.  Orange outline: abstract as printed.\n"
            "= agrees after rounding;  \u2260 disagrees.  Any TEAE, 16-week treatment period, safety analysis set.",
            fontsize=10, color=SEC, va="center")
    return fig


CANDIDATES = [("cand1_text_panels", cand_text_panels), ("cand2_dot_plot", cand_dot_plot),
              ("cand3_search_scatter", cand_search_scatter), ("cand4_bars", cand_bars),
              ("cand5_calc_flow", cand_flow)]
if WRITE_CANDIDATES:
    os.makedirs(CAND, exist_ok=True)
    for name, fn in CANDIDATES:
        save(fn(), os.path.join(CAND, name))
# Provisional choice for the write-up (to be swapped once a candidate is picked):
save(cand_text_panels(), os.path.join(HERE, "fig1"))
save(cand_search_scatter(), os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2" + (" and candidates/cand1..cand5" if WRITE_CANDIDATES else "") + " (.svg and .png)")

PIK3CA progression-free survival P = 0.016 cannot be derived from its reported HR and CI

BMC Med (2023) · doi:10.1186/s12916-023-03017-z · PMC10426213

Summary

This longitudinal ctDNA analysis of the phase Ib MONALEESASIA trial reports that patients with baseline PIK3CA alterations had shorter median progression-free survival (PFS) than patients without them, 12.7 vs 19.2 months, with P = 0.016. The Abstract gives this P value alongside the TP53 result and concludes that PIK3CA alterations "were associated with inferior outcomes". In the Results, the same P value is printed next to a hazard ratio of 0.93 (95% CI 0.52–1.69) in a sentence that calls the difference "not statistically significant". A Wald test back-calculated from that HR and CI gives z = −0.24 and P = 0.81, while the same check reproduces the paper's P values for TP53, bTMB and ctDNA detectability. The PIK3CA association in the Abstract therefore has no statistical support in the paper's own estimate. The paper's main findings on ctDNA monitoring do not depend on it.

What the paper reports

Abstract, Results:

Patients with alterations in PIK3CA and TP53 at baseline had shorter median progression-free survival compared with patients with wild-type PIK3CA and TP53, 12.7 and 7.3 months vs 19.2 and 19.4 months, respectively (P = .016 and P = .0001, respectively).

Abstract, Conclusions:

Higher ctDNA levels and PIK3CA and TP53 alterations detected at baseline were associated with inferior outcomes.

Results, "Gene alterations at baseline" (the PIK3CA and TP53 sentences, with the bTMB sentence that follows):

Median PFS of patients with TP53 mutations was significantly shorter than those without TP53 mutations (7.3 months vs 19.4 months; HR, 0.26; 95% CI, 0.12–0.53; P < 0.001; Fig. 7A). Median PFS of patients with PIK3CA mutations was 12.7 months vs 19.2 months for those without PIK3CA mutations; however, the difference was not statistically significant (HR, 0.93; 95% CI, 0.52–1.69; P = 0.016; Fig. 7B). Patients with undetectable ctDNA experienced a median PFS of 37.8 months (Fig. 7B).

In patients with detectable ctDNA, the median PFS of patients with high bTMB (> 10 mutations per megabase [mut/Mb]) was 12.7 months vs 19.4 months for those with low bTMB (< 10 mut/Mb); this numerically shorter PFS difference was not statistically significant (HR, 0.83; 95% CI, 0.46–1.5; P = 0.53; Fig. 7C).

Same paragraph, last sentence:

Therefore, in this particular cohort, neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction.

Results, "ctDNA fraction by BOR and PFS" (used below as a control comparison):

More generally, patients with no detectable ctDNA at baseline had a PFS almost twice that observed in patients with detectable ctDNA at baseline: 38.0 months vs 19.2 months, respectively (hazard ratio [HR], 0.45; 95% CI, 0.24–0.84; P = 0.01; Fig. 1C).

Methods, "Statistical analysis":

A Cox proportional-hazards model was used to evaluate differences in PFS in the MONALEESASIA cohorts by ctDNA detectable vs undetectable status and mutation status in PIK3CA, TP53.

Methods, "Sample collection and ctDNA analysis":

All comparisons between mutated vs nonmutated (e.g., TP53 mutated vs nonmutated) were performed on patients with detectable ctDNA.

Figure 7 caption:

Progression-free survival for ribociclib plus ET in patients with or without mutations. A TP53. B PIK3CA. C High versus low TMB. Analysis by TMB (high versus low) only included patients with detectable ctDNA.

Figure 7 itself is an image. In the publisher's full-resolution file (12916_2023_3017_Fig7_HTML.png), panel B prints "P=. 016", panel A "P=.0001" and panel C "P=.53". Panels A and B each draw three Kaplan–Meier curves (mutated, nonmutated and "not detected/low power"); panel C draws two. The numbers at risk at time 0 are 12, 42 and 28 in panel A, 24, 30 and 28 in panel B, and 27 and 27 in panel C.

What the paper's own numbers show

Method. For a Cox model, a 95% Wald confidence interval is exp(b ± 1.96·SE), where b is the log hazard ratio. The standard error can therefore be recovered from the printed limits as SE = (ln upper − ln lower) / (2 × 1.96). The Wald statistic is z = ln(HR) / SE, and the two-sided P value is 2·(1 − Φ(|z|)). I applied this to all four PFS hazard ratios that the paper prints with a CI and a P value, using the three non-PIK3CA comparisons as controls for the method. I checked that each printed HR sits at the log-scale midpoint of its CI, that rounding of the printed values cannot move the result, and what HR or CI a P value of 0.016 would require. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
PIK3CA mutated vs nonmutated, P Results; Abstract; Fig. 7B HR 0.93 (95% CI 0.52–1.69); P = 0.016 SE(ln HR) = 0.301, z = −0.24, Wald P = 0.81
PIK3CA: rounding range of HR and CI limits derived HR 0.93, CI 0.52–1.69 over HR 0.925–0.935 and limits ±0.005: P = 0.79–0.83
PIK3CA: CI log-midpoint derived HR 0.93 0.937, so the HR and CI fit each other
PIK3CA: |z| that P = 0.016 implies derived P = 0.016 |z| = 2.41, ten times the 0.24 that the CI gives
PIK3CA: CI that P = 0.016 would need at HR 0.93 derived CI 0.52–1.69 (SE 0.301) SE 0.030, CI 0.877–0.987 (one tenth of the printed width)
PIK3CA: HR that P = 0.016 would need with the printed CI width derived HR 0.93 0.485 (CI 0.27–0.87) or 2.063 (CI 1.14–3.72)
TP53 mutated vs nonmutated, P (control) Results; Abstract; Fig. 7A HR 0.26 (0.12–0.53); P < 0.001 (Results), P = .0001 (Abstract, Fig. 7A) z = −3.55, Wald P = 0.00038: consistent with < 0.001
bTMB high vs low, P (control) Results; Fig. 7C HR 0.83 (0.46–1.5); P = 0.53 z = −0.62, Wald P = 0.54: consistent
ctDNA undetectable vs detectable, P (control) Results; Fig. 1C HR 0.45 (0.24–0.84); P = 0.01 z = −2.50, Wald P = 0.012: consistent
Numbers at risk at time 0 Fig. 7A / 7B / 7C 12 + 42 + 28; 24 + 30 + 28; 27 + 27 82, 82 and 54; the mutated vs nonmutated comparisons use the 54 patients with detectable ctDNA
Median PFS, undetectable ctDNA Results (Fig. 1C text vs Fig. 7B text) 38.0 months; 37.8 months differ by 0.2 months (minor, separate from the P value issue)

The method reproduces three of the four printed P values to within rounding. The PIK3CA value is the exception, by a factor of about 50 in P and 10 in z. The PIK3CA confidence interval contains 1, and an HR of 0.93 is close to no difference, so the Results sentence's own description ("not statistically significant") matches the HR and CI. It does not match the P value printed in the same parentheses.

Figure

Reported P values and Wald P values recomputed from each reported HR and 95% CI

Figure 1. For each of the four PFS comparisons that the paper reports with an HR, a 95% CI and a P value, the orange dot is the P value printed in the paper and the blue dot is the two-sided Wald P value recomputed from that HR and CI. The axis is logarithmic; the dashed line marks P = 0.05. For TP53 the orange dot is the Abstract and Figure 7A value (.0001); the Results give P < 0.001, which the recomputed 0.00038 satisfies. For bTMB and ctDNA detectability the two values coincide within rounding. For PIK3CA the printed P = 0.016 lies on the other side of 0.05 from the P = 0.81 that its own HR and CI imply.

What P = 0.016 would require for the PIK3CA comparison

Figure 2. The PIK3CA hazard ratio on a log scale, with HR = 1 as the reference line. Orange: the reported HR 0.93 with its 95% CI 0.52–1.69, which gives Wald P = 0.81. Blue: what P = 0.016 would require, either a CI one tenth as wide around the same HR (0.88–0.99), or, with the reported CI width, an HR of 0.48 or 2.06 whose interval excludes 1. Neither resembles the reported estimate, so a misprint in the HR alone or in one CI limit alone cannot produce the printed P value.

Why it matters

The PIK3CA result appears in the Abstract's Results with P = .016, paired with the TP53 result, and the Abstract's Conclusions list PIK3CA alterations among the baseline features "associated with inferior outcomes". A reader of the Abstract would take PIK3CA as a statistically supported adverse prognostic marker in this cohort. If the hazard ratio and confidence interval are correct, the comparison gives P ≈ 0.81 and an HR close to 1 with a wide interval (0.52–1.69). On that reading the data show no evidence of a PFS difference by PIK3CA status among patients with detectable ctDNA, which is also what the Results sentence and the paragraph's closing statement ("neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction") say.

The effect is limited to this result. The full-text Conclusions do not rely on PIK3CA, and the Discussion describes the PIK3CA effect as "numerically shorter". The paper's main findings are unaffected: the decline of ctDNA on treatment, the relation between baseline ctDNA and PFS (HR 0.45, P = 0.01, reproduced as 0.012), the lead time of ctDNA increase before radiological progression, and the TP53 association (reproduced as P = 0.00038). The Discussion sentence "Patients with alterations in these genes experienced a shorter median PFS compared with patients with wild-type versions" remains true as a statement about medians but has no statistical support for PIK3CA.

A smaller point of interpretation: the printed HRs for TP53 (0.26), PIK3CA (0.93) and bTMB (0.83) are all below 1 although the mutated or high-bTMB groups have the shorter medians. They therefore appear to express the nonmutated or low-bTMB group relative to the other (for PIK3CA, about 1.08, 95% CI 0.59–1.92, mutated vs nonmutated). The direction does not change any P value in this write-up.

Explanations considered

Status

Suggested correction

  1. Report the two-group Cox P value for PIK3CA mutated vs nonmutated among patients with detectable ctDNA, which the printed HR 0.93 (95% CI 0.52–1.69) implies is about 0.8, in the Results sentence and in the Abstract.
  2. Revise the Abstract's Results and Conclusions so that PIK3CA alterations are not presented as associated with inferior PFS on the basis of P = .016, for example by limiting the statistical association to TP53 and describing the PIK3CA difference in medians as not statistically significant.
  3. State in the Methods or the Figure 7 legend which test produced the P values printed on Figure 7A and 7B. If they are global tests across three groups including the ctDNA-undetectable patients, say so, and do not cite them as mutated vs wild-type comparisons.
  4. State the reference group of the hazard ratios (the HRs below 1 appear to compare nonmutated or low-bTMB patients with the others).
  5. Minor: reconcile the median PFS for patients with undetectable ctDNA (38.0 months in the Fig. 1C text, 37.8 months in the Fig. 7B text), or explain the difference.
Recomputation script (python recompute.py.txt)
"""Recompute the PFS P values of PMC10426213
(BMC Med 2023, doi:10.1186/s12916-023-03017-z, longitudinal ctDNA analysis of MONALEESASIA).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs to candidates/ (c1..c5, .svg and .png).

Inputs are copied from the paper as archived in PMC:
  Results, "ctDNA fraction by BOR and PFS": ctDNA undetectable vs detectable, HR 0.45 (0.24-0.84), P = 0.01 (Fig. 1C)
  Results, "Gene alterations at baseline":  TP53   HR 0.26 (0.12-0.53), P < 0.001 (Fig. 7A)
                                            PIK3CA HR 0.93 (0.52-1.69), P = 0.016 (Fig. 7B)
                                            bTMB   HR 0.83 (0.46-1.5),  P = 0.53  (Fig. 7C)
  Abstract: PIK3CA P = .016, TP53 P = .0001
  Figure 7 (publisher image, numbers at risk at t = 0): A 12 mut / 42 nonmut / 28 not detected;
                                                        B 24 mut / 30 nonmut / 28 not detected; C 27 / 27
Method: for a Cox model, a 95% Wald CI is exp(b +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.96),
z = ln(HR) / SE and the two-sided Wald P = 2 * (1 - Phi(|z|)).
"""
import math
import os

import numpy as np
from scipy import stats

import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.isf(0.025)

# ---------------------------------------------------------------- inputs (from the paper)
# key: (label, HR, lo, hi, reported P as a number for plotting, reported P as printed, location)
ROWS = [
    ("pik3ca", "PIK3CA mut vs nonmut", 0.93, 0.52, 1.69, 0.016, "P = 0.016 (Results); P = .016 (Abstract, Fig. 7B)", "Results; Abstract; Fig. 7B"),
    ("tp53", "TP53 mut vs nonmut", 0.26, 0.12, 0.53, 0.0001, "P < 0.001 (Results); P = .0001 (Abstract, Fig. 7A)", "Results; Abstract; Fig. 7A"),
    ("tmb", "bTMB high vs low", 0.83, 0.46, 1.50, 0.53, "P = 0.53 (Results); P = .53 (Fig. 7C)", "Results; Fig. 7C"),
    ("ctdna", "ctDNA undetectable vs detectable", 0.45, 0.24, 0.84, 0.01, "P = 0.01 (Results, Fig. 1C)", "Results; Fig. 1C"),
]
AT_RISK = {"Fig. 7A (TP53)": (12, 42, 28), "Fig. 7B (PIK3CA)": (24, 30, 28), "Fig. 7C (bTMB)": (27, 27)}
MEDIANS = {"PIK3CA mut": 12.7, "PIK3CA nonmut": 19.2, "TP53 mut": 7.3, "TP53 nonmut": 19.4,
           "undetectable (Fig. 7B text)": 37.8, "undetectable (Fig. 1C text)": 38.0, "detectable (Fig. 1C text)": 19.2}


def wald(hr, lo, hi):
    se = (math.log(hi) - math.log(lo)) / (2 * Z975)
    z = math.log(hr) / se
    return se, z, 2 * stats.norm.sf(abs(z))


R = {}
print("=== 1. Wald P back-calculated from each reported HR and 95% CI (two-sided) ===")
for key, lab, hr, lo, hi, prep, prep_txt, loc in ROWS:
    se, z, p = wald(hr, lo, hi)
    mid = math.exp((math.log(lo) + math.log(hi)) / 2)
    R[key] = dict(lab=lab, hr=hr, lo=lo, hi=hi, se=se, z=z, p=p, prep=prep, prep_txt=prep_txt,
                  zrep=stats.norm.isf(prep / 2))
    print(f"  {lab:34s} HR {hr} ({lo}-{hi}): SE(lnHR) = {se:.3f}, z = {z:+.2f}, P = {p:.2g} ({p:.4f})"
          f"   reported: {prep_txt}")
    print(f"  {'':34s} CI log-midpoint = {mid:.3f} (HR printed {hr}); |z| implied by reported P = {R[key]['zrep']:.2f}")

pk = R["pik3ca"]
print("\n=== 2. What P = 0.016 would require for PIK3CA ===")
z016 = stats.norm.isf(0.016 / 2)
se_req = abs(math.log(pk["hr"])) / z016
lo_req, hi_req = math.exp(math.log(pk["hr"]) - Z975 * se_req), math.exp(math.log(pk["hr"]) + Z975 * se_req)
hr_lo_req, hr_hi_req = math.exp(-z016 * pk["se"]), math.exp(z016 * pk["se"])
R["req"] = dict(z016=z016, se_req=se_req, lo_req=lo_req, hi_req=hi_req, hr_lo_req=hr_lo_req, hr_hi_req=hr_hi_req)
print(f"  |z| for two-sided P = 0.016: {z016:.2f}; reported CI gives |z| = {abs(pk['z']):.2f} "
      f"({z016 / abs(pk['z']):.1f} times smaller)")
print(f"  at HR 0.93, P = 0.016 needs SE(lnHR) = {se_req:.4f} (reported CI implies {pk['se']:.3f}, "
      f"{pk['se'] / se_req:.1f} times larger), i.e. a 95% CI of {lo_req:.3f}-{hi_req:.3f}")
for h in (hr_lo_req, hr_hi_req):
    print(f"  with the reported CI width, P = 0.016 needs HR = {h:.3f}, CI "
          f"{math.exp(math.log(h) - Z975 * pk['se']):.2f}-{math.exp(math.log(h) + Z975 * pk['se']):.2f}")
print(f"  reported CI 0.52-1.69 contains HR = 1: {pk['lo'] < 1 < pk['hi']}")

print("\n=== 3. Sensitivity to rounding of the printed HR and CI limits (PIK3CA) ===")
ps = []
for hr in np.arange(0.925, 0.9351, 0.005):
    for lo in np.arange(0.515, 0.5251, 0.005):
        for hi in np.arange(1.685, 1.6951, 0.005):
            ps.append(wald(hr, lo, hi)[2])
print(f"  HR 0.925-0.935, lo 0.515-0.525, hi 1.685-1.695: Wald P ranges {min(ps):.3f}-{max(ps):.3f}")

print("\n=== 4. A possible source of the figure P values: a 3-group test (rough approximation) ===")
chi_det = R["ctdna"]["z"] ** 2
chi_pk = R["pik3ca"]["z"] ** 2
chi_tp = R["tp53"]["z"] ** 2
need016, need0001 = stats.chi2.isf(0.016, 2), stats.chi2.isf(0.0001, 2)
p3_pk = stats.chi2.sf(chi_det + chi_pk, 2)
p3_tp = stats.chi2.sf(chi_det + chi_tp, 2)
R["three"] = dict(chi_det=chi_det, chi_pk=chi_pk, chi_tp=chi_tp, p3_pk=p3_pk, p3_tp=p3_tp,
                  need016=need016, need0001=need0001)
print(f"  Wald chi2 components: detectable vs undetectable {chi_det:.2f}; PIK3CA pairwise {chi_pk:.3f}; TP53 pairwise {chi_tp:.2f}")
print(f"  2-df chi2 needed for P = 0.016: {need016:.2f}; for P = 0.0001: {need0001:.2f}")
print(f"  PIK3CA panel (mut / nonmut / not detected): approx chi2 {chi_det + chi_pk:.2f}, 2-df P ~ {p3_pk:.3f}"
      f"  (printed .016; pairwise Wald {R['pik3ca']['p']:.2f})")
print(f"  TP53 panel (mut / nonmut / not detected):   approx chi2 {chi_det + chi_tp:.2f}, 2-df P ~ {p3_tp:.1g}"
      f"  (printed .0001; pairwise Wald {R['tp53']['p']:.5f})")
print("  (Approximation only: adds a 1-df detectable-vs-undetectable component to the 1-df pairwise one.)")

print("\n=== 5. Numbers at risk at t = 0 in Figure 7 ===")
for k, v in AT_RISK.items():
    print(f"  {k}: {' + '.join(map(str, v))} = {sum(v)}")
print(f"  detectable ctDNA in panels A/B: {24 + 30} = {12 + 42}; PIK3CA mut share of detectable {24 / 54:.1%}, of all 82 {24 / 82:.1%}")
print(f"  TP53 mut share of detectable {12 / 54:.1%}, of all 82 {12 / 82:.1%}")

print("\n=== 6. Other small observations ===")
print(f"  median PFS, undetectable ctDNA: {MEDIANS['undetectable (Fig. 1C text)']} months (Fig. 1C text) vs "
      f"{MEDIANS['undetectable (Fig. 7B text)']} months (Fig. 7B text)")
print("  The TP53, PIK3CA and bTMB HRs are < 1 although the mutated / high-bTMB group has the shorter median,")
print("  so they appear to express the nonmutated / low group relative to the other; the Wald P is unaffected.")
print(f"  e.g. 1/0.93 = {1 / 0.93:.2f} (1/1.69 = {1 / 1.69:.2f} to 1/0.52 = {1 / 0.52:.2f}); "
      f"1/0.26 = {1 / 0.26:.2f} (1/0.53 = {1 / 0.53:.2f} to 1/0.12 = {1 / 0.12:.2f})")

# ---------------------------------------------------------------- figure style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, NEUTRAL = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
    "lines.linewidth": 1.5, "xtick.labelsize": 10, "ytick.labelsize": 10.5,
})
LW = 1.5    # 1.5 pt = 2 px at 96 dpi
MS = 9      # marker size in points (>= 8 px)


def save(fig, folder, stem):
    fig.savefig(os.path.join(folder, stem + ".svg"))
    fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
    plt.close(fig)


def clean(ax, grid_axis="x"):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)


ORDER = ["pik3ca", "tp53", "tmb", "ctdna"]
YLAB = {"pik3ca": "PIK3CA mutated vs not\n(Fig. 7B)", "tp53": "TP53 mutated vs not\n(Fig. 7A)",
        "tmb": "bTMB high vs low\n(Fig. 7C)", "ctdna": "ctDNA undetectable vs\ndetectable (Fig. 1C)"}
PREP_SHORT = {"pik3ca": "0.016", "tp53": ".0001", "tmb": "0.53", "ctdna": "0.01"}


def fmtp(p):
    if p < 0.001:
        return f"{p:.5f}".rstrip("0")
    return f"{p:.2g}" if p < 0.1 else f"{p:.2f}"


# ---- C1: P-value ladder (reported P vs P recomputed from HR/CI, log axis) ----
REP_LABEL = {"pik3ca": "reported 0.016", "tp53": "reported .0001 (Abstract); < 0.001 (Results)",
             "tmb": "reported 0.53", "ctdna": "reported 0.01"}


def draw_c1(folder, stem):
    fig, ax = plt.subplots(figsize=(7.2, 4.2))
    for i, k in enumerate(ORDER):
        r = R[k]
        y = len(ORDER) - 1 - i
        ax.plot([r["prep"], r["p"]], [y, y], color=NEUTRAL, lw=LW, zorder=2)
        ax.plot(r["prep"], y, "o", ms=MS, color=ORANGE, zorder=3)
        ax.plot(r["p"], y, "o", ms=MS, color=BLUE, zorder=4, mfc=BLUE if k != "tmb" else "none", mew=2)
        ax.annotate(REP_LABEL[k], (r["prep"], y), xytext=(-8 if k == "tp53" else 0, 8), textcoords="offset points",
                    ha={"tmb": "right", "tp53": "left"}.get(k, "center"), va="bottom", fontsize=10, color=TXT)
        ax.annotate("recomputed " + fmtp(r["p"]), (r["p"], y), xytext=(0, -9), textcoords="offset points",
                    ha="center" if k != "tmb" else "right", va="top", fontsize=10, color=TXT)
    ax.axvline(0.05, color=SEC, lw=0.8, ls=(0, (3, 2)), zorder=1)
    ax.text(0.052, -0.85, "P = 0.05", ha="left", va="bottom", fontsize=10, color=SEC)
    ax.set_xscale("log")
    ax.set_xlim(2e-5, 2)
    ax.set_ylim(-0.9, len(ORDER) - 0.45)
    ax.set_yticks(range(len(ORDER)))
    ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
    ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1])
    ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
    ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_xlabel("Two-sided P value (log scale)")
    clean(ax)
    ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P as reported in the paper"),
                       Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="Wald P from the paper's own HR and 95% CI")],
              loc="upper center", bbox_to_anchor=(0.4, -0.17), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, folder, stem)


# ---- C2: the interval P = 0.016 would require (HR log axis) ----
def draw_c2(folder, stem):
    q = R["req"]
    se = pk["se"]
    rows = [
        ("Reported\nHR 0.93 (0.52–1.69), P = 0.016", pk["hr"], pk["lo"], pk["hi"], ORANGE, "o"),
        ("95% CI that P = 0.016 would need\nat HR 0.93", pk["hr"], q["lo_req"], q["hi_req"], BLUE, "o"),
        ("HR that P = 0.016 would need\nwith the reported CI width", q["hr_lo_req"],
         math.exp(math.log(q["hr_lo_req"]) - Z975 * se), math.exp(math.log(q["hr_lo_req"]) + Z975 * se), BLUE, "D"),
        ("", q["hr_hi_req"], math.exp(math.log(q["hr_hi_req"]) - Z975 * se),
         math.exp(math.log(q["hr_hi_req"]) + Z975 * se), BLUE, "D"),
    ]
    ys = [3, 2, 1, 0.55]
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    ax.axvline(1, color=SEC, lw=0.8, zorder=1)
    for (lab, hr, lo, hi, col, mk), y in zip(rows, ys):
        ax.plot([lo, hi], [y, y], color=col, lw=LW * 1.4, solid_capstyle="butt", zorder=2)
        ax.plot(hr, y, mk, ms=MS, color=col, zorder=3)
        ax.annotate(f"{hr:.2f} ({lo:.2f}–{hi:.2f})", (hi, y), xytext=(6, 0), textcoords="offset points",
                    va="center", fontsize=10, color=TXT)
    ax.text(pk["hr"], 2.32, f"Wald P from this interval: {pk['p']:.2f}", ha="center", va="bottom", fontsize=10, color=SEC)
    ax.text(1.03, 0.2, "HR = 1 (no difference)", ha="left", va="bottom", fontsize=10, color=SEC)
    ax.set_xscale("log")
    ax.set_xlim(0.2, 10)
    ax.set_ylim(0.15, 3.45)
    ax.set_yticks([3, 2, 0.775])
    ax.set_yticklabels([rows[0][0], rows[1][0], rows[2][0]], fontsize=10)
    ax.set_xticks([0.25, 0.5, 1, 2, 4])
    ax.set_xticklabels(["0.25", "0.5", "1", "2", "4"])
    ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_xlabel("Hazard ratio for PFS, PIK3CA comparison, with 95% CI (log scale)")
    clean(ax)
    ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="reported in the paper"),
                       Line2D([], [], marker="o", color=BLUE, lw=LW, ms=MS, label="what P = 0.016 would require")],
              loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, folder, stem)


# ---- C3: z-score comparison (linear |z| axis, 1.96 reference) ----
def draw_c3(folder, stem):
    fig, ax = plt.subplots(figsize=(7.2, 3.4))
    for i, k in enumerate(ORDER):
        r = R[k]
        y = len(ORDER) - 1 - i
        a, b = r["zrep"], abs(r["z"])
        ax.plot([a, b], [y, y], color=NEUTRAL, lw=LW, zorder=2)
        ax.plot(a, y, "o", ms=MS, color=ORANGE, zorder=3)
        ax.plot(b, y, "o", ms=MS, color=BLUE, zorder=3)
        ax.annotate(f"{a:.2f}", (a, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10, color=TXT)
        ax.annotate(f"{b:.2f}", (b, y), xytext=(0, -17), textcoords="offset points", ha="center", fontsize=10, color=TXT)
    ax.axvline(Z975, color=SEC, lw=0.8, ls=(0, (3, 2)))
    ax.text(Z975, len(ORDER) - 0.45, "|z| = 1.96 (P = 0.05)", ha="center", va="bottom", fontsize=10, color=SEC)
    ax.set_xlim(0, 4.4)
    ax.set_ylim(-0.6, len(ORDER) - 0.2)
    ax.set_yticks(range(len(ORDER)))
    ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
    ax.set_xlabel("|z| (standard errors from no difference)")
    clean(ax)
    ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="|z| implied by the reported P"),
                       Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="|z| from the reported HR and 95% CI")],
              loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, folder, stem)


# ---- C4: forest plot with reported / recomputed P columns ----
def draw_c4(folder, stem):
    fig = plt.figure(figsize=(7.2, 3.3))
    ax = fig.add_axes([0.27, 0.19, 0.4, 0.62])
    ax.set_xscale("log")
    ax.set_xlim(0.1, 2.2)
    ax.set_ylim(-0.6, len(ORDER) - 0.4)
    for i, k in enumerate(ORDER):
        r = R[k]
        y = len(ORDER) - 1 - i
        ax.plot([r["lo"], r["hi"]], [y, y], color=TXT, lw=LW, zorder=2)
        ax.plot(r["hr"], y, "s", ms=MS - 1, color=TXT, zorder=3)
        fig.text(0.70, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, PREP_SHORT[k], color=ORANGE,
                 fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
        fig.text(0.83, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, fmtp(r["p"]), color=BLUE,
                 fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
    ax.axvline(1, color=SEC, lw=0.8)
    ax.set_yticks(range(len(ORDER)))
    ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)], fontsize=10)
    ax.set_xticks([0.125, 0.25, 0.5, 1, 2])
    ax.set_xticklabels(["0.125", "0.25", "0.5", "1", "2"])
    ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_xlabel("HR and 95% CI as printed (log scale)")
    clean(ax)
    fig.text(0.70, 0.84, "reported P", color=ORANGE, fontsize=10.5, ha="left", va="bottom")
    fig.text(0.83, 0.84, "recomputed P\n(Wald, HR/CI)", color=BLUE, fontsize=10.5, ha="left", va="bottom")
    save(fig, folder, stem)


# ---- C5: two text panels with an annotated connector ----
def draw_c5(folder, stem):
    fig = plt.figure(figsize=(7.2, 4.4))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 10)
    ax.set_ylim(0, 6.1)
    ax.axis("off")

    def panel(x, y, w, h, title, body, edge):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
                                    fc="#ffffff", ec=edge, lw=LW))
        ax.text(x + 0.2, y + h - 0.22, title, fontsize=10.5, fontweight="bold", va="top", color=TXT)
        ax.text(x + 0.2, y + h - 0.68, body, fontsize=10, va="top", color=TXT, linespacing=1.4)

    panel(0.2, 2.85, 4.55, 2.75, "Abstract (Results; Conclusions)",
          '"Patients with alterations in PIK3CA\nand TP53 at baseline had shorter\nmedian progression-free survival\n'
          '... (P = .016 and P = .0001,\nrespectively)."\n'
          '"... PIK3CA and TP53 alterations ...\nwere associated with inferior outcomes."', ORANGE)
    panel(5.25, 2.85, 4.55, 2.75, "Results, Gene alterations at baseline",
          '"Median PFS of patients with PIK3CA\nmutations was 12.7 months vs 19.2\nmonths ...; however, the difference\n'
          'was not statistically significant\n(HR, 0.93; 95% CI, 0.52–1.69;\nP = 0.016; Fig. 7B)."', ORANGE)
    panel(2.55, 0.2, 4.9, 1.75, "Recomputed from the HR and 95% CI",
          f"SE(ln HR) = {pk['se']:.3f}, z = {pk['z']:.2f}, Wald P = {pk['p']:.2f}\n"
          f"P = 0.016 would need |z| = {R['req']['z016']:.2f}, i.e. a\n"
          f"95% CI of {R['req']['lo_req']:.2f}–{R['req']['hi_req']:.2f} at HR 0.93", BLUE)
    ax.add_patch(FancyArrowPatch((4.78, 4.25), (5.22, 4.25), arrowstyle="<->", mutation_scale=14, color=SEC, lw=LW))
    ax.text(5.0, 5.85, "Same comparison and the same P value: cited as an association in the Abstract,"
            "\ncalled not statistically significant in the Results",
            ha="center", va="center", fontsize=10, color=SEC)
    ax.add_patch(FancyArrowPatch((7.5, 2.85), (6.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
    ax.text(7.25, 2.4, "CI 0.52–1.69 includes 1", fontsize=10, color=SEC, va="center", ha="left")
    ax.add_patch(FancyArrowPatch((2.5, 2.85), (3.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
    ax.text(2.75, 2.4, "P = .016 is not\nreproduced", fontsize=10, color=SEC, va="center", ha="right")
    save(fig, folder, stem)


DRAW = {"c1_p_ladder": draw_c1, "c2_required_interval": draw_c2, "c3_z_scores": draw_c3,
        "c4_forest_with_p_columns": draw_c4, "c5_text_panels": draw_c5}
for stem, fn in DRAW.items():
    fn(CAND, stem)
# defaults used in writeup.md
draw_c1(HERE, "fig1")
draw_c2(HERE, "fig2")
print("\nFigures written: fig1 (= candidate c1), fig2 (= candidate c2), candidates/c1..c5")

Abstract reports all pocket-depth measures significant for CVD mortality; Table 2 shows none are

BMC Med (2023) · doi:10.1186/s12916-023-03139-4 · PMC10642059

Summary

This cohort study (SHIP-START, n = 3047, 125 CVD deaths) relates periodontal and inflammatory markers to mortality. Its abstract states that "For CVD mortality, all PPD related variables showed significant associations". The paper's own Table 2 shows the opposite: for CVD mortality, none of the three pocket probing depth (PPD) measures is significant. All six confidence intervals (per unit and per SD) include 1, the printed P values are 0.275, 0.302 and 0.337, and the paper's Results text says the same thing. The numbers in the table are internally consistent: Wald P values recomputed from the printed intervals are 0.274, 0.303 and 0.336. The abstract sentence is the only single-exposure CVD result it reports. Read alone, it reverses the finding for the periodontal marker that the paper describes as best reflecting current periodontal inflammation. The only published correction (January 2025) concerns funding and leaves the sentence unchanged.

What the paper reports

Abstract, Results:

For CVD mortality, all PPD related variables showed significant associations.

Results, "Inflammation and periodontitis as risk factors for mortality":

However, compared with the results for CAL and missing teeth, the PPD measures showed only weak associations with mortality, as indicated by smaller effect sizes. For CVD mortality, neither mean PPD nor extent PPD ≥ 3 mm/4 mm showed a relevant association (Table 2). Classical Cox proportional hazards models with fixed baseline exposures and covariates (Additional file 1: Table S4) confirmed these results.

Table 2, PPD rows, CVD-mortality columns. The columns are HR (95% CI) per 1-unit increase | HR (95% CI) per SD increase | P-value:

Mean PPD, mm | 1.111 (0.920; 1.341) | 1.078 (0.942; 1.233) | 0.275

Sites with PPD ≥ 3 mm, % | 1.004 (0.997; 1.011) | 1.096 (0.921; 1.305) | 0.302

Sites with PPD ≥ 4 mm, % | 1.004 (0.996; 1.012) | 1.070 (0.932; 1.228) | 0.337

Methods, "Statistical analyses", on the significance threshold:

Regarding the interaction terms, two-sided P values < 0.10 were considered statistically significant. Otherwise, 0.05 was set as the cut-off for significance.

A related sentence in the Abstract, Results, which comes just before the CVD sentence:

All markers of periodontitis and inflammation showed apparent associations with all-cause mortality (HRs per SD-increase: mean PPD: 1.068 (95% confidence interval (CI): 0.988–1.155) […]

The Abstract, Conclusions, and the main-text Conclusions:

Both periodontitis and inflammation were significantly associated with all-cause mortality and CVD mortality.

Periodontitis and systemic inflammation were both significantly associated with all considered types of mortality.

Discussion, on how PPD is interpreted:

measures of PPD, which best reflects current periodontal inflammation

What the paper's own numbers show

Method. Table 2 prints, for each exposure and outcome, an HR and 95% CI per unit, an HR and 95% CI per SD, and a single P value. I assumed each interval is a Wald interval, symmetric on the log-HR scale, as Cox model output is. On that basis I recomputed P from each printed interval: SE = (ln U − ln L)/(2 × 1.96), z = ln HR / SE, P = 2[1 − Φ(|z|)]. I also worked the other way and rebuilt each per-SD interval from the HR and the printed P. To confirm that the table's rows and columns are aligned, I ran the same check on the other CVD rows and on the PPD rows for the other two outcomes. I repeated it for Table S4 in Additional file 1, the time-invariant sensitivity models. recompute.py.txt prints every number below.

Quantity Location Reported Recomputed from the paper's numbers
Mean PPD, CVD mortality, per SD Table 2 1.078 (0.942; 1.233), P = 0.275 CI includes 1; P = 0.274; CI rebuilt from P: 0.942–1.234
Mean PPD, CVD mortality, per 1 mm Table 2 1.111 (0.920; 1.341) CI includes 1; P = 0.273
Sites PPD ≥ 3 mm, CVD mortality, per SD Table 2 1.096 (0.921; 1.305), P = 0.302 CI includes 1; P = 0.303; CI rebuilt from P: 0.921–1.304
Sites PPD ≥ 3 mm, CVD mortality, per 1 % Table 2 1.004 (0.997; 1.011) CI includes 1; P = 0.262 (anywhere from 0.174 to 0.359 within 3-decimal rounding of the HR and limits)
Sites PPD ≥ 4 mm, CVD mortality, per SD Table 2 1.070 (0.932; 1.228), P = 0.337 CI includes 1; P = 0.336; CI rebuilt from P: 0.932–1.228
Sites PPD ≥ 4 mm, CVD mortality, per 1 % Table 2 1.004 (0.996; 1.012) CI includes 1; P = 0.326 (0.239–0.419 within rounding)
PPD measures significant for CVD mortality (P < 0.05) Abstract vs Table 2 Abstract: "all" (3 of 3) 0 of 3 printed P; 0 of 6 recomputed P (both scalings); 0 of 6 CIs exclude 1
One-sided P (printed P halved), the three PPD measures derived — 0.138, 0.151, 0.169, all still > 0.05
Control: mean CAL, CVD, per SD Table 2 1.225 (1.042; 1.439), P = 0.014 P = 0.014
Control: missing teeth, CVD, per SD Table 2 1.226 (1.025; 1.466), P = 0.026 P = 0.026
Control: periodontitis score, CVD, per SD Table 2 1.399 (1.076; 1.820), P = 0.012 P = 0.012
Control: fibrinogen, CVD, per SD Table 2 1.128 (0.981; 1.297), P = 0.090 P = 0.091
Control: sites CAL ≥ 3 mm, CVD, per SD Table 2 1.288 (0.976; 1.702), P = 0.074 P = 0.074
Control: sites CAL ≥ 4 mm, CVD, per SD Table 2 1.217 (1.002; 1.479), P = 0.047 P = 0.048
PPD measures, time-invariant models, CVD, per SD Table S4 1.080 (0.927; 1.259) P = 0.321; 1.057 (0.883; 1.264) P = 0.547; 1.057 (0.917; 1.219) P = 0.444 P = 0.324, 0.545, 0.445; 0 of 3 significant
PPD measures significant, other outcomes Table 2 all-cause P = 0.098, 0.025, 0.030; non-CVD P = 0.166, 0.030, 0.037 2 of 3 for each outcome; mean PPD not significant for either
PPD measures significant, other outcomes Table S4 all-cause P = 0.020, 0.031, 0.042; non-CVD P = 0.023, 0.018, 0.035 3 of 3 for each outcome
Mean PPD, all-cause mortality, per SD (abstract: "apparent association") Abstract; Table 2 1.068 (0.988–1.155), P = 0.098 CI includes 1; P = 0.099

Every recomputed P agrees with the printed P to within rounding, both for the PPD rows and for the control rows. The only exceptions are the per-unit rows for extent PPD, and those have little precision because the HR and its limits are printed to three decimals close to 1. The table is therefore internally consistent. What disagrees with it is the abstract sentence.

Figure

Abstract sentence beside the Results sentence and the Table 2 CVD-mortality rows for the three PPD measures

Figure 1. The left panel shows the abstract sentence on CVD mortality. The right panel shows the Results sentence and the three Table 2 rows it summarises, with the per-SD hazard ratio (95% CI), the P value printed in the paper (orange circle) and the P value recomputed from the printed interval (blue square). The connector names the conflict. The abstract implies 3 of 3 PPD measures are significant; the table has 0 of 3. All numbers are printed in the paper or by recompute.py.txt. The paper's own figures are not reproduced.

Forest plot of per-SD hazard ratios for CVD mortality, printed and rebuilt 95% CIs, for the three PPD measures and three control measures

Figure 2. Hazard ratios per SD increase for CVD mortality from Table 2, on a log axis with a reference line at HR = 1. For each row, the orange interval is the 95% CI printed in the paper and the blue interval is the 95% CI rebuilt from the HR and the printed P. The printed P values are listed on the right. The two intervals coincide for every row, so the HR, CI and P columns are mutually consistent. All three PPD intervals cross 1. The three control rows from the same table (mean CAL, missing teeth, periodontitis score) do not, which shows that significant CVD associations in this table are reported as such.

Why it matters

Most readers see only the abstract, and in it this sentence is the only result given for a single exposure and CVD mortality. It tells them that every pocket-depth measure is significantly associated with cardiovascular death, but the paper's analysis finds no PPD measure significantly associated with CVD mortality as a main effect, in either the main models (Table 2) or the sensitivity models (Table S4). The difference matters for interpretation. The paper describes PPD as the measure that "best reflects current periodontal inflammation". In this paper, the CVD association is carried by markers of cumulative periodontal damage instead: mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score. Someone relying on the abstract might cite this study as evidence that current periodontal inflammation, measured as pocket depth, predicts cardiovascular death, but the paper's main-effect analyses do not support that.

The error does not overturn the paper's other main results. The periodontitis score, mean CAL, missing teeth, leukocytes, CRP and the inflammation score are all significant for CVD mortality in Table 2. The additive-interaction (RERI) results are separate analyses and are not affected. Two related statements are broader than the table supports, though. First, the abstract lists mean PPD (P = 0.098) among markers with "apparent associations" with all-cause mortality. Second, both Conclusions say periodontitis and inflammation were significantly associated with CVD mortality, and that is not true for any PPD measure, for extent CAL ≥ 3 mm (P = 0.074) or for fibrinogen (P = 0.090).

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the Abstract, Results, replace "For CVD mortality, all PPD related variables showed significant associations." with a sentence consistent with Table 2 and the Results text. For example: "For CVD mortality, none of the PPD-related variables showed a significant association (HRs per SD increase: mean PPD 1.078 (95% CI 0.942–1.233); extent PPD ≥ 3 mm 1.096 (0.921–1.305); extent PPD ≥ 4 mm 1.070 (0.932–1.228)), whereas mean CAL, extent CAL ≥ 4 mm, missing teeth, the periodontitis score and the inflammation markers other than fibrinogen did." The authors may also wish to review three related statements. First, the description of mean PPD (P = 0.098) as showing an "apparent association" with all-cause mortality. Second and third, the two Conclusions that periodontitis was significantly associated with CVD mortality, which hold for mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score but not for the PPD measures (or for extent CAL ≥ 3 mm, P = 0.074).

Recomputation script (python recompute.py.txt)
"""Recompute the CVD-mortality results for the pocket probing depth (PPD) measures
in PMC10642059 (BMC Med 2023;21:430, doi:10.1186/s12916-023-03139-4).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
  fig1.svg/png  (= candidates/cand_A)  abstract vs Results/Table 2 text panels
  fig2.svg/png  (= candidates/cand_B)  reported vs recomputed 95% CIs, log HR axis
  candidates/cand_A..cand_E.svg/png    five candidate figure designs

Inputs are copied from the paper as archived in PMC (Table 2) and from
Additional file 1 (12916_2023_3139_MOESM1_ESM.docx, Table S4).

Method. Each 95% CI is assumed symmetric on the log-HR scale (Wald), so
  SE = (ln U - ln L) / (2 * 1.959964),   z = ln(HR) / SE,   P = 2 * (1 - Phi(|z|)).
In reverse, a CI is rebuilt from the HR and the reported two-sided P:
  SE = |ln HR| / Phi^-1(1 - P/2),   CI = HR * exp(+/- 1.959964 * SE).
"""
import os
import shutil

import numpy as np
from scipy.stats import norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = norm.ppf(0.975)

# ------------------------------------------------------------------ inputs (paper)
# Table 2: (per-unit HR, lo, hi), (per-SD HR, lo, hi), reported P  -- one P per row
T2 = {
    "all-cause": {
        "Mean PPD":     ((1.097, 0.983, 1.224), (1.068, 0.988, 1.155), 0.098),
        "PPD >= 3 mm":  ((1.005, 1.000, 1.009), (1.125, 1.015, 1.247), 0.025),
        "PPD >= 4 mm":  ((1.005, 1.000, 1.010), (1.091, 1.008, 1.181), 0.030),
    },
    "CVD": {
        "Mean PPD":     ((1.111, 0.920, 1.341), (1.078, 0.942, 1.233), 0.275),
        "PPD >= 3 mm":  ((1.004, 0.997, 1.011), (1.096, 0.921, 1.305), 0.302),
        "PPD >= 4 mm":  ((1.004, 0.996, 1.012), (1.070, 0.932, 1.228), 0.337),
    },
    "non-CVD": {
        "Mean PPD":     ((1.100, 0.961, 1.258), (1.070, 0.972, 1.178), 0.166),
        "PPD >= 3 mm":  ((1.006, 1.001, 1.011), (1.151, 1.014, 1.308), 0.030),
        "PPD >= 4 mm":  ((1.006, 1.000, 1.012), (1.108, 1.006, 1.220), 0.037),
    },
}
# Table 2, CVD mortality, other exposures (per-SD HR, lo, hi, reported P) -- controls
T2_CVD_OTHER = {
    "Mean CAL":             (1.225, 1.042, 1.439, 0.014),
    "CAL >= 3 mm":          (1.288, 0.976, 1.702, 0.074),
    "CAL >= 4 mm":          (1.217, 1.002, 1.479, 0.047),
    "Missing teeth":        (1.226, 1.025, 1.466, 0.026),
    "Periodontitis score":  (1.399, 1.076, 1.820, 0.012),
    "Leukocytes":           (1.316, 1.143, 1.514, None),   # printed "< 0.001"
    "Fibrinogen":           (1.128, 0.981, 1.297, 0.090),
    "CRP (log)":            (1.236, 1.028, 1.488, 0.024),
    "Inflammation score":   (1.385, 1.141, 1.681, 0.001),
}
# Additional file 1, Table S4 (time-invariant models), per-SD HR, lo, hi, reported P
S4 = {
    "all-cause": {
        "Mean PPD":    (1.109, 1.017, 1.209, 0.020),
        "PPD >= 3 mm": (1.127, 1.011, 1.255, 0.031),
        "PPD >= 4 mm": (1.091, 1.003, 1.186, 0.042),
    },
    "CVD": {
        "Mean PPD":    (1.080, 0.927, 1.259, 0.321),
        "PPD >= 3 mm": (1.057, 0.883, 1.264, 0.547),
        "PPD >= 4 mm": (1.057, 0.917, 1.219, 0.444),
    },
    "non-CVD": {
        "Mean PPD":    (1.128, 1.017, 1.253, 0.023),
        "PPD >= 3 mm": (1.176, 1.028, 1.345, 0.018),
        "PPD >= 4 mm": (1.116, 1.008, 1.237, 0.035),
    },
}
PPD = ["Mean PPD", "PPD >= 3 mm", "PPD >= 4 mm"]
OUTCOMES = ["all-cause", "CVD", "non-CVD"]


def p_from_ci(hr, lo, hi):
    se = (np.log(hi) - np.log(lo)) / (2 * Z975)
    z = np.log(hr) / se
    return 2 * norm.sf(abs(z)), se, z


def ci_from_p(hr, p):
    se = abs(np.log(hr)) / norm.isf(p / 2)
    return hr * np.exp(-Z975 * se), hr * np.exp(Z975 * se)


# ------------------------------------------------------------------ 1. CVD PPD rows
print("1. Table 2, CVD mortality, PPD measures (Wald P recomputed from the printed CI)")
print(f"   {'measure':12s} {'scale':8s} {'HR (95% CI)':22s} {'SE(lnHR)':>8s} {'z':>6s} "
      f"{'P recomp':>8s} {'P printed':>9s} CI includes 1")
REC = {}
for m in PPD:
    unit, sd, prep = T2["CVD"][m]
    for scale, (hr, lo, hi) in (("per unit", unit), ("per SD", sd)):
        p, se, z = p_from_ci(hr, lo, hi)
        REC[(m, scale)] = p
        print(f"   {m:12s} {scale:8s} {hr:.3f} ({lo:.3f}; {hi:.3f})   {se:8.4f} {z:6.2f} "
              f"{p:8.3f} {prep:9.3f} {lo <= 1 <= hi}")
n_sig_sd = sum(T2["CVD"][m][2] < 0.05 for m in PPD)
n_sig_rec = sum(REC[(m, s)] < 0.05 for m in PPD for s in ("per unit", "per SD"))
n_ci_excl = sum(not (c[1] <= 1 <= c[2]) for m in PPD for c in T2["CVD"][m][:2])
print(f"   PPD measures with printed P < 0.05 for CVD mortality: {n_sig_sd} of 3")
print(f"   recomputed P < 0.05: {n_sig_rec} of 6 (both scalings); CIs excluding 1: {n_ci_excl} of 6")
print("   per-SD P range printed: %.3f-%.3f; recomputed: %.3f-%.3f" % (
    min(T2["CVD"][m][2] for m in PPD), max(T2["CVD"][m][2] for m in PPD),
    min(REC[(m, "per SD")] for m in PPD), max(REC[(m, "per SD")] for m in PPD)))
print("   one-sided P (printed P halved): " +
      ", ".join(f"{m} {T2['CVD'][m][2] / 2:.3f}" for m in PPD))
# per-unit rows with CI printed to 3 decimals near 1: how much can rounding move P?
for m in ("PPD >= 3 mm", "PPD >= 4 mm"):
    hr, lo, hi = T2["CVD"][m][0]
    ps = [p_from_ci(h, l, u)[0] for h in (hr - 5e-4, hr + 5e-4)
          for l in (lo - 5e-4, lo + 5e-4) for u in (hi - 5e-4, hi + 5e-4)]
    print(f"   {m} per unit: P compatible with 3-decimal rounding of HR/CI: "
          f"{min(ps):.3f}-{max(ps):.3f} (printed {T2['CVD'][m][2]:.3f})")

# ------------------------------------------------------------------ 2. CI rebuilt from P
print("\n2. 95% CI rebuilt from the per-SD HR and the printed P (CVD mortality)")
REBUILT = {}
for m in PPD:
    hr, lo, hi = T2["CVD"][m][1]
    rl, rh = ci_from_p(hr, T2["CVD"][m][2])
    REBUILT[m] = (rl, rh)
    print(f"   {m:12s} printed ({lo:.3f}; {hi:.3f})  rebuilt ({rl:.3f}; {rh:.3f})")

# ------------------------------------------------------------------ 3. control rows
print("\n3. Control rows, Table 2 CVD mortality, per SD (table internal consistency)")
for k, (hr, lo, hi, p) in T2_CVD_OTHER.items():
    pc = p_from_ci(hr, lo, hi)[0]
    ptxt = "< 0.001" if p is None else f"{p:.3f}"
    pctxt = "< 0.001" if pc < 0.001 else f"{pc:.3f}"
    print(f"   {k:20s} {hr:.3f} ({lo:.3f}; {hi:.3f})  P recomp {pctxt}  printed {ptxt}  "
          f"{'sig' if (p is None or p < 0.05) else 'n.s.'}")

# ------------------------------------------------------------------ 4. other outcomes
print("\n4. Same PPD rows for the other outcomes (Table 2 per SD and Table S4 per SD)")
SIGCOUNT = {}
for src, data in (("Table 2", {o: {m: (*T2[o][m][1], T2[o][m][2]) for m in PPD} for o in OUTCOMES}),
                  ("Table S4", S4)):
    for o in OUTCOMES:
        cells = []
        n = 0
        for m in PPD:
            hr, lo, hi, p = data[o][m]
            pc = p_from_ci(hr, lo, hi)[0]
            n += p < 0.05
            cells.append(f"{m} P {p:.3f} (recomp {pc:.3f})")
        SIGCOUNT[(src, o)] = n
        print(f"   {src:8s} {o:9s} {n} of 3 significant: " + "; ".join(cells))
print("   all-cause mean PPD, Table 2 per SD: 1.068 (0.988; 1.155), P recomp "
      f"{p_from_ci(1.068, 0.988, 1.155)[0]:.3f} (printed 0.098)")

# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
LW = 2 * 72 / 100          # 2 px at 100 px/in in points
MS = 8                     # marker size in points (>= 8 px)


def save(fig, name, also=None):
    for ext in ("svg", "png"):
        p = os.path.join(CAND, f"{name}.{ext}")
        fig.savefig(p, dpi=200, bbox_inches="tight", pad_inches=0.12)
        if also:
            shutil.copyfile(p, os.path.join(HERE, f"{also}.{ext}"))
    plt.close(fig)


def box(ax, x, y, w, h, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc="white", ec=edge, lw=1.2, transform=ax.transAxes))


LBL = {"Mean PPD": "Mean PPD", "PPD >= 3 mm": "Sites PPD ≥ 3 mm",
       "PPD >= 4 mm": "Sites PPD ≥ 4 mm"}

# ---- A: two text panels joined by an annotated connector -----------------------
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.01, 0.36, 0.33, 0.60, INK2)
box(ax, 0.47, 0.02, 0.52, 0.94, INK2)
ax.text(0.03, 0.92, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.03, 0.83, "“For CVD mortality, all\nPPD related variables\nshowed significant\nassociations.”",
        fontsize=10.5, va="top", style="italic", linespacing=1.4)
ax.text(0.03, 0.52, "Implies: 3 of 3 PPD\nmeasures significant\nfor CVD mortality",
        fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.49, 0.92, "Results text and Table 2", fontsize=11, weight="bold", va="top")
ax.text(0.49, 0.85, "“For CVD mortality, neither mean PPD nor\nextent PPD ≥ 3 mm/4 mm showed a relevant\n"
        "association (Table 2).”", fontsize=10, va="top", style="italic", linespacing=1.4)
yh = 0.585
ax.text(0.49, yh, "CVD mortality,\nHR per SD (95% CI)", fontsize=10, color=INK2, va="center", linespacing=1.2)
ax.plot([0.775], [yh], "o", ms=MS, color=ORANGE)
ax.text(0.79, yh, "P\nprinted", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.885], [yh], "s", ms=MS, color=BLUE)
ax.text(0.90, yh, "P\nrecomp.", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.49, 0.975], [yh - 0.055, yh - 0.055], color=GRID, lw=1)
for i, m in enumerate(PPD):
    y = yh - 0.13 - i * 0.115
    hr, lo, hi = T2["CVD"][m][1]
    ax.text(0.49, y + 0.025, LBL[m], fontsize=10, va="center")
    ax.text(0.49, y - 0.028, f"{hr:.3f} ({lo:.3f}–{hi:.3f})", fontsize=10, va="center", color=INK2)
    ax.text(0.79, y, f"{T2['CVD'][m][2]:.3f}", fontsize=10, va="center")
    ax.text(0.90, y, f"{REC[(m, 'per SD')]:.3f}", fontsize=10, va="center")
ax.text(0.49, 0.055, "Every CI includes 1; every P > 0.05\n(the paper's cut-off for main effects).",
        fontsize=10, color=INK2, va="bottom", linespacing=1.3)
ax.add_patch(FancyArrowPatch((0.345, 0.72), (0.465, 0.72), arrowstyle="<->", mutation_scale=14,
                             lw=LW, color=INK))
ax.text(0.405, 0.69, "Conflict", ha="center", va="top", fontsize=10.5, weight="bold")
ax.text(0.405, 0.63, "Abstract:\n3 of 3\n\nTable 2:\n0 of 3", ha="center", va="top", fontsize=10,
        linespacing=1.3)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="printed in the paper"),
                   Line2D([], [], marker="s", ls="", ms=MS, color=BLUE,
                          label="recomputed from the\nprinted 95% CI (Wald,\nlog scale)")],
          loc="lower left", bbox_to_anchor=(0.01, 0.03), frameon=False, fontsize=10, handletextpad=0.4)
save(fig, "cand_A", also="fig1")

# ---- B: reported vs rebuilt CI on one log axis, with control rows ---------------
rows = [(LBL[m], *T2["CVD"][m][1], REBUILT[m]) for m in PPD]
ctrl = []
for k in ("Mean CAL", "Missing teeth", "Periodontitis score"):
    hr, lo, hi, p = T2_CVD_OTHER[k]
    ctrl.append((k + " (control)", hr, lo, hi, ci_from_p(hr, p)))
allrows = rows + ctrl
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.array([6.5, 5.5, 4.5, 2.5, 1.5, 0.5])
for y, (lab, hr, lo, hi, (rl, rh)) in zip(ys, allrows):
    ax.plot([lo, hi], [y + 0.15] * 2, color=ORANGE, lw=LW, solid_capstyle="round")
    ax.plot(hr, y + 0.15, "o", ms=MS, color=ORANGE, mec=BG, mew=1)
    ax.plot([rl, rh], [y - 0.15] * 2, color=BLUE, lw=LW, solid_capstyle="round")
    ax.plot(hr, y - 0.15, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1)
ax.axvline(1, color=INK, lw=1, zorder=1.5)
ax.set_xscale("log")
ticks = [0.9, 1.0, 1.2, 1.4, 1.6, 1.8]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks]); ax.minorticks_off()
ax.set_xlim(0.86, 2.2)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in allrows], fontsize=10)
for t in ax.get_yticklabels()[3:]:
    t.set_color(INK2)
ax.set_ylim(-0.1, 7.6)
ax.axhline(3.5, color=GRID, lw=1)
ax.text(0.87, 7.25, "PPD measures (abstract: “significant”)", fontsize=10, weight="bold", va="center",
        bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(2.18, 7.25, "printed P", ha="right", fontsize=10, color=INK2, va="center")
ax.text(0.87, 3.15, "Other periodontal measures, same table", fontsize=10, color=INK2, va="center",
        bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
for y, m in zip(ys[:3], PPD):
    ax.text(2.18, y, f"{T2['CVD'][m][2]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
for y, k in zip(ys[3:], ("Mean CAL", "Missing teeth", "Periodontitis score")):
    ax.text(2.18, y, f"{T2_CVD_OTHER[k][3]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xlabel("Hazard ratio per SD increase, CVD mortality (log scale; line at HR = 1)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="95% CI printed in Table 2"),
                   Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS - 1,
                          label="95% CI rebuilt from HR and printed P")],
          loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B", also="fig2")

# ---- C: significance grid, PPD x outcome x table --------------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.4))
xs = {("Table 2", o): i for i, o in enumerate(OUTCOMES)}
xs.update({("Table S4", o): i + 3.6 for i, o in enumerate(OUTCOMES)})
for (src, o), x in xs.items():
    if o == "CVD":
        ax.add_patch(plt.Rectangle((x - 0.45, -0.5), 0.9, 3.0, fc="#f1f0ec", ec="none", zorder=0))
    for j, m in enumerate(PPD):
        y = 2 - j
        if src == "Table 2":
            hr, lo, hi = T2[o][m][1]; p = T2[o][m][2]
        else:
            hr, lo, hi, p = S4[o][m]
        pc = p_from_ci(hr, lo, hi)[0]
        filled = p < 0.05
        ax.plot(x - 0.17, y + 0.12, "o", ms=11, mfc=ORANGE if filled else BG, mec=ORANGE, mew=2)
        ax.plot(x + 0.17, y + 0.12, "s", ms=10, mfc=BLUE if pc < 0.05 else BG, mec=BLUE, mew=2)
        ax.text(x, y - 0.25, f"{p:.3f}", ha="center", va="center", fontsize=10,
                color=INK if filled else INK2)
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([LBL[m] for m in PPD])
ax.set_xticks(list(xs.values())); ax.set_xticklabels([o for (_, o) in xs], fontsize=10)
for (src, x0) in (("Table 2 (time-varying, main)", 1), ("Table S4 (time-invariant)", 4.6)):
    ax.text(x0, 3.55, src, ha="center", fontsize=10.5, weight="bold")
ax.text(1, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.text(4.6, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.set_xlim(-0.6, 6.2); ax.set_ylim(-1.0, 3.1)
ax.xaxis.tick_top()
for s in ax.spines.values():
    s.set_visible(False)
ax.tick_params(length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=10, mfc=ORANGE, mec=ORANGE, label="printed P < 0.05"),
                   Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=ORANGE, mew=2, label="printed P ≥ 0.05"),
                   Line2D([], [], marker="s", ls="", ms=9, mfc=BLUE, mec=BLUE, label="recomputed P < 0.05"),
                   Line2D([], [], marker="s", ls="", ms=9, mfc=BG, mec=BLUE, mew=2, label="recomputed P ≥ 0.05")],
          loc="upper center", bbox_to_anchor=(0.5, -0.12), ncol=4, frameon=False, fontsize=9.5,
          handletextpad=0.3, columnspacing=1.0,
          title="Number under each pair: printed P (HR per SD)", title_fontsize=10)
save(fig, "cand_C")

# ---- D: P values on one log axis with the 0.05 line -----------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.6))
items = [(f"{LBL[m]} (Table 2)", T2["CVD"][m][2], REC[(m, 'per SD')]) for m in PPD]
items += [(f"{LBL[m]} (Table S4)", S4["CVD"][m][3], p_from_ci(*S4["CVD"][m][:3])[0]) for m in PPD]
items += [(f"{k} (Table 2, control)", T2_CVD_OTHER[k][3], p_from_ci(*T2_CVD_OTHER[k][:3])[0])
          for k in ("Mean CAL", "Missing teeth")]
yy = np.arange(len(items))[::-1].astype(float)
yy[:6] += 0.5
for y, (lab, p, pc) in zip(yy, items):
    ax.plot([min(p, pc), max(p, pc)], [y, y], color=GRID, lw=LW, zorder=1)
    ax.plot(p, y + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1, zorder=3)
    ax.plot(pc, y - 0.13, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1, zorder=3)
ax.axvline(0.05, color=INK, lw=1)
ax.text(0.052, yy[-1] - 0.75, "P = 0.05", fontsize=10, va="bottom")
ax.axvspan(0.005, 0.05, color="#f1f0ec", zorder=0)
ax.text(0.0065, yy[0] + 0.15, "significant\nat α = 0.05", fontsize=10, color=INK2, va="top")
ax.set_xscale("log"); ax.set_xlim(0.005, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1]); ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.minorticks_off()
ax.set_yticks(yy); ax.set_yticklabels([i[0] for i in items], fontsize=10)
ax.set_ylim(yy[-1] - 0.8, yy[0] + 0.6)
ax.set_xlabel("P value, CVD mortality, HR per SD (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P printed in the paper"),
                   Line2D([], [], marker="s", ls="", ms=MS - 1, color=BLUE, label="P recomputed from printed CI")],
          loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_D")

# ---- E: count comparison --------------------------------------------------------
fig, ax = plt.subplots(figsize=(7.2, 2.6))
bars = [("Abstract claim", 3, ORANGE, "o"),
        ("Table 2, printed P", SIGCOUNT[("Table 2", "CVD")], ORANGE, "o"),
        ("Table 2, P recomputed from CI", sum(REC[(m, 'per SD')] < 0.05 for m in PPD), BLUE, "s"),
        ("Table S4, printed P", SIGCOUNT[("Table S4", "CVD")], ORANGE, "o"),
        ("Table S4, P recomputed from CI", sum(p_from_ci(*S4["CVD"][m][:3])[0] < 0.05 for m in PPD), BLUE, "s")]
yb = np.arange(len(bars))[::-1]
for y, (lab, n, c, mk) in zip(yb, bars):
    ax.plot([0, n], [y, y], color=c, lw=LW, solid_capstyle="round")
    ax.plot(n, y, mk, ms=MS + 1, color=c, mec=BG, mew=1)
    ax.text(n + 0.08, y, f"{n} of 3", va="center", fontsize=10)
ax.set_yticks(yb); ax.set_yticklabels([b[0] for b in bars], fontsize=10)
ax.set_xlim(-0.1, 3.5); ax.set_xticks([0, 1, 2, 3])
ax.set_xlabel("PPD measures significant for CVD mortality (P < 0.05), out of 3")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="stated or printed in the paper"),
                   Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="recomputed from the paper's CIs")],
          loc="upper center", bbox_to_anchor=(0.35, -0.3), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E")

print("\nwrote fig1/fig2 and candidates/cand_A..cand_E (svg, png) in", HERE)

All reported HPV prevalence confidence intervals are about 20 to 140 times narrower than the counts allow

BMC Med (2025) · doi:10.1186/s12916-025-04246-0 · PMC12211446

Summary

The Abstract, Results and Table 1 report a cervicovaginal HPV prevalence of 6.2% (51 of 822 women) with a 95% confidence interval of 6.18%–6.22%, and give intervals of ±0.01 to ±0.02 percentage points for every genotype-specific prevalence. From the paper's own counts, the exact binomial 95% interval for 51/822 is 4.65%–8.08%, 86 times as wide as the reported one, and the genotype intervals are 22 to 143 times as wide as reported; the point estimates themselves are correct. Four of the six reported intervals are reproduced to the printed decimal by one arithmetic step, a 95% half-width computed on the proportion scale (0.0165 for 51/822) added to a prevalence expressed in percent. Prevalence is the study's primary outcome, so its stated precision is overstated about 80-fold, which bears on the Discussion's comparison with 2009 estimates (the 2009 HPV16/18 value of 1.2% lies inside the corrected interval) and on any meta-analysis that uses the published interval.

What the paper reports

The prevalence of HPV infection was 6.2% (95% CI: 6.18%-6.22%), while the prevalence of high-risk genotypes 16 and 18 was 1.94% (95% CI: 1.93%-1.95%). The prevalence of 12 pooled high-risk HPV infection was 4.14% (95% CI:4.13%-4.15%).

Abstract, Results

The prevalence of cervicovaginal HPV infection among ever-married women of the 35-year-old age cohort was 6.2% (95% CI: 6.18%-6.22%), as 51, had tested positive for HPV infection out of 822 participants. The prevalence of high-risk carcinogenic HPV genotypes 16(n = 14) and 18(n = 2) was 1.94% (95% CI: 1.93%-1.95%), while the other 12 pooled high-risk HPV genotypes (n = 34) was 4.14% (95% CI: 4.13%-4.15%) (Table 1).

Results, paragraph 2

Table 1 ("Prevalence of cervico-vaginal HPV infection in the District") has a column headed "95% CI for the prevalence %". Its rows are: 12 pooled positive, 34, 4.14, 4.13-4.15; 16 positive, 14, 1.7, 1.69-1.71; 18 positive, 2, 0.24, 0.23-0.25; 16 & 12 pooled positive, 01, 0.12, 0.1-0.13; Negative, 771, 93.8; Total, 822.

On the design and analysis, the paper says:

In the district, there are 413 PHM areas.

Methods, paragraph 3 (each Public Health Midwife area is a cluster, and three women were selected from each)

the degree of accuracy desired specified was 0.016 (3.3/100 × 2). Therefore we needed 479 women.

Methods, paragraph 2 (the required sample size was then "multiplied by the design effect, which was taken as 1.1")

Data were entered by using SPSS version 20.

Methods, paragraph 8

Moreover, high-risk genotypes 16 and 18 prevalence were elevated from 1.2% to 1.94% within the same duration.

Discussion, paragraph 2

What the paper's own numbers show

Method. Each prevalence is a count out of n = 822. For each count I computed the 95% interval three ways: the Clopper–Pearson exact interval, the Wilson interval, and the Wald interval p ± 1.96·√(p(1−p)/n). I also computed the Wald interval with its variance inflated by the paper's stated design effect of 1.1. I then tested one candidate arithmetic route: take 1.96·SE on the proportion scale and add it, unconverted, to the prevalence in percent. All numbers come from recompute.py.txt.

The counts are internally consistent: 34 + 14 + 2 + 1 = 51 positives, 51 + 771 = 822, and 14 + 2 = 16 for HPV16 or 18. Every reported point estimate equals its count divided by 822 (for 16/822 = 1.946% the paper prints 1.94%, truncated rather than rounded, which does not matter here).

Quantity Location Reported Recomputed from the paper's counts
Any HPV, 51/822 Abstract; Results p2 6.2% (95% CI 6.18–6.22) 6.204%; exact 4.65–8.08; Wilson 4.75–8.07; Wald 4.56–7.85; Wald with design effect 1.1 4.47–7.93
HPV16 or 18, 16/822 Abstract; Results p2 1.94% (1.93–1.95) 1.946%; exact 1.12–3.14; Wilson 1.20–3.14
12 pooled high-risk types, 34/822 Abstract; Results p2; Table 1 4.14% (4.13–4.15) 4.136%; exact 2.88–5.73; Wilson 2.97–5.72
HPV16, 14/822 Table 1 1.7% (1.69–1.71) 1.703%; exact 0.93–2.84; Wilson 1.02–2.84
HPV18, 2/822 Table 1 0.24% (0.23–0.25) 0.243%; exact 0.03–0.88; Wilson 0.07–0.88
HPV16 & 12 pooled, 1/822 Table 1 0.12% (0.1–0.13) 0.122%; exact 0.00–0.68; Wilson 0.02–0.69
Half-width of the overall interval derived 0.02 points 1.65 points (Wald), 82 times the reported half-width; the exact interval is 86 times as wide as the reported one
How much too narrow, genotype rows (exact width ÷ reported width) derived — HPV16 or 18: 101×; 12 pooled: 143×; HPV16: 95×; HPV18: 42×; HPV16 & 12 pooled: 22×
Sample size a ±0.02-point interval would need at 6.2% derived — about 5.59 million
Population size at which a finite-population correction would give ±0.02 points derived — 822.1, i.e. the sample itself
Percent prevalence ± 1.96·SE in proportion units, centred on the reported rounded prevalence derived 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71; 0.23–0.25; 0.1–0.13 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71 (all four exact matches); 0.24–0.24; 0.12–0.12
Sample-size calculation Methods p2 479; 915 1.96²·0.033·0.967/0.016² = 478.9; 479·1.1/(1−0.424) = 914.8 (consistent)
Planned precision vs achieved Methods p2 (derived) ±1.6 points (d = 0.016) at an assumed 3.3% The 95% half-width at the observed 6.2% with n = 822 is 1.65 points, close to the planned ±1.6 points and about 80 times the reported 0.02

The paper's own sample-size calculation targeted a precision of ±1.6 percentage points (d = 0.016). The correct half-width at the observed prevalence, 1.65 points, is close to the planned ±1.6 points (the plan assumed 3.3% and n = 479, so the two need not coincide). The reported interval is roughly 80 times narrower than the precision the study was designed to achieve.

The arithmetic route above reproduces four of the six reported intervals exactly: 6.18–6.22, 1.93–1.95, 4.13–4.15 and 1.69–1.71. For the two smallest counts (2 and 1 positives), that route gives half-widths of 0.0034 and 0.0024 points. The paper reports 0.23–0.25 and 0.1–0.13, half-widths of 0.010 and 0.015 points, about 3 and 6 times what that route gives, so the route does not reproduce these two rows. They are still 42 and 22 times narrower than the exact intervals.

(An aside that does not affect the finding: the parenthetical "3.3/100 × 2" evaluates to 0.066, not 0.016. The stated 479 follows from d = 0.016, which is close to 3.3/100 ÷ 2 = 0.0165.)

Figure

Reported vs exact 95% intervals for each prevalence in the paper

Figure 1. Each row is one prevalence reported in the Abstract, the Results or Table 1, with its count out of 822. Orange diamonds show the paper's reported estimate and 95% CI; every reported interval is narrower than its marker, so its bar is hidden. Blue circles with capped bars show the exact (Clopper–Pearson) 95% CI computed from the same count. The labels to the right give each interval, marked "exact" or "reported". The dashed gray lines mark the 2009 values that the Discussion compares against (3.3%, 2.1% and 1.2%). The 1.2% value for HPV16 or 18 lies inside the exact interval for this study. The axis starts at zero and is linear.

Arithmetic chain for the overall interval

Figure 2. Arithmetic for the overall interval. Start from 51/822, compute the standard error and the 95% half-width (0.0165 on the proportion scale), then express the interval in percent. Multiplying the half-width by 100 gives 6.20% ± 1.65 points (blue, 4.56%–7.85% by the Wald method). Adding 0.0165 unconverted to the printed 6.2% gives 6.1835%–6.2165%, which rounds to 6.18%–6.22% (orange), the reported interval to the second decimal place. The same step reproduces the reported intervals for HPV16 or 18, the 12 pooled types and HPV16 (Table above).

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Replace the prevalence intervals with intervals computed from the counts. Exact (Clopper–Pearson) 95% CIs, which a design effect of 1.1 would widen slightly, are:

Apply these in the Abstract, Results paragraph 2 and Table 1. If the authors prefer a design-based interval, the Wald interval with the stated design effect for the overall prevalence is 4.47%–7.93%. The Discussion's statement that HPV16/18 prevalence rose from 1.2% to 1.94% should note that 1.2% lies within the corrected interval for this study. The same intervals in the Abstract, Results and Table 4 of the 2021 BMC Public Health article from this survey (doi:10.1186/s12889-021-10190-4) would need the same correction.

Recomputation script (python recompute.py.txt)
# Recomputation for PMC12211446 (BMC Med 2025, doi:10.1186/s12916-025-04246-0)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the counts printed in the paper
# (Results paragraph 2, Table 1); nothing else is assumed.
# Paragraph numbers count from 1 (show.py's [sec:X#pK] tags are 0-based, so #p1 = paragraph 2).
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
N = 822                       # participants tested (Results p2, Table 1 total)
Z = stats.norm.ppf(0.975)     # 1.95996
DEFF = 1.1                    # design effect stated in Methods p2

# label, count, reported prevalence (%), reported 95% CI (%), where reported
ROWS = [
    ("Any HPV", 51, 6.2, (6.18, 6.22), "Abstract; Results p2"),
    ("HPV16 or 18", 16, 1.94, (1.93, 1.95), "Abstract; Results p2"),
    ("12 pooled hrHPV", 34, 4.14, (4.13, 4.15), "Abstract; Results p2; Table 1"),
    ("HPV16", 14, 1.7, (1.69, 1.71), "Table 1"),
    ("HPV18", 2, 0.24, (0.23, 0.25), "Table 1"),
    ("HPV16 & 12 pooled", 1, 0.12, (0.10, 0.13), "Table 1"),
]


def wald(x, n, deff=1.0):
    p = x / n
    se = np.sqrt(deff * p * (1 - p) / n)
    return 100 * (p - Z * se), 100 * (p + Z * se)


def wilson(x, n):
    p = x / n
    den = 1 + Z**2 / n
    c = (p + Z**2 / (2 * n)) / den
    h = Z * np.sqrt(p * (1 - p) / n + Z**2 / (4 * n * n)) / den
    return 100 * (c - h), 100 * (c + h)


def clopper_pearson(x, n):
    lo = stats.beta.ppf(0.025, x, n - x + 1) if x > 0 else 0.0
    hi = stats.beta.ppf(0.975, x + 1, n - x) if x < n else 1.0
    return 100 * lo, 100 * hi


def unit_slip(x, n):
    """Percent prevalence +/- 1.96*SE where the SE is left on the proportion scale."""
    p = x / n
    hw = Z * np.sqrt(p * (1 - p) / n)          # proportion units
    return 100 * p - hw, 100 * p + hw, hw


print("=== 1. Counts")
print(f"34 + 14 + 2 + 1 = {34 + 14 + 2 + 1} positives; 51 + 771 = {51 + 771} total")
print(f"HPV16 or 18 = 14 + 2 = {14 + 2}")

print("\n=== 2. Point estimates and 95% CIs (percent), n = 822")
res = {}
for lab, x, prep, (lo, hi), loc in ROWS:
    p = 100 * x / N
    w, wd, wi, cp = wald(x, N), wald(x, N, DEFF), wilson(x, N), clopper_pearson(x, N)
    s_lo, s_hi, s_hw = unit_slip(x, N)
    rep_hw = (hi - lo) / 2
    wald_hw = (w[1] - w[0]) / 2
    res[lab] = dict(x=x, p=p, rep=(lo, hi), cp=cp, wil=wi, slip=(s_lo, s_hi))
    print(f"{lab:18s} {x:2d}/822 = {p:.3f}% (reported {prep}) | reported CI {lo:.2f}-{hi:.2f} (half-width {rep_hw:.3f} pp)")
    print(f"{'':18s} Wald {w[0]:.2f}-{w[1]:.2f} | Wald x DEFF 1.1 {wd[0]:.2f}-{wd[1]:.2f} | "
          f"Wilson {wi[0]:.2f}-{wi[1]:.2f} | Clopper-Pearson {cp[0]:.2f}-{cp[1]:.2f}")
    print(f"{'':18s} Wald half-width {wald_hw:.2f} pp = {wald_hw / rep_hw:.0f} x reported half-width")
    print(f"{'':18s} percent p +/- 1.96*SE(proportion units, {s_hw:.4f}): {s_lo:.3f}-{s_hi:.3f}"
          f" -> rounded {s_lo:.2f}-{s_hi:.2f}")
    r_lo, r_hi = prep - s_hw, prep + s_hw
    match = (round(r_lo, 2), round(r_hi, 2)) == (lo, hi)
    res[lab]["slip_rep"] = (r_lo, r_hi)
    print(f"{'':18s} same, centred on the reported (rounded) {prep}: {r_lo:.4f}-{r_hi:.4f}"
          f" -> rounded {r_lo:.2f}-{r_hi:.2f}; equals reported CI: {match}")
    print(f"{'':18s} reported half-width / route half-width (pp): {rep_hw:.3f} / {s_hw:.4f} = {rep_hw / s_hw:.1f}")
    print(f"{'':18s} exact CI width {cp[1] - cp[0]:.2f} pp = {(cp[1] - cp[0]) / (hi - lo):.0f} x reported width")

print("\n=== 3. How the overall interval compares")
p = 51 / N
se = np.sqrt(p * (1 - p) / N)
print(f"SE of 51/822 = {se:.5f} as a proportion = {100 * se:.3f} percentage points")
print(f"95% half-width = {100 * Z * se:.3f} pp; reported half-width 0.02 pp; ratio {100 * Z * se / 0.02:.1f}")
print(f"SE alone / 0.02 = {100 * se / 0.02:.1f}")
cp51 = clopper_pearson(51, N)
print(f"Exact (Clopper-Pearson) half-width = {(cp51[1] - cp51[0]) / 2:.2f} pp")
n_req = p * (1 - p) * (Z / 0.0002) ** 2
print(f"Sample size a +/-0.02 pp Wald interval would need: {n_req:,.0f}")
f = (0.02 / (100 * Z * se)) ** 2           # required finite-population factor (N-n)/(N-1)
Npop = (N - f) / (1 - f)
print(f"Finite-population factor needed: (Npop-n)/(Npop-1) = {f:.5f} -> Npop = {Npop:.1f}")
print(f"Inverse-variance weight implied by reported CI vs Wald CI: {(100 * Z * se / 0.02) ** 2:,.0f} x")

print("\n=== 4. Sample-size calculation in Methods p2")
pp, d = 0.033, 0.016
n0 = 1.96**2 * pp * (1 - pp) / d**2
print(f"1.96^2*0.033*0.967/0.016^2 = {n0:.1f} (reported 479)")
print(f"479*1.1/(1-0.424) = {479 * 1.1 / (1 - 0.424):.1f} (reported 915)")
print(f"Planned 95% half-width at 3.3%: {100 * d:.1f} pp; at observed 6.2%, n=822: {100 * Z * se:.2f} pp")
print(f"(The parenthetical '3.3/100 x 2' evaluates to {3.3 / 100 * 2:.3f}; 0.016 is close to 3.3/100/2 = {3.3 / 100 / 2:.4f})")

print("\n=== 5. Discussion comparisons with 2009 values (one-sample exact binomial, two-sided;")
print("       treats the 2009 figure as fixed, so it ignores that study's own sampling error)")
for lab, x, ref in [("Any HPV vs 3.3%", 51, 0.033), ("HPV16 or 18 vs 1.2%", 16, 0.012),
                    ("12 pooled vs 2.1%", 34, 0.021)]:
    t = stats.binomtest(x, N, ref)
    cp = clopper_pearson(x, N)
    inside = cp[0] <= 100 * ref <= cp[1]
    print(f"{lab:22s} {x}/822 = {100 * x / N:.2f}%; exact CI {cp[0]:.2f}-{cp[1]:.2f}; "
          f"2009 value inside exact CI: {inside}; p = {t.pvalue:.4f}"
          f"{' (< 1e-4: ' + format(t.pvalue, '.1e') + ')' if t.pvalue < 1e-4 else ''}")

# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
                     "axes.edgecolor": INK2, "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
    plt.close(fig)


# Fig 1: dot-and-interval, reported vs exact CI, one linear axis in percent
refs2009 = {"Any HPV": 3.3, "HPV16 or 18": 1.2, "12 pooled hrHPV": 2.1}
order = ["Any HPV", "12 pooled hrHPV", "HPV16 or 18", "HPV16", "HPV18", "HPV16 & 12 pooled"]
fig, ax = plt.subplots(figsize=(7.2, 5.2))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
for i, lab in enumerate(order):
    r = res[lab]
    yb, yo = i - 0.17, i + 0.17
    ax.plot(r["cp"], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
    for xv in r["cp"]:
        ax.plot([xv, xv], [yb - 0.09, yb + 0.09], color=BLUE, lw=2)
    ax.plot(r["p"], yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
    ax.plot(r["rep"], [yo, yo], color=ORANGE, lw=2)
    ax.plot(r["p"], yo, "D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
    ax.text(r["cp"][1] + 0.15, yb, f"exact {r['cp'][0]:.2f}–{r['cp'][1]:.2f}", va="center",
            fontsize=10, color=INK)
    ax.text(r["cp"][1] + 0.15, yo, f"reported {r['rep'][0]:.2f}–{r['rep'][1]:.2f}", va="center",
            fontsize=10, color=INK2)
    if lab in refs2009:
        v = refs2009[lab]
        ax.plot([v, v], [i - 0.38, i + 0.38], color=INK2, lw=1, ls=(0, (2, 2)))
        ax.text(v, i - 0.42, f"2009: {v}%", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n({res[lab]['x']}/822)" for lab in order])
ax.invert_yaxis()
ax.set_xlim(-0.15, 11.2)
ax.set_xticks(range(0, 11, 2))
ax.spines["bottom"].set_bounds(0, 10)
ax.set_xlabel("Prevalence, % (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "-D", color=ORANGE, lw=2, ms=8, label="Reported in the paper (95% CI; bars hidden behind the markers)")[0]
h2 = ax.plot([], [], "-o", color=BLUE, lw=2, ms=8, label="Exact (Clopper–Pearson) 95% CI recomputed from the paper's counts")[0]
h3 = ax.plot([], [], color=INK2, lw=1, ls=(0, (2, 2)), label="2009 value cited in the Discussion")[0]
ax.legend(handles=[h1, h2, h3], loc="upper left", bbox_to_anchor=(-0.05, -0.15), ncol=1,
          frameon=False, fontsize=10)
ax.set_title("Reported vs exact 95% intervals for the same counts (n = 822)",
             loc="left", fontsize=11, color=INK)
fig.tight_layout()
save(fig, "fig1")

# Fig 2: arithmetic chain for the overall interval; the divergent step highlighted
hw_prop = Z * se
fig = plt.figure(figsize=(7.2, 3.6))
ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 65)
ax.axis("off")


def box(x, y, w, h, title, body, edge=INK2, lw=1.0):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                fc=BG, ec=edge, lw=lw))
    ax.text(x + w / 2, y + h - 2.6, title, ha="center", va="top", fontsize=10, color=INK2)
    ax.text(x + w / 2, y + h / 2 - 2.2, body, ha="center", va="center", fontsize=10.5, color=INK)


def arrow(p0, p1, color=INK2, lw=1.2):
    ax.add_patch(FancyArrowPatch(p0, p1, arrowstyle="-|>", mutation_scale=12, color=color, lw=lw))


box(1, 44, 29, 15, "Counts (Results p2)", f"51 positive / 822\np = {p:.5f}")
box(35.5, 44, 29, 15, "SE (proportion)", f"√(p(1−p)/n) = {se:.5f}")
box(70, 44, 29, 15, "Half-width (proportion)", f"1.96 × SE = {hw_prop:.4f}")
arrow((30.6, 51.5), (35, 51.5))
arrow((65.1, 51.5), (69.5, 51.5))
# branch
box(3, 6, 42, 24, "Same units: convert half-width to %",
    f"{hw_prop:.4f} × 100 = {100 * hw_prop:.2f} pp\n6.20% ± {100 * hw_prop:.2f} pp\n→ {wald(51, N)[0]:.2f}% to {wald(51, N)[1]:.2f}%",
    edge=BLUE, lw=2)
box(55, 6, 42, 24, "Mixed units: proportion added to %",
    f"6.2% ± {hw_prop:.4f}\n→ {res['Any HPV']['slip_rep'][0]:.2f}% to {res['Any HPV']['slip_rep'][1]:.2f}%\n(paper: 6.18% to 6.22%)",
    edge=ORANGE, lw=2)
arrow((84.5, 43.3), (76, 31), color=ORANGE, lw=2)
arrow((84.5, 43.3), (24, 31), color=BLUE, lw=2)
ax.text(83, 36, "factor of 100\nlost here", ha="left", va="center",
        fontsize=10, color=INK)
ax.text(24, 2.0, "Recomputed (blue)", ha="center", fontsize=10, color=INK2)
ax.text(76, 2.0, "Reproduces the reported interval (orange)", ha="center", fontsize=10, color=INK2)
ax.text(1, 62.5, "How the overall 95% CI of 6.18%–6.22% is reproduced", ha="left", va="center",
        fontsize=11, color=INK)
save(fig, "fig2")
print("\nFigures written to", HERE)

Abstract and conclusions call the 57.4% concordant trials discordant; discordance was 42.6%

BMC Med (2025) · doi:10.1186/s12916-025-04328-z · PMC12403310

Summary

The study compares the data sharing plans that 376 clinical trials stated in their registrations with those stated in their publications, and its Abstract (Results and Conclusions) and main Conclusions section say that more than half of the trials were discordant, citing "216/376, 57.4%". By the paper's own Methods definitions and Table 2 (155 Yes/Yes, 13 Yes/No, 147 No/Yes, 61 No/No), those 216 trials (57.4%) are the concordant ones, and 160/376 (42.6%, 95% CI 37.7–47.6%) are discordant, as the Results text, Table 3, Supplementary Table S6 and the Discussion ("over 40% were discordant") also state. Concordance is one of the study's two stated outcomes, so a reader of the abstract alone gets its headline figure the wrong way round: a minority of trials, not a majority, had discordant plans. No test result or odds ratio changes; the correction is to three sentences.

What the paper reports

Abstract, Results:

Over half (216/376, 57.4%) had discordance between registration and publications.

Abstract, Conclusions:

More than half of published trials have data sharing discordance between registration and publication.

Main text, Conclusions section:

More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications.

The rest of the paper uses the same 216 trials as the concordant group.

Methods, "Study outcomes":

Subsequently, data sharing concordance was assessed: (1) plan to share data in registration and publication (both Yes in registration and publication, i.e., “Yes/Yes”) and (2) no plan to share data in registration and publication (both No in registration and publication, i.e., “No/No”). Discordance between registered and published plans to share data included (1) plan to share data in registration but no plan to share data in the publication (Yes in registration but No in publication, i.e., “Yes/No”) and (2) no plan to share in registration but a plan to share in publication (No in registration but Yes in publication, i.e., “No/Yes”).

Results, "Concordance between registered and published plans to share data":

A total of 376 trials were included for assessing the concordance, among which 216 (57.4%) had concordance between registered and published plans to share data (n = 155 and 61 for Yes/Yes and No/No, respectively; Table 2). For the discordance, there were 13 (3.5%) and 147 (39.1%) trials as Yes/No and No/Yes, respectively.

Table 2 (registration rows × publication columns): Yes/Yes 155 (41.2%), Yes/No 13 (3.5%), No/Yes 147 (39.1%), No/No 61 (16.2%); total 376 (100%).

Table 3 column headers: "Concordance between registered and published plans to share data", with sub-columns "(n = 160)" and "Yes(n = 216)". The archived text has no label in front of the first sub-column; the counts identify it as the discordant group.

Discussion, paragraph 1:

It was found that more than half of trials published in high-profile journals did not plan to share data in registration, and over 40% were discordant between registration and publication plans.

Discussion, paragraph 4:

Despite the ICMJE requirement, we found discordant data sharing plans in over 40% of trials, a finding that requires attention to increase consistency and reliability.

All quotes above were checked verbatim against the PMC-archived article text.

What the paper's own numbers show

Method. I took the four cell counts from Table 2 and applied the Methods definitions: concordant = Yes/Yes + No/No, discordant = Yes/No + No/Yes. I gave each share a Wilson 95% confidence interval. I re-ran the McNemar test on the two discordant cells as a consistency check. I then applied the same definitions to the two sensitivity tables in Additional file 1 (Table S7a, 78 undecided/missing registrations removed; Table S7b, those registrations counted as "Yes"). All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
Trials assessed for concordance Results; Table 2 376 155 + 13 + 147 + 61 = 376
Concordant (Yes/Yes + No/No) Results; Table 2; Table 3 "Yes" 216 (57.4%) "had concordance" 155 + 61 = 216/376 = 57.4% (95% CI 52.4–62.3%)
Discordant (Yes/No + No/Yes) Results; Table 2; Table 3 13 (3.5%) + 147 (39.1%); Table 3 n = 160 13 + 147 = 160/376 = 42.6% (95% CI 37.7–47.6%)
Share discordant Abstract, Results "Over half (216/376, 57.4%) had discordance" 160/376 = 42.6%; 216/376 is the concordant share
Is discordance more than half? Abstract Conclusions; main Conclusions "More than half" No: 160 < 188 (half of 376); the CI upper bound, 47.6%, is below 50%
Discordance Discussion (twice) "over 40%" 42.6%, consistent
Overstatement in abstract Abstract vs Table 2 57.4% 57.4 − 42.6 = 14.8 percentage points
Concordance, undecided removed Results; Table S7a 64.7% 193/298 = 64.8%; discordant 105/298 = 35.2% (30.0–40.8%)
Concordance, undecided as Yes Results; Table S7b 65.9% 248/376 = 66.0%; discordant 128/376 = 34.0% (29.4–39.0%)
McNemar test, 13 vs 147 Results P < 0.001 χ² = 112.2 (110.6 with continuity correction), P ≈ 3e-26; exact P ≈ 7e-30
Registered plan to share Abstract 44.6% (171/383) 171/383 = 44.6%

Every Table 2 cell and margin percentage reproduces exactly. The sensitivity figures differ from the recomputed ones by 0.1 point because the paper sums its printed component percentages (52.0% + 12.7% and 55.8% + 10.1%), two of which are rounded down (38/298 = 12.75%, printed 12.7%; 210/376 = 55.85%, printed 55.8%). This difference does not affect the finding. Only the abstract and conclusion sentences disagree with the data.

Figure

Abstract and conclusions label the 216 concordant trials of Table 2 as discordant

Figure 1. On the left (orange) are the Abstract Results and Conclusions sentences, which call 216/376 (57.4%) of trials discordant; the main Conclusions section repeats this. On the right is the paper's Table 2, redrawn as a 2 × 2 grid (rows: plan to share in the registration; columns: plan to share in the publication). Cells with solid blue borders are concordant by the Methods definition: Yes/Yes 155 + No/No 61 = 216 (57.4%). Cells with dashed borders are discordant: Yes/No 13 + No/Yes 147 = 160 (42.6%). The Results text describes the same 216 trials as having "concordance". The arrow links the abstract's 216 to the Table 2 total it equals, the concordant sum.

Share of trials with discordant plans: reported versus recomputed

Figure 2. Share of trials whose registered and published data sharing plans disagree, on a full 0–100% axis. Orange: the value given in the abstract and conclusions (57.4%, "over half"). Blue: values recomputed from the paper's own counts, with Wilson 95% confidence intervals. The rows are the main analysis (Table 2, 160/376) and the two sensitivity analyses in Additional file 1 (Table S7a, 105/298; Table S7b, 128/376). The thin vertical line marks half of the trials. Every recomputed estimate and confidence interval lies below that line. The only value above it is the reported one, and that figure is the concordant share.

Why it matters

Concordance between registered and published data sharing plans is one of the study's two stated outcomes, and the abstract and conclusions give its central figure the wrong way round. A reader of the abstract alone learns that a majority (57.4%) of trials in six high-profile journals changed their data sharing plans between registration and publication. By the paper's own counts, a majority (57.4%) were consistent and 42.6% were discordant. The 95% interval for discordance, 37.7–47.6%, excludes 50%, so the error is not a matter of sampling uncertainty. The size of the problem is overstated by about 15 percentage points and pushed across the majority threshold. Abstract and conclusion sentences are the parts most likely to be quoted in later literature and in policy discussion of ICMJE data sharing requirements.

The following results are not affected: the 44.6% (171/383) of registrations with a plan to share; the drug-intervention odds ratio (2.71); the COVID-19 odds ratio for concordance (0.59), which was modelled with the correct groups (Table 3, n = 216 vs n = 160); the McNemar test; the content-level discordance among the 155 Yes/Yes trials; and the sensitivity analyses. The broader message, that a substantial share of trials report inconsistent data sharing plans, still holds at 42.6%. The Discussion already states it correctly ("over 40%").

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Abstract, Results. Replace "Over half (216/376, 57.4%) had discordance between registration and publications." with:

Over half (216/376, 57.4%) had concordance between registration and publications, and 160/376 (42.6%) had discordance.

Abstract, Conclusions. Replace "More than half of published trials have data sharing discordance between registration and publication." with:

More than 40% of published trials have data sharing discordance between registration and publication.

Main Conclusions section. Replace "More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications." with:

More than 40% of published trials (160/376, 42.6%) have discordance between data sharing plans that they submit in trial registration versus publications.

The corrected wording matches the Discussion's own statement ("over 40% were discordant"). Optionally, the unlabeled first sub-column of Table 3 could be headed "No" to match "Yes(n = 216)".

Recomputation script (python recompute.py.txt)
"""Recompute the registration-vs-publication data sharing concordance in PMC12403310
(BMC Med 2025; doi:10.1186/s12916-025-04328-z) from the paper's own printed counts,
and redraw fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy, scipy, matplotlib)
Inputs are copied from the published article (Abstract, Methods, Results, Table 2,
Table 3, Discussion) and from Additional file 1 (Tables S6 and S7).
"""
import os
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
# Table 2: rows = plan to share in registration, columns = plan to share in publication
YY, YN, NY, NN = 155, 13, 147, 61
N_REP = 376
ABSTRACT_K, ABSTRACT_PCT = 216, 57.4          # "Over half (216/376, 57.4%) had discordance"
# Additional file 1, Table S7 (same cell order: Yes/Yes, Yes/No, No/Yes, No/No)
S7A = (155, 13, 92, 38)   # after removing 78 trials with undecided/missing registered plans
S7B = (210, 36, 92, 38)   # undecided/missing registered plans treated as "Yes"
S7_REPORTED_CONC = {"S7a": 64.7, "S7b": 65.9}  # Results, sensitivity paragraph
REG_YES, REG_N = 171, 383                      # Abstract: 44.6% (171/383)


def pct(k, n):
    return 100.0 * k / n


def wilson(k, n, z=1.959964):
    p = k / n
    d = 1 + z**2 / n
    c = (p + z**2 / (2 * n)) / d
    h = z * np.sqrt(p * (1 - p) / n + z**2 / (4 * n**2)) / d
    return 100 * (c - h), 100 * (c + h)


print("=== 1. Table 2 cells and margins ===")
N = YY + YN + NY + NN
print(f"N = {N} (reported {N_REP})")
for lab, k in [("Yes/Yes", YY), ("Yes/No", YN), ("No/Yes", NY), ("No/No", NN)]:
    print(f"  {lab:8s} {k:4d}  {pct(k, N):5.1f}%")
print(f"  registration Yes {YY+YN} ({pct(YY+YN, N):.1f}%), No {NY+NN} ({pct(NY+NN, N):.1f}%)")
print(f"  publication  Yes {YY+NY} ({pct(YY+NY, N):.1f}%), No {YN+NN} ({pct(YN+NN, N):.1f}%)")

print("\n=== 2. Concordance and discordance (Methods definitions) ===")
conc, disc = YY + NN, YN + NY
lo_c, hi_c = wilson(conc, N)
lo_d, hi_d = wilson(disc, N)
print(f"concordant  = Yes/Yes + No/No = {YY} + {NN} = {conc}/{N} = {pct(conc, N):.1f}%  "
      f"(Wilson 95% CI {lo_c:.1f} to {hi_c:.1f})")
print(f"discordant  = Yes/No + No/Yes = {YN} + {NY} = {disc}/{N} = {pct(disc, N):.1f}%  "
      f"(Wilson 95% CI {lo_d:.1f} to {hi_d:.1f})")
print(f"Abstract figure {ABSTRACT_K}/{N_REP} = {pct(ABSTRACT_K, N_REP):.1f}% "
      f"-> equals the concordant count: {ABSTRACT_K == conc}")
print(f"Is discordance more than half? {disc > N/2}  (half of {N} = {N/2:.0f})")
print(f"Discordance overstated by {ABSTRACT_PCT - round(pct(disc, N), 1):.1f} percentage points "
      f"(unrounded {pct(conc, N) - pct(disc, N):.2f})")
print(f"Discussion 'over 40% were discordant': {pct(disc, N):.1f}% > 40% -> {pct(disc, N) > 40}")
print("Table 3 column sizes printed: n = 160 and 'Yes' n = 216 "
      f"-> discordant {disc}, concordant {conc}")

print("\n=== 3. McNemar test on the discordant cells (reported P < 0.001) ===")
x2 = (YN - NY) ** 2 / (YN + NY)
x2cc = (abs(YN - NY) - 1) ** 2 / (YN + NY)
p_exact = stats.binomtest(YN, YN + NY, 0.5).pvalue
print(f"chi2 = {x2:.1f}, P = {stats.chi2.sf(x2, 1):.1e}; continuity-corrected chi2 = {x2cc:.1f}, "
      f"P = {stats.chi2.sf(x2cc, 1):.1e}; exact P = {p_exact:.1e}")

print("\n=== 4. Sensitivity analyses (Additional file 1, Table S7) ===")
sens = {}
for lab, cells in [("S7a", S7A), ("S7b", S7B)]:
    a, b, c, d = cells
    n = a + b + c + d
    cc, dd = a + d, b + c
    lo, hi = wilson(dd, n)
    sens[lab] = (dd, n, pct(dd, n), lo, hi)
    print(f"{lab}: N = {n}; concordant {cc} ({pct(cc, n):.1f}%; paper {S7_REPORTED_CONC[lab]}%"
          f"); discordant {dd} ({pct(dd, n):.1f}%, "
          f"Wilson 95% CI {lo:.1f} to {hi:.1f})")

print("\n=== 5. Registration-level figure (different denominator, consistent) ===")
print(f"{REG_YES}/{REG_N} = {pct(REG_YES, REG_N):.1f}% (reported 44.6%)")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
                     "svg.fonttype": "none", "font.family": "DejaVu Sans"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
    fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
    plt.close(fig)


# ---- fig1: abstract/conclusions vs Results/Table 2, with the 2x2 table drawn
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off"); ax.set_facecolor(BG)

# left panel: what the abstract and conclusions say
ax.add_patch(FancyBboxPatch((1.5, 6), 38.5, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
                            fc="white", ec=ORANGE, lw=1.5))
ax.text(3.5, 51.2, "Abstract (Results, Conclusions)\n+ main Conclusions section",
        fontsize=10.5, weight="bold", va="top")
ax.text(3.5, 41.5, "As reported", color=ORANGE, fontsize=10, weight="bold", va="top")
ax.text(3.5, 37, "“Over half (216/376, 57.4%)\nhad discordance between\nregistration and publications.”",
        fontsize=10, va="top", style="italic")
ax.text(3.5, 22.5, "“More than half of published\ntrials have data sharing\ndiscordance …”",
        fontsize=10, va="top", style="italic")
ax.text(3.5, 9.5, "Reads 216 (57.4%) as discordant", fontsize=10, color=SEC, va="center")

# right panel: Table 2 as a 2x2 grid (rows = registration, columns = publication)
x0, y0, cw, ch = 60, 14, 16, 12
ax.text(59, 53.2, "Results text + Table 2 (same 376 trials)", fontsize=10.5, weight="bold", va="top")
ax.text(x0 + cw, 47.6, "Publication: plan to share?", ha="center", fontsize=10, color=SEC)
ax.text(x0 + cw / 2, 40.5, "Yes", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 1.5 * cw, 40.5, "No", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 1.5 * ch, "Yes", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 0.5 * ch, "No", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 6.3, y0 + ch, "Registration: plan to share?", ha="center", va="center",
        rotation=90, fontsize=10, color=SEC)
cells = [(0, 1, "Yes/Yes", YY, True), (1, 1, "Yes/No", YN, False),
         (0, 0, "No/Yes", NY, False), (1, 0, "No/No", NN, True)]
for cx, cy, lab, k, is_conc in cells:
    ax.add_patch(Rectangle((x0 + cx * cw, y0 + cy * ch), cw, ch, fc="#e6eef9" if is_conc else "white",
                           ec=BLUE if is_conc else SEC, lw=2 if is_conc else 1,
                           ls="-" if is_conc else (0, (3, 2))))
    ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.62, f"{k}", ha="center", va="center",
            fontsize=12, weight="bold", color=BLUE if is_conc else TXT)
    ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.27, f"{lab} ({pct(k, N):.1f}%)",
            ha="center", va="center", fontsize=10, color=SEC)
ax.text(x0 + cw, 10, f"Concordant (solid): {YY} + {NN} = {conc} ({pct(conc, N):.1f}%)",
        ha="center", va="center", fontsize=10, color=BLUE, weight="bold")
ax.text(x0 + cw, 6.3, f"Discordant (dashed): {YN} + {NY} = {disc} ({pct(disc, N):.1f}%)",
        ha="center", va="center", fontsize=10, color=TXT)
ax.text(x0 + cw, 2.4, "Results: \u201c216 (57.4%) had concordance\u201d",
        ha="center", va="center", fontsize=10, color=SEC, style="italic")

# connector naming the conflict
arr = FancyArrowPatch((41.2, 33), (52.0, 10.6), connectionstyle="arc3,rad=-0.3",
                      arrowstyle="-|>", mutation_scale=14, lw=2, color=TXT)
ax.add_patch(arr)
ax.text(49.6, 41.5, "same 216 trials:\n\u201cdiscordance\u201d in\nthe abstract,\n\u201cconcordance\u201d in\nResults & Table 2",
        ha="center", va="center", fontsize=10, color=TXT)
save(fig, "fig1")

# ---- fig2: share of trials discordant, reported vs recomputed, 0-100% axis
rows = [
    ("Abstract / Conclusions\n(as reported)", ABSTRACT_PCT, None, ORANGE, "o"),
    (f"Table 2, main analysis\n({disc}/{N}, recomputed)", pct(disc, N), (lo_d, hi_d), BLUE, "o"),
    (f"Table S7a, undecided removed\n({sens['S7a'][0]}/{sens['S7a'][1]}, recomputed)",
     sens["S7a"][2], sens["S7a"][3:], BLUE, "s"),
    (f"Table S7b, undecided as Yes\n({sens['S7b'][0]}/{sens['S7b'][1]}, recomputed)",
     sens["S7b"][2], sens["S7b"][3:], BLUE, "s"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, ci, col, mk) in zip(ys, rows):
    if ci is not None:
        ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt")
    ax.plot(v, y, mk, color=col, ms=9, zorder=3)
    txt = f"{v:.1f}%" if ci is None else f"{v:.1f}% ({ci[0]:.1f}–{ci[1]:.1f})"
    ax.text((ci[1] if ci else v) + 2, y, txt, va="center", fontsize=10, color=col)
ax.axvline(50, color=SEC, lw=0.8)
ax.text(50.8, ys[0] + 0.5, "half of trials", fontsize=10, color=SEC, va="center")
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_xlabel("Trials with discordant registered vs published data sharing plans (%)")
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=8, label="reported in paper")
ax.plot([], [], "o", color=BLUE, ms=8, label="recomputed from the paper's own counts (Wilson 95% CI)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)

Abstract labels the age-and-sex adjusted all-cancer hazard ratio as the multivariable adjusted estimate

BMJ (2017) · doi:10.1136/bmj.j108 · PMC5266623

Summary

The abstract and the Discussion give the hazard ratio for death from all cancers combined, for the most distressed group (GHQ-12 score 7-12) against the least distressed (0-6), as "multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48". In the paper's own Table 5, 1.32 (1.18 to 1.48) is the age and sex adjusted estimate (4353 deaths, n=163 363), and the Results text and Figures 2 and 3 describe it the same way. The multivariable estimate in Table 5 is 1.26 (1.11 to 1.42) (3800 deaths, n=146 008). The other six hazard ratios in the same abstract sentence all equal Table 5 multivariable rows exactly, so only the all-cancer figure comes from the other row. The association stays positive and statistically significant, but the fully adjusted excess risk is 26% rather than 32%. The abstract does not show that adjustment lowered the all-cancer estimate, and the Discussion uses the mislabelled figure to call an earlier meta-analysis estimate "of markedly lower magnitude".

What the paper reports

Abstract, Results. The all-cancer estimate is the first of seven, all under one "After adjustment for ..." clause:

After adjustment for age, sex, education, socioeconomic status, body mass index (BMI), and smoking and alcohol intake, and with reverse causality (by left censoring) and missing data (by imputation) taken into account, relative to people in the least distressed group (GHQ-12 score 0-6), death rates in the most distressed group (score 7-12) were consistently raised for cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48) and cancers not related to smoking (1.45, 1.23 to 1.71), as well as carcinoma of the colorectum (1.84, 1.21 to 2.78), prostate (2.42, 1.29 to 4.54), pancreas (2.76, 1.47 to 5.19), oesophagus (2.59, 1.34 to 5.00), and for leukaemia (3.86, 1.42 to 10.5).

Discussion, "Comparison with other studies":

the aggregated result for depression and cancer for all malignancies across the 13 studies included unity (relative risk 1.12, 95% confidence interval 0.99 to 1.26). This was of markedly lower magnitude than our estimate between psychological distress and overall cancer (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).

Results, paragraph beginning "During a mean (SD) follow-up":

In the 16 studies in aggregate, after adjustment for age and sex, higher levels of distress were associated with a 32% greater risk of total cancer mortality (hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).

Table 5, "All cancers". The columns are deaths, number of people, distress score 0, 1-3, 4-6 and 7-12, P value for trend, and distress score 7-12 v 0-6:

Age and sex | 4353 | 163 363 | 1 (ref) | 0.99 (0.92 to 1.08) | 1.05 (0.91 to 1.20) | 1.31 (1.16 to 1.49) | <0.001 | 1.32 (1.18 to 1.48)

Multivariable† | 3800 | 146 008 | 1 | 0.95 (0.88 to 1.03) | 0.93 (0.78 to 1.10) | 1.22 (1.07 to 1.38) | 0.04 | 1.26 (1.11 to 1.42)

Table 5 footnote:

†Hazard ratio adjusted for age, sex (except in single sex analyses), BMI, educational attainment, smoking status, and frequency of alcohol consumption.

Methods, "Statistical analyses", which defines the two models:

Hazard ratios were minimally adjusted (age and sex only) and maximally adjusted (age, sex, BMI, educational attainment, smoking status, and frequency of alcohol consumption).

Figure 3 legend. In the typeset figure, the "All cancers combined" row reads 1.32 (1.18 to 1.48) under "adjusted for age and sex" and 1.26 (1.11 to 1.42) under "multivariable adjusted":

Hazard ratios are for psychological distress score of 7-12 (most distressed) relative to 0-6, and are age and—except in single sex analyses—sex adjusted (I2=15%), or multivariable adjusted (age, sex, BMI, educational attainment, smoking status, and alcohol consumption; I2=37%)

Figure 2 legend. The pooled "Overall" row of this forest plot is 4353 deaths, 1.32 (1.18 to 1.48):

Hazard ratios (adjusted for age and sex) are for psychological distress score of 7-12 (most distressed) relative to 0-6.

What the paper's own numbers show

Method. I copied the seven hazard ratios and 95% CIs from the abstract's Results sentence. For the same seven outcomes I copied both Table 5 rows, age (and sex) and multivariable, from the column "Distress score (7-12 v 0-6)". I then checked which row each abstract value equals, digit for digit. To show that the two all-cancer rows are separate estimates and not rounding variants of one number, I computed the geometric (log-scale) midpoint of each CI and the standard error of the log hazard ratio, (ln upper − ln lower)/(2 × 1.96). Finally, I repeated the Discussion's comparison with the earlier meta-analysis for each all-cancer estimate. That comparison is a z test on the difference of log ratios, treating the two estimates as independent. It is illustrative only: one estimate is a hazard ratio for distress and death, the other a relative risk for major depression and cancer occurrence. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
All cancers, labelled "multivariable adjusted" Abstract, Results 1.32 (1.18 to 1.48) Equals the Table 5 age and sex row in all three numbers. The Table 5 multivariable row is 1.26 (1.11 to 1.42)
All cancers, labelled "multivariable adjusted" Discussion, "Comparison with other studies" 1.32 (1.18 to 1.48) Same as above
All cancers, labelled "after adjustment for age and sex" Results text 1.32 (1.18 to 1.48) Equals the Table 5 age and sex row
All cancers, age and sex Table 5 1.32 (1.18 to 1.48); 4353 deaths; n=163 363; P for trend <0.001 CI log-midpoint 1.322; SE(log HR) 0.0578; z = 4.80
All cancers, multivariable Table 5 1.26 (1.11 to 1.42); 3800 deaths; n=146 008; P for trend 0.04 CI log-midpoint 1.255; SE(log HR) 0.0628; z = 3.68
Other six abstract hazard ratios (not related to smoking, colorectum, prostate, pancreas, oesophagus, leukaemia) Abstract vs Table 5 1.45, 1.84, 2.42, 2.76, 2.59, 3.86, each with its CI 6 of 6 equal the Table 5 multivariable rows exactly. None equals the age and sex rows (1.38, 1.75, 2.29, 2.52, 2.42, 2.89)
Effect of adjustment Table 5 — All cancers: 1.32 → 1.26 (down). The other six outcomes: all go up (for example leukaemia 2.89 → 3.86)
Excess all-cancer mortality, most v least distressed Abstract (implied) vs Table 5 multivariable 32% 26%. The reported excess is 1.23 times the multivariable excess, and the multivariable excess is 19% smaller
Comparison with the earlier meta-analysis, RR 1.12 (0.99 to 1.26) Discussion 1.32 / 1.12 = 1.18; excess 32% v 12% With 1.32: z = 1.95, P = 0.052. With 1.26: ratio 1.13, excess 26% v 12%, z = 1.34, P = 0.18

The two all-cancer estimates are each internally consistent: each printed hazard ratio sits at the log-midpoint of its own CI. They also come from different samples, with 553 deaths and 17 355 people fewer in the multivariable model. The paper states the reason. Results, paragraph beginning "Table 5 shows the four categories":

With different models being based on different analytical samples because of missing covariates, we recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged.

So the paper attributes the different sample sizes to missing covariates, and it reports that re-estimating both models in the same (non-missing) sample left the results "unchanged". It gives no figures for that analysis, so the share of the 1.32 v 1.26 gap due to the extra covariates rather than the change of sample cannot be read from the paper.

Figure

The abstract's "multivariable adjusted hazard ratio 1.32 (1.18 to 1.48)" is identical to Table 5's age and sex row; the multivariable row reads 1.26 (1.11 to 1.42)

Figure 1. Where the all-cancer estimate appears in the paper. Left: the abstract's wording, shortened at the ellipsis; the Discussion uses the same label and numbers. The estimate is in orange. Right: the two Table 5 rows for all cancers with their deaths and sample sizes. The solid orange arrow joins the abstract value to the Table 5 row that prints the identical numbers, which is the age and sex row. The dashed blue arrow joins the abstract's label, "multivariable adjusted", to the row that label names, which prints a different estimate, 1.26 (1.11 to 1.42). The quoted text and numbers are from the paper; the arrows and annotations are mine.

Dot-and-interval plot on a log scale of the seven abstract hazard ratios beside the Table 5 age and sex and multivariable estimates

Figure 2. The seven hazard ratios (95% CI, log scale) in the abstract's Results sentence (orange filled circles), shown with the Table 5 multivariable estimate (blue diamonds) and the Table 5 age and sex estimate (open gray circles) for the same outcome. The thin vertical line marks a hazard ratio of 1. The label on the right of each row says which Table 5 row the abstract value equals. For the six site-specific or subgroup outcomes it is the multivariable row. For all cancers combined it is the age and sex row. All plotted values are from the paper's abstract and Table 5 as printed.

Why it matters

Explanations considered

  1. The 1.32 comes from another multivariable analysis, such as the imputed-data or left-censored sensitivity analyses the abstract mentions. This does not account for the numbers. Those analyses are in the web appendices (appendix 1 table C; appendix 2 fig A), which are not deposited in PMC or Europe PMC and which I could not inspect. Four things in the paper argue against it. (a) The other six estimates in the same sentence, under the same "left censoring ... imputation" wording, equal the non-imputed Table 5 multivariable rows exactly. (b) The Results say the imputed meta-analysis "gave similar results to those seen with the non-missing dataset as reported in the present paper", which identifies the reported estimates as the non-imputed ones. (c) The Results say that in the left-censored analysis "most of the associations between distress and cancer were largely unaffected (fig A in appendix 2)", which presents it as a check rather than the source of the headline figure. (d) 1.32 (1.18 to 1.48) matches the Table 5 age and sex row in all three numbers, and the Results text and Figures 2 and 3 label it age and sex adjusted. Another model reproducing all three numbers to two decimals would be a coincidence. A related point: the abstract says all seven estimates take left censoring and imputation into account, but the Table 5 footnote, which defines the rows six of them come from, lists covariates only.
  2. A model with additional covariates (appendix 1 table B). I could not inspect that table. The Results say only that, in this analysis, "The strength of the relation between distress and total cancer was little changed", and give no figure for it. The abstract's covariate list is close to the main model (Table 5 footnote, Methods, Figure 3 legend) but not identical: the abstract names "socioeconomic status" as well as education, while those three places list educational attainment only. The Results describe the main model's covariates as including "socioeconomic position (education)", so the abstract's "socioeconomic status" corresponds to education in the main model. Table B's extra covariates do include one socioeconomic measure, area based deprivation, but the abstract does not mention the other three (physical activity, C reactive protein, quantity of alcohol consumed), and the six other estimates in the same sentence equal the main-model rows of Table 5.
  3. The same-sample re-analysis reported in the Results. The paper says it "recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged". This does not account for 1.32 being labelled multivariable. In that analysis the multivariable model is fitted to the complete-case sample, and the multivariable estimate in the complete-case sample is the 1.26 (1.11 to 1.42) already shown in Table 5 (3800 deaths, n=146 008). Refitting in the complete-case sample changes the age and sex estimate, not the multivariable one, so the re-analysis gives no route by which the multivariable estimate becomes 1.32 (1.18 to 1.48), the numbers of the full-sample age and sex row.
  4. Rounding, or two presentations of one estimate. Ruled out. The two CIs have log-scale midpoints of 1.322 and 1.255, each matching its own printed hazard ratio, and they come from different samples (4353 deaths in 163 363 people v 3800 deaths in 146 008).
  5. The error is in Table 5, not in the abstract. Not supported. The Table 5 all-cancer rows agree with the Results text, with Figure 2 (pooled age and sex estimate 1.32 (1.18 to 1.48), 4353 deaths) and with Figure 3 (1.32 under age and sex, 1.26 under multivariable).
  6. An artefact of the PMC text conversion. Ruled out. The same abstract and Discussion wording and the same Table 5 cells appear in the publisher's typeset PDF (see Status). No minus signs, superscripts or special decimal marks are involved.

Status

Suggested correction

  1. In the abstract's Results, replace "cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48)" with "cancer of all sites combined (multivariable adjusted hazard ratio 1.26, 95% confidence interval 1.11 to 1.42)". Alternatively, keep 1.32 (1.18 to 1.48) and label it "age and sex adjusted", which would also need the sentence's opening clause to be reworded.
  2. Make the same change in the Discussion, "Comparison with other studies", and reconsider describing the earlier meta-analysis estimate, 1.12 (0.99 to 1.26), as "of markedly lower magnitude".
  3. Clarify in the abstract whether the reported estimates are from the main (complete-case, not left-censored) analysis, as Table 5 and the Results indicate, rather than "with reverse causality (by left censoring) and missing data (by imputation) taken into account".
  4. Separately, reconcile the liver multivariable upper confidence limit in Figure 3 (8.52) with Table 5 (27.0).
Recomputation script (python recompute.py.txt)
"""Recompute the all-cancer hazard ratios of PMC5266623
(BMJ 2017;356:j108, doi:10.1136/bmj.j108).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PubMed Central:
  Abstract, Results: seven hazard ratios (GHQ-12 score 7-12 v 0-6), all introduced as
      adjusted for age, sex, education, SES, BMI, smoking and alcohol
  Table 5, last column "Distress score (7-12 v 0-6)": the "Age and sex" (or "Age") row and the
      "Multivariable" row for each outcome, with deaths and number of people
  Discussion, "Comparison with other studies": prior meta-analysis RR 1.12 (0.99 to 1.26)
"""
import math
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from scipy.stats import norm  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (from the paper)
# Table 5: outcome -> (minimal row (HR, lo, hi), multivariable row (HR, lo, hi))
# For prostate the minimal row is labelled "Age" (single-sex outcome).
T5 = {
    "All cancers":            ((1.32, 1.18, 1.48), (1.26, 1.11, 1.42)),
    "Not related to smoking": ((1.38, 1.19, 1.60), (1.45, 1.23, 1.71)),
    "Colorectal":             ((1.75, 1.21, 2.53), (1.84, 1.21, 2.78)),
    "Prostate":               ((2.29, 1.36, 3.86), (2.42, 1.29, 4.54)),
    "Pancreas":               ((2.52, 1.47, 4.32), (2.76, 1.47, 5.19)),
    "Oesophagus":             ((2.42, 1.36, 4.32), (2.59, 1.34, 5.00)),
    "Leukaemia":              ((2.89, 1.32, 6.31), (3.86, 1.42, 10.5)),
}
# Table 5, All cancers: deaths and number of people in each model
T5_ALL_N = {"age_sex": (4353, 163363), "multivariable": (3800, 146008)}

# Abstract, Results (the first is explicitly "multivariable adjusted hazard ratio")
ABSTRACT = {
    "All cancers":            (1.32, 1.18, 1.48),
    "Not related to smoking": (1.45, 1.23, 1.71),
    "Colorectal":             (1.84, 1.21, 2.78),
    "Prostate":               (2.42, 1.29, 4.54),
    "Pancreas":               (2.76, 1.47, 5.19),
    "Oesophagus":             (2.59, 1.34, 5.00),
    "Leukaemia":              (3.86, 1.42, 10.5),
}
# Discussion, "Comparison with other studies": the same all-cancer figure, "multivariable adjusted"
DISCUSSION_ALL = (1.32, 1.18, 1.48)
# Results p4: "after adjustment for age and sex ... (hazard ratio 1.32, 95% CI 1.18 to 1.48)"
RESULTS_P4_AGESEX = (1.32, 1.18, 1.48)
# Discussion: prior meta-analysis (depression and cancer incidence), "relative risk 1.12, 0.99 to 1.26"
PRIOR_META = (1.12, 0.99, 1.26)


def log_se(t):
    _, lo, hi = t
    return (math.log(hi) - math.log(lo)) / (2 * Z)


def midpoint(t):
    _, lo, hi = t
    return math.exp((math.log(lo) + math.log(hi)) / 2)


def num(x):
    return f"{x:.1f}" if x >= 10 else f"{x:.2f}"


def fmt(t):
    return f"{num(t[0])} ({num(t[1])} to {num(t[2])})"


print("=== 1. Which Table 5 row does each abstract hazard ratio come from? ===")
print(f"{'outcome':24s} {'abstract':20s} {'T5 age(/sex)':20s} {'T5 multivariable':20s} matches")
MATCH = {}
for k, v in ABSTRACT.items():
    a, m = T5[k]
    tag = "multivariable row" if v == m else ("AGE AND SEX ROW" if v == a else "neither")
    MATCH[k] = tag
    print(f"{k:24s} {fmt(v):20s} {fmt(a):20s} {fmt(m):20s} {tag}")
n_mv = sum(t == "multivariable row" for t in MATCH.values())
print(f"-> {n_mv} of {len(ABSTRACT)} abstract estimates equal the multivariable row; "
      f"the all-cancer estimate equals the age and sex row.")
print("Change from age(/sex) to multivariable estimate, by outcome:")
for k in ABSTRACT:
    a, m = T5[k]
    print(f"  {k:24s} {a[0]:.2f} -> {m[0]:.2f}  ({'up' if m[0] > a[0] else 'down'})")
print("Discussion figure equals Table 5 age and sex row:", DISCUSSION_ALL == T5["All cancers"][0])
print("Results p4 (age and sex) figure equals Table 5 age and sex row:",
      RESULTS_P4_AGESEX == T5["All cancers"][0])

print("\n=== 2. Are the two all-cancer rows separate, internally consistent estimates? ===")
AS, MV = T5["All cancers"]
for lab, t in (("age and sex", AS), ("multivariable", MV)):
    se = log_se(t)
    zval = math.log(t[0]) / se
    p = 2 * norm.sf(abs(zval))
    print(f"{lab:14s} HR {t[0]:.2f}: log SE {se:.4f}; geometric midpoint of CI {midpoint(t):.3f}; "
          f"z = {zval:.2f}; two-sided P ~ {p:.1e}")
print(f"deaths/people: age and sex {T5_ALL_N['age_sex']}, multivariable {T5_ALL_N['multivariable']}; "
      f"difference {T5_ALL_N['age_sex'][0] - T5_ALL_N['multivariable'][0]} deaths, "
      f"{T5_ALL_N['age_sex'][1] - T5_ALL_N['multivariable'][1]} people")

print("\n=== 3. Size of the overstatement ===")
ex_rep, ex_mv = AS[0] - 1, MV[0] - 1
print(f"excess risk: reported {ex_rep*100:.0f}% v multivariable {ex_mv*100:.0f}%; "
      f"reported excess is {ex_rep/ex_mv:.2f} x the multivariable excess; "
      f"multivariable excess is {(1 - ex_mv/ex_rep)*100:.0f}% smaller")
print(f"ratio of hazard ratios 1.32/1.26 = {AS[0]/MV[0]:.3f}")
print(f"lower CI bound: {AS[1]:.2f} v {MV[1]:.2f}; upper: {AS[2]:.2f} v {MV[2]:.2f}")

print("\n=== 4. Discussion contrast with the prior meta-analysis (illustrative only) ===")
# Naive two-sample z test on the log scale, treating the estimates as independent. The two
# studies differ in exposure (distress v major depression) and outcome (mortality v incidence);
# this only shows how much the contrast depends on which all-cancer figure is used.
se_prior = log_se(PRIOR_META)
print(f"prior meta-analysis RR 1.12: log SE {se_prior:.4f}; excess {(PRIOR_META[0]-1)*100:.0f}%")
for lab, t in (("as reported (1.32)", AS), ("multivariable (1.26)", MV)):
    d = math.log(t[0]) - math.log(PRIOR_META[0])
    se = math.sqrt(log_se(t) ** 2 + se_prior ** 2)
    zval = d / se
    print(f"{lab:22s}: ratio to 1.12 = {t[0]/PRIOR_META[0]:.3f}; excess {(t[0]-1)*100:.0f}% v 12% "
          f"({(t[0]-1)/(PRIOR_META[0]-1):.1f} x); z for difference = {zval:.2f}, "
          f"P = {2*norm.sf(abs(zval)):.3f}; CIs overlap from {max(t[1], PRIOR_META[1]):.2f} "
          f"to {min(t[2], PRIOR_META[2]):.2f}")

print("\n=== 5. Separate minor observation: liver, multivariable (Table 5 v typeset Figure 3) ===")
LIVER_T5_MV = (4.24, 0.66, 27.0)    # Table 5 (PMC and typeset PDF)
LIVER_F3_MV = (4.24, 0.66, 8.52)    # Figure 3 text in the typeset PDF
LIVER_T5_AS = (2.88, 0.97, 8.52)    # Table 5 age and sex row
for lab, t in (("Table 5 multivariable", LIVER_T5_MV), ("Figure 3 multivariable", LIVER_F3_MV),
               ("Table 5 age and sex", LIVER_T5_AS)):
    print(f"{lab:24s} {fmt(t):22s} geometric midpoint of CI {midpoint(t):.2f} (printed HR {t[0]})")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8d8c88"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": "#c9c8c4",
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# ---- Figure 1: abstract/discussion label v Table 5 rows
fig = plt.figure(figsize=(7.2, 4.7))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.7)
ax.axis("off")


def panel(x, y, w, h, title, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.14, y + h - 0.18, title, fontsize=11, fontweight="bold", va="top", color=TXT)


# Left panel: abstract and discussion
panel(0.15, 1.2, 2.95, 3.35, "Abstract, Results\n(repeated in the Discussion)", ORANGE)
ax.text(0.29, 3.62,
        "\u201cAfter adjustment for age, sex,\neducation, socioeconomic status,\nbody mass index (BMI), and\n"
        "smoking and alcohol intake \u2026\ncancer of all sites combined\n(multivariable adjusted hazard\nratio",
        fontsize=10, va="top", color=SEC, linespacing=1.3)
ax.text(0.29, 1.88, f"{AS[0]:.2f}, 95% CI {AS[1]:.2f} to {AS[2]:.2f})\u201d",
        fontsize=11.5, fontweight="bold", color=ORANGE, va="center")
ax.text(0.29, 1.5, "label: multivariable adjusted", fontsize=10, color=SEC, va="center", style="italic")

# Right panel: Table 5 rows
panel(4.0, 1.2, 3.05, 3.35, "Table 5, All cancers\n(score 7-12 v 0-6)", "#9b9a96")
rows = [
    (3.6, "Age and sex", f"{T5_ALL_N['age_sex'][0]} deaths, n = {T5_ALL_N['age_sex'][1]:,}".replace(",", " "),
     fmt(AS), ORANGE),
    (2.6, "Multivariable\u2020", f"{T5_ALL_N['multivariable'][0]} deaths, n = {T5_ALL_N['multivariable'][1]:,}"
     .replace(",", " "), fmt(MV), BLUE),
]
for y, lab, nn, val, col in rows:
    ax.text(4.14, y, lab, fontsize=10.5, fontweight="bold", va="center", color=TXT)
    ax.text(4.14, y - 0.27, nn, fontsize=10, va="center", color=SEC)
    ax.text(4.14, y - 0.57, val, fontsize=11.5, fontweight="bold", va="center", color=col)
ax.text(4.14, 1.5, "\u2020 age, sex, BMI, education,\n   smoking, alcohol", fontsize=10, color=SEC, va="center",
        linespacing=1.25)

# Connectors
ax.add_patch(FancyArrowPatch((3.14, 1.95), (3.98, 3.03), arrowstyle="-|>", mutation_scale=14,
                             color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
ax.text(3.52, 3.1, "identical\nvalue", fontsize=10, color=ORANGE, ha="center", va="center")
ax.add_patch(FancyArrowPatch((3.14, 1.5), (3.98, 2.03), arrowstyle="-|>", mutation_scale=14,
                             color=BLUE, lw=2, linestyle=(0, (4, 3))))
ax.text(3.55, 1.38, "row the\nlabel names", fontsize=10, color=BLUE, ha="center", va="top")

ax.text(0.15, 0.55,
        "The Results text gives the same 1.32 (1.18 to 1.48) \u201cafter adjustment for age and sex\u201d.\n"
        f"The other {n_mv} hazard ratios in the abstract all equal Table 5 multivariable rows (fig 2).",
        fontsize=10, color=SEC, va="center", linespacing=1.35)
save(fig, "fig1")

# ---- Figure 2: all seven abstract estimates against both Table 5 rows (log scale)
names = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.25, right=0.76, top=0.86, bottom=0.14)
off = {"abstract": 0.0, "mv": -0.24, "as": 0.24}
for i, k in enumerate(names):
    y = -i
    a, m = T5[k]
    v = ABSTRACT[k]
    for t, dy, col, mk, fc in ((a, off["as"], GRAY, "o", BG), (m, off["mv"], BLUE, "D", BLUE),
                               (v, off["abstract"], ORANGE, "o", ORANGE)):
        ax.plot([t[1], t[2]], [y + dy] * 2, color=col, lw=2, solid_capstyle="butt", zorder=2)
        ax.plot(t[0], y + dy, marker=mk, ms=8, mfc=fc, mec=col, mew=2, zorder=3, ls="none")
    tag = "= age and sex row" if MATCH[k] != "multivariable row" else "= multivariable row"
    ax.text(1.02, y, tag, fontsize=10, va="center", ha="left", transform=ax.get_yaxis_transform(),
            color=ORANGE if MATCH[k] != "multivariable row" else SEC,
            fontweight="bold" if MATCH[k] != "multivariable row" else "normal")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.9, 12)
ticks = [1, 1.5, 2, 3, 5, 10]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([-i for i in range(len(names))])
ax.set_yticklabels(names)
ax.set_ylim(-len(names) + 0.5, 0.5)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio, distress score 7-12 v 0-6 (log scale; 95% CI)")
h = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Abstract (“multivariable adjusted”)"),
     plt.Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Table 5, multivariable row"),
     plt.Line2D([], [], color=GRAY, marker="o", ms=8, mfc=BG, mew=2, lw=2, label="Table 5, age and sex row")]
fig.legend(handles=h, loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Primary-outcome 95% confidence interval is about 80 times too narrow for the reported failure counts

BMJ Glob Health (2024) · doi:10.1136/bmjgh-2023-014170 · PMC11331853

Summary

This non-inferiority trial compared spinal anaesthesia failure between trained medical officers (MOs) and consultant anaesthesiologists (CAs). It reports 7/231 (3.0%) versus 5/191 (2.6%) failures, a difference of 0.4 percentage points, with a 95% CI of 0.36–0.43% in the Abstract and Results and 0.36 to 0.44 in Table 3. From those counts, the 95% CI for the difference is −2.75 to 3.58 percentage points (Wald) or −3.29 to 3.85 (Newcombe). The printed interval is therefore about 79–90 times too narrow, and it excludes zero although the printed p = 0.7997, which the same counts reproduce exactly. The non-inferiority conclusion survives, because every recomputed upper bound stays below the 5-point margin. But the margin is cleared by about 1.2–1.4 points, not the 4.6 points the printed interval implies, and the emergent-case subgroup interval has the same defect.

What the paper reports

Abstract, Findings:

Spinal anaesthesia failure rate for MOs (7, 3.0%) was non-inferior to those of CA (5, 2.6%); difference in success rate of 0.4% (95% CI=0.36–0.43%; p=0.80).

Results, third paragraph (primary outcome):

There was no statistically significant difference in the primary outcome–delivery of spinal anaesthesia into the intrathecal space in three or fewer attempts (table 3). MOs were unsuccessful in 7 (3.0%) patients, while the CAs were unsuccessful in 5 (2.6%) patients, with a difference in success rate of 0.4% (CI=0.36–0.43%; p=0.7997).

Table 3, "Primary outcome measure: failure to deliver spinal anaesthesia". The columns are: outcome | N | Medical officers (n=231) | Consultant anaesthesiologists (n=191) | % between-group difference (95% CI) | P value. The footer reads "*Fisher’s exact test, otherwise χ2 test."

Failure to deliver spinal anaesthesia in all cases | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997

Failure to deliver spinal anaesthesia in emergent cases | 61 | 1 (3.6%) | 2 (6.1%) | -2.5 (-2.92 to -2.08) | 1.0000*

Failure due to inaccessibility of intrathecal space | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997

Table 2 gives the emergent-case denominators (columns: MO | CA | P value):

Emergency | 28 (12.2%) | 33 (17.3%) | 0.1435

Abstract, Methods, which defines the margin:

The primary outcome measure was spinal anaesthesia failure with a non-inferiority margin of 5%.

Methods, Statistical analysis, which names the analysis methods:

Patient characteristics, spinal anaesthesia failure rates and incidence of postoperative complications were analysed with T-test and Wilcoxon rank sum for continuous variables, and χ squared and Fisher’s exact tests for discrete variables.

The conclusions that rest on this result. Abstract, Interpretation:

This study demonstrates that failure rates of spinal anaesthesia care provided by trained MOs are non-inferior to care provided by CAs in low-risk surgical patients. This may support policy measures that use task-sharing as a means of expanding anaesthesia care capacity in rural Indian hospitals.

Discussion, fourth paragraph:

We believe this study may facilitate regional and national policy measures on the practical viability of task-sharing.

What the paper's own numbers show

Method. I took the failure counts and arm sizes from Table 3 (7/231 and 5/191) and the emergent-case denominators from Table 2 (28 and 33). Those emergent denominators are the only pair summing to 61 that reproduces the printed 3.6% and 6.1%. For each comparison I computed the difference in failure proportions (MO − CA) and its unpooled standard error, then the 95% CI by three standard methods for a difference of two independent proportions: Wald, Newcombe (hybrid Wilson score) and Agresti–Caffo. I also computed the uncorrected and Yates-corrected chi-square and the Fisher exact p-values, to check that the counts match the printed p-values. From the half-width of each printed interval I derived the standard error it implies under a normal approximation, and the p-value that standard error would imply. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
MO failure rate, all cases Abstract; Results; Table 3 7 (3.0%), n = 231 3.03%
CA failure rate, all cases Abstract; Results; Table 3 5 (2.6%), n = 191 2.62%
Difference, MO − CA Abstract; Results; Table 3 0.4 0.413 percentage points (pp)
p-value, all cases Table 3; Results (0.80 in Abstract) 0.7997 (χ²) 0.7997, uncorrected chi-square (Yates 1.0000; Fisher 1.0000)
95% CI of difference, all cases Abstract; Results 0.36–0.43% Wald −2.75 to 3.58 pp; Newcombe −3.29 to 3.85; Agresti–Caffo −3.06 to 3.71
95% CI of difference, all cases Table 3, rows 1 and 3 0.36 to 0.44 as above
Width of the 95% CI, all cases Table 3; Abstract 0.08 (Table 3); 0.07 (Abstract) 6.33 pp (Wald): 79 times the Table 3 width and 90 times the Abstract width
Standard error, all cases not reported 0.0204 pp, implied by the Table 3 interval 1.615 pp (Wald), 79 times larger
p-value implied by the printed interval, all cases derived z = 0.40 / 0.0204 = 19.6 p ≈ 1.6 × 10⁻⁸⁵, against the printed 0.7997
Emergent cases, rates and difference Table 3, row 2; Table 2 1 (3.6%) vs 2 (6.1%); −2.5 1/28 = 3.57%, 2/33 = 6.06%; −2.49 pp
95% CI, emergent cases Table 3, row 2 −2.92 to −2.08 Wald −13.14 to 8.17 pp; Newcombe −16.35 to 12.32; width 21.31 pp, 25 times the printed 0.84
p-value, emergent cases Table 3, row 2 1.0000 (Fisher) Fisher 1.0000; the printed interval would imply p ≈ 1.9 × 10⁻³¹
Upper 95% bound vs the 5 pp margin, all cases Abstract; Methods 0.43 or 0.44, leaving about 4.6 pp 3.58 (Wald), 3.71 (Agresti–Caffo), 3.85 (Newcombe), leaving 1.42, 1.29 and 1.15 pp; non-inferiority still holds
Upper bound of a two-sided 90% interval (one-sided 5% test) derived not reported 3.07 pp, leaving 1.93 pp

The counts and the p-value agree exactly, so the counts are not in doubt. Only the intervals are inconsistent with them. Both printed all-cases intervals lie entirely above zero, which would mean a significant difference in favour of CAs, while the stated p = 0.80 and the text's "no statistically significant difference" say the opposite. The emergent-case interval lies entirely below zero, which would mean MOs had significantly fewer failures, while its printed p is 1.0000.

Figure

Printed and recomputed 95% confidence intervals for the difference in spinal anaesthesia failure rate (MO − CA), all cases and emergent cases, on one percentage-point axis

Each row shows a point estimate and its 95% confidence interval for the difference in failure rate between medical officers and consultant anaesthesiologists (MO − CA, percentage points), on a single linear axis. Orange rows are the paper's printed values: 0.4 (0.36 to 0.43) in the Abstract and Results text, 0.4 (0.36 to 0.44) in Table 3, and −2.5 (−2.92 to −2.08) for emergent cases. All three printed intervals are narrower than the plotted dot. Blue rows are recomputed from the paper's own counts (7/231 vs 5/191; 1/28 vs 2/33) with the Wald and Newcombe methods. The thin solid line marks no difference (0), and the dashed line marks the 5-point non-inferiority margin from the Methods. The recomputed all-cases intervals span zero, as the printed p = 0.7997 requires, and still end below the margin. The recomputed emergent-case interval is roughly 21 points wide and crosses both zero and the margin.

Why it matters

The confidence interval's upper bound is the statistic on which a non-inferiority trial's conclusion rests, and this one appears in the Abstract, the Results text and Table 3. As printed, it suggests that the difference in failure rates is known to within about ±0.04 percentage points. The counts support only about ±3.2 points, so the data are compatible with MOs having a failure rate up to about 3.6–3.9 points higher than CAs. Non-inferiority at the 5-point margin still holds under every standard method I checked. But the evidence is much less decisive than the printed interval suggests, and a reader using the printed interval would overstate it. That matters because the Interpretation and Discussion present the trial as support for regional and national task-sharing policy.

The printed intervals also contradict the paper's own tests. An interval that excludes zero cannot accompany p = 0.80, so a reader cannot tell from the paper which of the two to believe without recomputing. In the emergent-case subgroup, the printed interval implies that MOs were reliably better in emergencies. Recomputed, the subgroup (61 patients, 3 failures) is uninformative in either direction. Any later meta-analysis or sample-size calculation that took the printed interval or its implied standard error at face value would understate the uncertainty about 80-fold.

Explanations considered

Status

Suggested correction

  1. Replace the 95% CI for the all-cases difference in the Abstract, the Results text and Table 3 (rows 1 and 3) with an interval computed from 7/231 versus 5/191. For example, the Wald interval is −2.75 to 3.58 percentage points and the Newcombe interval is −3.29 to 3.85. State which method is used.
  2. Replace the emergent-case interval in Table 3 (row 2), for example with the Wald interval −13.1 to 8.2 or the Newcombe interval −16.4 to 12.3 percentage points.
  3. Report the non-inferiority comparison explicitly, as the corrected upper bound against the 5-point margin. State whether the test is one-sided at 2.5% (95% CI) or at 5% (90% CI, upper bound 3.07).
  4. Make the Abstract, the Results text and Table 3 agree. They currently give 0.43 and 0.44 as the upper bound.
  5. Describe 0.4 as a difference in failure rates (MO − CA), or report −0.4 if it is meant as a difference in success rates.
  6. Review the wording of the Interpretation and Discussion in light of the wider interval: non-inferiority holds, but with about 1.2–1.4 points of headroom rather than about 4.6.
Recomputation script (python recompute.py.txt)
"""Recompute the primary-outcome confidence intervals of PMC11331853
(BMJ Glob Health 2024, doi:10.1136/bmjgh-2023-014170) from the paper's own counts,
and regenerate the figures in this script's folder.

Run:  python recompute.py.txt
Needs numpy, scipy, matplotlib.

Inputs, all from the paper (Table 2, Table 3, abstract, Results):
  all cases:      MO 7/231 failures, CA 5/191 failures
  emergent cases: MO 1/28, CA 2/33  (denominators: Table 2 row "Emergency", 28 | 33;
                  they also reproduce the printed 3.6% and 6.1%)
  printed CIs:    0.36-0.43 (abstract, Results), 0.36-0.44 (Table 3), -2.92 to -2.08 (Table 3 emergent)
  NI margin:      5 percentage points (Methods)
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975)  # 1.959964

def wilson(x, n, z=Z):
    p = x / n; den = 1 + z * z / n
    c = (p + z * z / (2 * n)) / den
    h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
    return c - h, c + h

def analyse(a, n1, b, n2, label):
    p1, p2 = a / n1, b / n2; d = p1 - p2
    se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
    r = dict(p1=p1, p2=p2, d=d, se=se)
    r["wald"] = (d - Z * se, d + Z * se)
    r["wald90"] = (d - 1.645 * se, d + 1.645 * se)
    l1, u1 = wilson(a, n1); l2, u2 = wilson(b, n2)
    r["newcombe"] = (d - np.sqrt((p1 - l1) ** 2 + (u2 - p2) ** 2), d + np.sqrt((u1 - p1) ** 2 + (p2 - l2) ** 2))
    q1, q2 = (a + 1) / (n1 + 2), (b + 1) / (n2 + 2)
    seac = np.sqrt(q1 * (1 - q1) / (n1 + 2) + q2 * (1 - q2) / (n2 + 2))
    r["agresti_caffo"] = (q1 - q2 - Z * seac, q1 - q2 + Z * seac)
    t = np.array([[a, n1 - a], [b, n2 - b]])
    r["p_chi2"] = stats.chi2_contingency(t, correction=False)[1]
    r["p_yates"] = stats.chi2_contingency(t, correction=True)[1]
    r["p_fisher"] = stats.fisher_exact(t)[1]
    print(f"--- {label}")
    print(f"  MO {a}/{n1} = {100*p1:.2f}%   CA {b}/{n2} = {100*p2:.2f}%   difference MO-CA = {100*d:.3f} pp")
    print(f"  Wald SE = {100*se:.3f} pp")
    for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
        lo, hi = r[k]; print(f"  {k:14s} CI: {100*lo:6.2f} to {100*hi:5.2f} pp   (width {100*(hi-lo):.2f} pp)")
    print(f"  p: uncorrected chi-square {r['p_chi2']:.4f}; Yates {r['p_yates']:.4f}; Fisher {r['p_fisher']:.4f}")
    return r

print("Emergent denominators consistent with 3.6% and 6.1% and N=61:",
      [(n1, 61 - n1) for n1 in range(10, 52) if round(100 / n1, 1) == 3.6 and round(200 / (61 - n1), 1) == 6.1])
A = analyse(7, 231, 5, 191, "All cases (Table 3 rows 1 and 3)")
E = analyse(1, 28, 2, 33, "Emergent cases (Table 3 row 2)")

print("\n--- Printed intervals")
printed = {"abstract": (0.40, 0.36, 0.43), "table": (0.40, 0.36, 0.44), "emergent": (-2.5, -2.92, -2.08)}
for k, (est, lo, hi) in printed.items():
    se_imp = (hi - lo) / (2 * Z)
    print(f"  {k:9s}: {est} ({lo} to {hi}); width {hi-lo:.2f} pp; implied SE {se_imp:.4f} pp")
wa = 100 * (A["wald"][1] - A["wald"][0]); we = 100 * (E["wald"][1] - E["wald"][0])
print(f"  all cases: Wald width {wa:.2f} pp = {wa/0.08:.0f} x Table 3 width, {wa/0.07:.0f} x abstract width")
print(f"  all cases: Wald SE {100*A['se']:.3f} pp = {100*A['se']/((0.44-0.36)/(2*Z)):.0f} x SE implied by Table 3 interval")
print(f"  emergent:  Wald width {we:.2f} pp = {we/0.84:.0f} x printed width")
# p-value that the printed Table 3 interval would imply under a normal approximation
se_t = (0.44 - 0.36) / (2 * Z); z_t = 0.40 / se_t
p_implied = 2 * stats.norm.sf(z_t)
print(f"  Table 3 interval implies z = {z_t:.1f}, two-sided p = {p_implied:.1e}  (printed p = 0.7997)")
se_e = (-2.08 + 2.92) / (2 * Z); z_e = 2.5 / se_e
p_implied_e = 2 * stats.norm.sf(z_e)
print(f"  emergent interval implies z = {z_e:.1f}, two-sided p = {p_implied_e:.1e}  (printed p = 1.0000)")

print("\n--- Arithmetic route that reproduces the all-cases interval (diagnostic only)")
d_pp = 100 * A["d"]
print(f"  {d_pp:.3f} pp +/- 1.96 x {A['se']:.4f} (SE left in proportion units) = {d_pp - Z*A['se']:.3f} to {d_pp + Z*A['se']:.3f}")
print(f"  0.4 pp   +/- 1.96 x {A['se']:.4f} = {0.4 - Z*A['se']:.3f} to {0.4 + Z*A['se']:.3f}")
de_pp = 100 * E["d"]
print(f"  emergent: {de_pp:.3f} +/- 1.96 x {E['se']:.4f} = {de_pp - Z*E['se']:.3f} to {de_pp + Z*E['se']:.3f}  (printed -2.92 to -2.08: not reproduced)")

print("\n--- Other scales and summaries (Explanations considered)")
print(f"  Wald CI on the proportion scale: {A['wald'][0]:.3f} to {A['wald'][1]:.3f}")
rr = A["p1"] / A["p2"]; se_lrr = np.sqrt(1/7 - 1/231 + 1/5 - 1/191)
print(f"  risk ratio {rr:.2f}, 95% CI {rr*np.exp(-Z*se_lrr):.2f} to {rr*np.exp(Z*se_lrr):.2f}")
print(f"  half-widths: printed Table 3 {(0.44-0.36)/2:.2f} pp; Wald {Z*100*A['se']:.2f} pp")
print(f"  90% upper bound {100*A['wald90'][1]:.2f} pp = {100*A['wald90'][1]/0.44:.1f} x the printed 0.44")

print("\n--- Non-inferiority margin (5 pp)")
for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
    hi = 100 * A[k][1]; print(f"  {k:14s} upper bound {hi:.2f} pp -> headroom {5 - hi:.2f} pp; below margin: {hi < 5}")
for k in ("abstract", "table"):
    print(f"  printed {k:8s} upper bound {printed[k][2]} pp -> headroom {5 - printed[k][2]:.2f} pp")

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "axes.edgecolor": INK2, "svg.fonttype": "none",
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})

def save(fig, path):
    fig.savefig(path + ".svg"); fig.savefig(path + ".png", dpi=200); plt.close(fig)

def style(ax):
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)

def interval_rows(ax, rows):
    for i, (lab, est, lo, hi, col, txt) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
        ax.plot([lo, lo], [y - .12, y + .12], color=col, lw=2); ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
        ax.plot(est, y, "o", ms=8, color=col, mec=BG, mew=1.5, zorder=3)
        if txt:
            ax.text(1.02, y, txt, va="center", fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1])
    ax.set_ylim(-0.7, len(rows) - 0.3)

def pp(x): return 100 * x

# Candidate A (default fig1): every printed vs recomputed interval on one axis
def cand_a(path):
    fig, ax = plt.subplots(figsize=(7.2, 4.6))
    rows = [
        ("All cases, printed (Abstract, text)", 0.40, 0.36, 0.43, ORANGE, "0.4 (0.36 to 0.43)"),
        ("All cases, printed (Table 3)", 0.40, 0.36, 0.44, ORANGE, "0.4 (0.36 to 0.44)"),
        ("All cases, recomputed Wald", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f})"),
        ("All cases, recomputed Newcombe", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['newcombe'][0]):.2f} to {pp(A['newcombe'][1]):.2f})"),
        ("Emergent, printed (Table 3)", -2.5, -2.92, -2.08, ORANGE, "−2.5 (−2.92 to −2.08)"),
        ("Emergent, recomputed Wald", pp(E["d"]), pp(E["wald"][0]), pp(E["wald"][1]), BLUE, f"{pp(E['d']):.2f} ({pp(E['wald'][0]):.1f} to {pp(E['wald'][1]):.1f})"),
    ]
    rows = [(a, b, c, d, e, f.replace("-", "−")) for a, b, c, d, e, f in rows]
    style(ax); ax.axvline(0, color=INK2, lw=0.8)
    ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
    ax.text(5.3, -0.55, "NI margin, 5 pp", fontsize=10, color=INK2, va="center")
    interval_rows(ax, rows)
    ax.axhline(1.5, color=GRID, lw=0.8)
    ax.set_xlim(-15, 10); ax.set_xlabel("Failure-rate difference, MO − CA (pp), with 95% CI")
    ax.text(1.02, len(rows) - 0.3, "estimate (95% CI)", fontsize=10, color=INK, transform=ax.get_yaxis_transform(), va="bottom")
    ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
    ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from the paper's counts (7/231 vs 5/191; 1/28 vs 2/33)")
    fig.legend(*ax.get_legend_handles_labels(), loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False)
    fig.subplots_adjust(left=0.37, right=0.75, top=0.84, bottom=0.12); save(fig, path)

# Candidate B: primary outcome only, emphasising headroom to the margin
def cand_b(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    rows = [
        ("Printed, Abstract and Results", 0.40, 0.36, 0.43, ORANGE, ""),
        ("Printed, Table 3", 0.40, 0.36, 0.44, ORANGE, ""),
        ("Recomputed, Wald 95%", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, ""),
        ("Recomputed, Newcombe 95%", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, ""),
        ("Recomputed, Agresti–Caffo 95%", pp(A["d"]), pp(A["agresti_caffo"][0]), pp(A["agresti_caffo"][1]), BLUE, ""),
    ]
    style(ax); ax.axvline(0, color=INK2, lw=0.8); ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
    interval_rows(ax, rows)
    for y, hi, col in ((4, 0.43, ORANGE), (2, pp(A["wald"][1]), BLUE)):
        ax.annotate("", xy=(5, y + 0.3), xytext=(hi, y + 0.3), arrowprops=dict(arrowstyle="<->", color=INK2, lw=1))
        ax.text(hi - 0.1, y + 0.3, f"{5 - hi:.2f} pp to margin", ha="right", va="center", fontsize=10, color=INK2)
    ax.text(5.1, 0, "NI margin\n5 pp", fontsize=10, color=INK2, va="center")
    ax.set_xlim(-4, 6.6); ax.set_xlabel("Failure-rate difference, MO − CA (pp), all cases, 95% CI")
    ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
    ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from 7/231 vs 5/191")
    ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
    fig.tight_layout(); save(fig, path)

# Candidate C: arithmetic flow, showing the step where units diverge
def cand_c(path):
    fig, ax = plt.subplots(figsize=(7.2, 4.0)); ax.axis("off"); ax.set_xlim(0, 10); ax.set_ylim(0.4, 6.0)
    def box(x, y, w, h, txt, ec, fc=BG, bold=False):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.04,rounding_size=0.12", fc=fc, ec=ec, lw=2))
        ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10.5, fontweight="bold" if bold else "normal")
    box(0.2, 4.7, 2.6, 1.1, "Counts\n7/231 vs 5/191", INK2)
    box(3.4, 4.7, 3.0, 1.1, f"Difference\n{A['d']:.5f} = {pp(A['d']):.2f} pp", INK2)
    box(7.0, 4.7, 2.8, 1.1, f"Wald SE\n{A['se']:.4f} = {pp(A['se']):.2f} pp", INK2)
    for x0, x1 in ((2.8, 3.4), (6.4, 7.0)):
        ax.annotate("", xy=(x1, 5.25), xytext=(x0, 5.25), arrowprops=dict(arrowstyle="->", color=INK2, lw=1.5))
    box(0.3, 1.9, 4.4, 1.5, f"Same units (pp):\n{pp(A['d']):.2f} ± 1.96 × {pp(A['se']):.2f}\n= {pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f} pp".replace("-", "−"), BLUE)
    box(5.3, 1.9, 4.4, 1.5, f"Mixed units:\n{pp(A['d']):.2f} pp ± 1.96 × {A['se']:.4f}\n= {pp(A['d'])-Z*A['se']:.2f} to {pp(A['d'])+Z*A['se']:.2f}", ORANGE, fc="#fdf0ea")
    ax.annotate("", xy=(2.5, 3.4), xytext=(4.9, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
    ax.annotate("", xy=(3.2, 3.4), xytext=(7.6, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
    ax.annotate("", xy=(6.8, 3.4), xytext=(5.1, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
    ax.annotate("", xy=(7.5, 3.4), xytext=(8.4, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
    ax.text(2.5, 1.2, "Recomputed interval: spans 0,\nconsistent with the printed p = 0.7997", ha="center", va="center", fontsize=10, color=INK2)
    ax.text(7.5, 1.2, "Printed: 0.36 to 0.43 (Abstract), 0.36 to 0.44 (Table 3).\nA reconstruction only: this route does not\nreproduce the emergent row (−2.92 to −2.08)", ha="center", va="center", fontsize=10, color=INK2)
    ax.text(7.6, 4.05, "SE taken as a proportion\n(×100 omitted)", ha="left", fontsize=10, color=INK, style="italic", va="center")
    fig.tight_layout(); save(fig, path)

# Candidate D: interval widths, printed vs recomputed
def cand_d(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    labs = ["All cases", "Emergent cases"]
    rep = [0.44 - 0.36, -2.08 + 2.92]; rec = [wa, we]
    y = np.array([1, 0]); h = 0.34
    ax.barh(y + h / 2 + 0.01, rep, height=h, color=ORANGE, label="Printed 95% CI width")
    ax.barh(y - h / 2 - 0.01, rec, height=h, color=BLUE, label="Recomputed Wald 95% CI width")
    for yy, r, c in zip(y, rep, rec):
        ax.text(r + 0.3, yy + h / 2, f"{r:.2f} pp", va="center", fontsize=10, color=INK2)
        ax.text(c + 0.3, yy - h / 2, f"{c:.2f} pp  ({c/r:.0f}× wider)", va="center", fontsize=10, color=INK2)
    style(ax); ax.set_yticks(y); ax.set_yticklabels(labs); ax.set_xlim(0, 28)
    ax.set_xlabel("Width of the 95% CI for the difference in failure rate (percentage points)")
    ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
    fig.tight_layout(); save(fig, path)

# Candidate E: p-values, printed vs implied by the printed CI (log scale)
def cand_e(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.4))
    rows = [("All cases", 0.7997, A["p_chi2"], p_implied), ("Emergent cases", 1.0, E["p_fisher"], p_implied_e)]
    for i, (lab, prep, prec, pimp) in enumerate(rows):
        y = 1.5 * (1 - i)
        ax.plot([pimp, prep], [y, y], color=GRID, lw=2, zorder=1)
        ax.plot(prep, y + 0.08, "o", ms=9, color=ORANGE, zorder=3)
        ax.plot(prec, y - 0.08, "D", ms=8, color=BLUE, zorder=3)
        ax.plot(pimp, y, "o", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=3)
        ax.text(pimp, y + 0.22, f"implied by printed CI: p ≈ {pimp:.0e}", ha="center", fontsize=10, color=INK2, va="bottom")
        ax.text(prep * 0.3, y - 0.22, f"printed p = {prep:.4f}; recomputed {prec:.4f}", ha="right", fontsize=10, color=INK2, va="top")
    ax.axvline(0.05, color=INK2, lw=0.8); ax.text(0.05, 2.15, "p = 0.05 ", fontsize=10, color=INK2, ha="right")
    ax.set_xscale("log"); ax.set_xlim(1e-95, 30); style(ax)
    ax.set_yticks([1.5, 0]); ax.set_yticklabels([r[0] for r in rows]); ax.set_ylim(-0.7, 2.3)
    ax.set_xlabel("Two-sided p-value (log scale)")
    ax.plot([], [], "o", color=ORANGE, ms=9, label="Printed p")
    ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=9, label="p implied by printed CI")
    ax.plot([], [], "D", color=BLUE, ms=8, label="Recomputed from counts")
    ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=3, frameon=False)
    fig.tight_layout(); save(fig, path)

cand_a(os.path.join(HERE, "fig1"))
for f, n in ((cand_a, "a_all_intervals"), (cand_b, "b_margin_headroom"), (cand_c, "c_unit_flow"),
             (cand_d, "d_ci_widths"), (cand_e, "e_pvalues")):
    f(os.path.join(CAND, "cand_" + n))
print("\nFigures written to", HERE, "and", CAND)

Reported complete-abortion risk ratio of 2.55 should be about 1.02 by the paper's own conversion

BMJ Glob Health (2025) · doi:10.1136/bmjgh-2024-016024 · PMC11749663

Summary

For complete abortion in the review's only task-shifting trial (midlevel providers vs doctors, 1032 participants), the Abstract, the Results text and Table 2 all report a risk ratio (RR) of 2.55 (95% CI 0.82 to 4.27), described as "equivalent" or "little or no difference". The same Table 2 row gives risks of 96.1% and 97.3% and an adjusted risk difference (RD) of 1.49% (−0.17% to 3.14%), and its footnote § says the RR was converted from that RD with a 96% control risk; done in consistent units, that conversion gives RR 1.016 (0.998 to 1.033), whereas leaving the RD in percentage points, (0.96 + 1.49)/0.96, reproduces the published 2.55 (0.82 to 4.27) exactly. With a 96.1% control risk no RR can exceed 1/0.961 = 1.04, so the published value is impossible, and its interval is 100 times too wide on the linear scale. The verbal conclusion of no meaningful difference stands, but the first quantitative result in the Abstract misstates the size and precision of the effect, the correct value appears nowhere in the paper, and no correction has been found.

What the paper reports

Low certainty evidence indicates task shifting of medical abortion by doctors to midlevel providers may result in equivalent complete abortion (RR: 2.55, 95% CI: 0.82 to 4.27).

— Abstract, Results

Low certainty evidence indicates that compared with doctor-led medical abortion, nurse-led medical abortion may lead to little or no difference in abortion success (RR: 2.55, 95% CI: 0.82 to 4.27, one trial, 1032 participants) or the occurrence of an incomplete abortion (RR: 0.92, 95% CI: 0.45 to 1.89, one trial, 1032 participants, table 2). The certainty of evidence was downgraded by two levels due to serious imprecision (data from a single trial and CI including both benefit and harm) and serious risk of bias (no blinding of participants, personnel and outcome assessors).

— Results › Effects of interventions › Comparison 1. Who provides care (task shifting)

Patient health status: complete abortion | 96.1% | 97.3% | RR 2.55 (0.82 to 4.27)§ | 1032 (1 RCT) | Low†‡⊕⊕⊝⊝ | RD 1.49%¶ (–0.17% to 3.14%)

— Table 2 (Summary of findings 1), "complete abortion" row. Columns: outcome | assumed risk, control | corresponding risk, intervention | relative effect (95% CI) | participants | certainty | comments

§The effect in RR was converted from the adjusted risk difference provided in the report using control group risk of 96%.

— Table 2, footnote §

¶The risk difference calculated with intention-to-treat analysis estimated from a generalised estimating equation model with treatment as the fixed effect and service provider as a random effect and adjusted with woman’s age and length of gestation.

— Table 2, footnote ¶

the assumed risk in the comparison group and the relative effect of the intervention

— Table 2, footnote * (which says that the intervention-group risk is based on these two quantities)

What the paper's own numbers show

Method. Footnote § describes the conversion. If the control risk is p₀ and the risk difference is RD (both as proportions), the intervention risk is p₀ + RD, so RR = (p₀ + RD) / p₀. Applied to the point estimate and to each CI limit, this maps the RD interval linearly onto an RR interval. I applied it with p₀ = 0.96 (the footnote) and p₀ = 0.961 (Table 2), first with the RD as a proportion and then with the RD left as a percentage-point number. I also checked the result against the two risks in the same row of Table 2, and against the shape of the published interval. As an outside cross-check I used the counts in the source trial's abstract (Lancet 2011;377:1155-61, PMID 21458058). All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed
RR, complete abortion Abstract; Results Comparison 1; Table 2 2.55 (0.82 to 4.27) 1.016 (0.998 to 1.033), from footnote § with RD as a proportion (same to 3 dp with p₀ 0.96 or 0.961)
Same formula, RD left in percentage points Table 2 footnote § — (0.96 + 1.49)/0.96 = 2.552; (0.96 − 0.17)/0.96 = 0.823; (0.96 + 3.14)/0.96 = 4.271 → 2.55 (0.82 to 4.27), an exact match to the published values
Intervention risk implied by the conversion Table 2 97.3% 0.96 + 0.0149 = 0.975 (correct units); 0.96 + 1.49 = 2.45, i.e. 245% (published units)
Ratio of the two risks in Table 2 Table 2 96.1% vs 97.3% 97.3 / 96.1 = 1.012
Largest RR possible with control risk 96.1% Table 2 — 1 / 0.961 = 1.041
Intervention risk implied by RR 2.55 (footnote *) Table 2 97.3% 0.961 × 2.55 = 2.451 (245%)
Shape of the reported RR interval Table 2 0.82 to 4.27 around 2.55 Linear gaps 1.73 below, 1.72 above (symmetric on the linear scale); log gaps 1.135 vs 0.516. RR width / RD width = 1.042 ≈ 1/0.96 = 1.042, i.e. a linear rescaling of the RD interval by 1/0.96
Width of the published interval relative to the recomputed one Table 2 0.82 to 4.27 100 times wider on the linear scale; 48.6 times wider on the log scale
Adjusted RD, complete abortion Table 2 comments 1.49% (−0.17% to 3.14%) Not reproducible from counts (GEE-adjusted, footnote ¶); taken as given. Unadjusted from trial counts: 1.19% (−0.99% to 3.37%); trial abstract: 1.24% (−0.53 to 3.02)
Unadjusted RR, complete abortion (outside check) Source trial abstract: 504/518 vs 494/514 — 1.012 (0.990 to 1.035)
RR, incomplete abortion (control check) Results; Table 2 row 2 0.92 (0.45 to 1.89) 14/518 vs 15/514: 0.926 (0.45 to 1.90), consistent

The adjacent incomplete-abortion RR is reproduced from the trial's counts. This shows that the rest of the row's arithmetic holds, and that the discrepancy is confined to the RD-to-RR conversion.

Figure

Arithmetic chain of the RD-to-RR conversion in Table 2 footnote §, done in consistent units (blue) and with the RD left in percentage points (orange)

Figure 1. The conversion described in Table 2 footnote §, from the inputs stated in Table 2 to the risk ratio. The two lanes differ only at the first step, which is shaded: whether the adjusted risk difference enters the formula as a proportion (0.0149, blue, recomputed) or as a percentage-point number (1.49, orange). The orange lane reproduces the published RR 2.55 (0.82 to 4.27) to two decimals. It does so by way of an intervention "risk" of 245%, and its result lies above 1/0.961 = 1.04, the largest RR possible with the 96.1% control risk in the same table row. The blue lane gives RR 1.016 (0.998 to 1.033), which agrees with the ratio of the tabulated risks (1.012). All values are computed in recompute.py.txt.

Published and recomputed risk ratios for complete abortion on a log scale, with the region above RR 1.04 shaded as impossible

Figure 2. Risk ratio for complete abortion (midlevel providers vs doctors) on a log scale, with a thin reference line at RR = 1. Orange: the value published in the Abstract, Results and Table 2. Blue: the footnote § conversion recomputed with the RD as a proportion (its interval, 0.998 to 1.033, is narrower than the marker at this scale), and the plain ratio of the two Table 2 risks. The shaded region marks RRs above 1/0.961 = 1.04, which cannot occur when the control risk is 96.1%. The published point estimate and most of its interval lie in that region.

Why it matters

Explanations considered

None of these accounts for the published numbers. The stated conversion, carried out with the RD in percentage points rather than as a proportion, reproduces them exactly.

Status

As of 2026-10-23:

Suggested correction

In the Abstract, in Results › Comparison 1 and in Table 2 (complete abortion row), replace "RR: 2.55, 95% CI: 0.82 to 4.27" with "RR: 1.02, 95% CI: 1.00 to 1.03". Unrounded, this is 1.016 (0.998 to 1.033), from (0.96 + RD)/0.96 with RD = 0.0149 (−0.0017 to 0.0314). If two decimals would hide the fact that the lower limit lies just below 1, report three decimals: 1.016 (0.998 to 1.033). The authors may also wish to:

Recomputation script (python recompute.py.txt)
"""Recompute the complete-abortion risk ratio in PMC11749663 (BMJ Glob Health 2025,
doi:10.1136/bmjgh-2024-016024), Table 2, and regenerate fig1/fig2 in this folder.

Run:  python recompute.py.txt
Needs numpy, scipy, matplotlib.
"""
from pathlib import Path

import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle

HERE = Path(__file__).resolve().parent
z = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (paper)
RR_REP = (2.55, 0.82, 4.27)          # Abstract; Results Comparison 1; Table 2 row "complete abortion"
RD_PP = (1.49, -0.17, 3.14)          # Table 2 comments: adjusted RD, percentage points (footnote ¶)
P0_FOOT = 0.96                       # footnote §: "control group risk of 96%"
CTRL, INTV = 0.961, 0.973            # Table 2 illustrative risks, control / intervention

print("=" * 72)
print("1. Reported values (paper)")
print(f"   RR {RR_REP[0]} (95% CI {RR_REP[1]} to {RR_REP[2]})")
print(f"   adjusted RD {RD_PP[0]}% ({RD_PP[1]}% to {RD_PP[2]}%); control 96.1%, intervention 97.3%")


def rr_from_rd(rd, p0):
    """Footnote §: RR = (p0 + RD) / p0, with RD and p0 on the same scale."""
    return (p0 + rd) / p0


print("\n2. Footnote § conversion done in consistent units (RD / 100)")
correct = [rr_from_rd(x / 100, P0_FOOT) for x in RD_PP]
print("   p0 = 0.96 : RR = %.4f (%.4f to %.4f)  -> %.3f (%.3f to %.3f)" % (*correct, *correct))
correct961 = [rr_from_rd(x / 100, CTRL) for x in RD_PP]
print("   p0 = 0.961: RR = %.4f (%.4f to %.4f)" % tuple(correct961))
print("   implied intervention risk 0.96 + 0.0149 = %.4f" % (P0_FOOT + RD_PP[0] / 100))

print("\n3. Same formula with RD left in percentage points (not divided by 100)")
slip = [rr_from_rd(x, P0_FOOT) for x in RD_PP]
print("   p0 = 0.96 : RR = %.3f (%.3f to %.3f)  -> rounds to %.2f (%.2f to %.2f)" % (*slip, *slip))
slip961 = [rr_from_rd(x, CTRL) for x in RD_PP]
print("   p0 = 0.961: RR = %.3f (%.3f to %.3f)" % tuple(slip961))
print("   implied intervention 'risk' 0.96 + 1.49 = %.2f (i.e. %.0f%%)" % (P0_FOOT + RD_PP[0], 100 * (P0_FOOT + RD_PP[0])))

print("\n4. Consistency checks within Table 2")
ratio_tab = INTV / CTRL
print(f"   ratio of Table 2 risks 97.3/96.1 = {ratio_tab:.4f}")
print(f"   largest possible RR with control risk 96.1%: 1/0.961 = {1 / CTRL:.4f}")
print(f"   intervention risk implied by RR 2.55 (footnote *): 0.961 x 2.55 = {CTRL * 2.55:.3f} ({100 * CTRL * 2.55:.0f}%)")
print(f"   published / correct RR = {RR_REP[0] / correct[0]:.2f}; published / correct CI width = "
      f"{(RR_REP[2] - RR_REP[1]) / (correct[2] - correct[1]):.1f} (exactly 100 before rounding: "
      f"{(slip[2] - slip[1]) / (correct[2] - correct[1]):.1f})")
print(f"   published / correct CI width on the log scale = "
      f"{np.log(RR_REP[2] / RR_REP[1]) / np.log(correct[2] / correct[1]):.1f}")
print(f"   intervention risk implied by RR {correct[0]:.4f}: 0.961 x {correct[0]:.4f} = {CTRL * correct[0]:.4f}")

print("\n5. Shape of the reported interval")
lo, hi = RR_REP[0] - RR_REP[1], RR_REP[2] - RR_REP[0]
print(f"   linear gaps: {lo:.2f} below, {hi:.2f} above (midpoint {(RR_REP[1] + RR_REP[2]) / 2:.3f})")
llo, lhi = np.log(RR_REP[0]) - np.log(RR_REP[1]), np.log(RR_REP[2]) - np.log(RR_REP[0])
print(f"   log gaps:    {llo:.3f} below, {lhi:.3f} above")
print(f"   RR width / RD width = {(RR_REP[2] - RR_REP[1]) / (RD_PP[2] - RD_PP[1]):.4f}; 1/0.96 = {1 / 0.96:.4f}")

print("\n6. Source trial (Lancet 2011;377:1155-61, PMID 21458058, abstract counts)")
a, n1 = 504, 518      # midlevel providers: complete abortions / analysed
b, n0 = 494, 514      # doctors
p1, p0 = a / n1, b / n0
rr = p1 / p0
se = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
print(f"   complete: {a}/{n1} = {p1:.4f} vs {b}/{n0} = {p0:.4f}; n = {n1 + n0}")
print(f"   unadjusted RR = {rr:.4f} ({np.exp(np.log(rr) - z * se):.4f} to {np.exp(np.log(rr) + z * se):.4f})"
      f"  -> {rr:.3f} ({np.exp(np.log(rr) - z * se):.3f} to {np.exp(np.log(rr) + z * se):.3f})")
rd = p1 - p0
serd = np.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
print(f"   unadjusted RD = {100 * rd:.2f}% ({100 * (rd - z * serd):.2f}% to {100 * (rd + z * serd):.2f}%);"
      " trial abstract reports 1.24% (-0.53 to 3.02)")
print(f"   footnote § conversion of the trial-abstract RD 1.24%: RR = {rr_from_rd(0.0124, P0_FOOT):.4f}")
inc1, inc0 = n1 - a, n0 - b - 5          # 5 failed abortions in doctor arm, none in midlevel arm
r = (inc1 / n1) / (inc0 / n0)
s = np.sqrt(1 / inc1 - 1 / n1 + 1 / inc0 - 1 / n0)
print(f"   incomplete: {inc1}/{n1} vs {inc0}/{n0}: RR {r:.3f} ({np.exp(np.log(r) - z * s):.2f} to "
      f"{np.exp(np.log(r) + z * s):.2f}); paper reports 0.92 (0.45 to 1.89)")

print("\n7. Other effect measures, to rule them out as the source of 2.55 (0.82 to 4.27)")
orr = (a / (n1 - a)) / (b / (n0 - b))
seo = np.sqrt(1 / a + 1 / (n1 - a) + 1 / b + 1 / (n0 - b))
print(f"   odds ratio, complete abortion = {orr:.3f} ({np.exp(np.log(orr) - z * seo):.2f} to {np.exp(np.log(orr) + z * seo):.2f})")
print(f"   RR of non-complete abortion, doctor vs midlevel = {((n0 - b) / n0) / ((n1 - a) / n1):.3f}")

# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(HERE / f"{name}.svg", facecolor=BG)
    fig.savefig(HERE / f"{name}.png", dpi=200, facecolor=BG)
    plt.close(fig)


# fig1: arithmetic chain, two lanes
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0.5, 64)
ax.axis("off")


def box(x0, y0, w, h, text, edge, lw=2, face="white", bold_first=True, fs=10):
    ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0,rounding_size=1.2",
                                fc=face, ec=edge, lw=lw))
    lines = text.split("\n")
    n = len(lines)
    for i, ln in enumerate(lines):
        yy = y0 + h / 2 + (n - 1) / 2 * 3.3 - i * 3.3
        ax.text(x0 + w / 2, yy, ln, ha="center", va="center", fontsize=fs, color=INK,
                fontweight="bold" if (bold_first and i == 0) else "normal")


def arrow(x0, x1, y, color):
    ax.annotate("", xy=(x1, y), xytext=(x0, y),
                arrowprops=dict(arrowstyle="-|>", color=color, lw=2, mutation_scale=12))


box(2, 50.5, 96, 12,
    "Inputs stated in Table 2\n"
    "adjusted RD = 1.49% (−0.17% to 3.14%)  ·  control risk 96% (footnote §)\n"
    "conversion: RR = (0.96 + RD) / 0.96",
    INK2, lw=1.2, face="#f3f2ef")

cols = [(2, 22), (27, 22), (52, 22), (77, 21)]
# highlight the diverging step (behind the first box of each lane only)
for yy in (29, 7):
    ax.add_patch(FancyBboxPatch((0.9, yy - 1.1), 24.2, 15.7, boxstyle="round,pad=0,rounding_size=1.2",
                                fc="#e6e5e0", ec="none", zorder=0))
ax.text(13, 2.6, "units diverge here", ha="center", va="center", fontsize=10,
        color=INK, fontweight="bold")

lanes = [
    (29, BLUE, "Recomputed: RD divided by 100 before use (consistent units)",
     ["RD entered\n0.0149\n(−0.0017 to 0.0314)",
      "Intervention risk\n0.96 + 0.0149\n= 0.975 (97.5%)",
      f"RR = 0.975 / 0.96\n{correct[0]:.3f}\n({correct[1]:.3f} to {correct[2]:.3f})",
      f"Check vs Table 2\n97.3 / 96.1 = {ratio_tab:.3f}\nconsistent"]),
    (7, ORANGE, "Published: RD left in percentage points (not divided by 100)",
     ["RD entered\n1.49\n(−0.17 to 3.14)",
      f"Intervention risk\n0.96 + 1.49\n= {P0_FOOT + RD_PP[0]:.2f} (245%)",
      f"RR = 2.45 / 0.96\n{slip[0]:.2f}\n({slip[1]:.2f} to {slip[2]:.2f})",
      f"Check vs Table 2\nabove max {1 / CTRL:.2f}\ninconsistent"]),
]
for y0, color, label, texts in lanes:
    ax.add_patch(Rectangle((2.2, y0 + 15.2), 2.4, 2.4, fc=color, ec="none"))
    ax.text(5.6, y0 + 16.4, label, ha="left", va="center", fontsize=10, color=INK)
    for (x0, w), t in zip(cols, texts):
        box(x0, y0, w, 13.5, t, color)
    for (xa, wa), (xb, _) in zip(cols[:-1], cols[1:]):
        arrow(xa + wa, xb, y0 + 6.75, color)
ax.text(62.0, 2.6, "= published 2.55 (0.82 to 4.27)", ha="center", va="center",
        fontsize=10, color=INK, fontstyle="italic")
save(fig, "fig1")

# fig2: dot-and-interval on log scale
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
fig.subplots_adjust(left=0.035, right=0.97, top=0.97, bottom=0.18)
ax.set_facecolor(BG)
xmax_possible = 1 / CTRL
ax.axvspan(xmax_possible, 8, color="#efeeea", zorder=0, lw=0)
ax.text(1.08, -0.42, f"RR > 1/0.961 = {xmax_possible:.2f} is impossible with a 96.1% control risk",
        fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
rows = [
    (2.0, ORANGE, "o", RR_REP, "Published (Abstract, Results, Table 2): RR 2.55 (0.82 to 4.27)"),
    (1.0, BLUE, "o", correct, f"Footnote § conversion, recomputed: RR {correct[0]:.3f} ({correct[1]:.3f} to {correct[2]:.3f})"),
    (0.0, BLUE, "D", (ratio_tab, None, None), f"Ratio of Table 2 risks 97.3% / 96.1% = {ratio_tab:.3f}"),
]
for y, c, m, (pt, l, h), lab in rows:
    if l is not None:
        ax.plot([l, h], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=3)
    ax.plot(pt, y, marker=m, ms=9 if m == "o" else 8, color=c, mec=BG, mew=1.5, zorder=4, ls="none")
    ax.text(0.52, y + 0.36, lab, fontsize=10, color=INK, va="center", ha="left")
ax.set_xscale("log")
ax.set_xlim(0.5, 8)
ax.set_ylim(-0.7, 2.6)
ax.set_xticks([0.5, 0.82, 1, 1.5, 2, 2.55, 4.27, 8])
ax.set_xticklabels(["0.5", "0.82", "1", "1.5", "2", "2.55", "4.27", "8"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("left", "right", "top"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.set_xlabel("Risk ratio, complete abortion, midlevel providers vs doctors (log scale; line at RR = 1)", fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)

Abstract's patient-handwashing confidence interval is impossible and conflicts with the paper's own non-significant clustered estimate

BMJ Glob Health (2016) · doi:10.1136/bmjgh-2016-000103 · PMC5321376

Summary

The abstract reports two behaviour changes after an infection prevention and control (IPC) workshop: maintenance of distance, RR 1.09 (95% CI 1.02 to 1.16), and patient handwashing, RR 1.19 (95% CI 1.3 to 1.25). The handwashing interval cannot be a confidence interval, because its lower limit (1.3) is above its upper limit (1.25) and the point estimate (1.19) lies outside it even if the limits are swapped. The paper's own clustered GEE analysis in Table 4, the analysis the Methods specify, gives handwashing RR 1.27 (0.95 to 1.71) and distance RR 1.11 (0.83 to 1.48). Neither is significant, and the Results text says handwashing "increased though not significantly". The abstract's distance figure equals, to two decimals, a crude risk ratio computed from the Table 4 counts that ignores clustering. As printed, the abstract presents as statistically significant two improvements that the body of the paper reports as non-significant.

What the paper reports

Abstract, Results:

The maintenance of distance (RR 1.09, 95% CI 1.02 to 1.16) and patient handwashing (RR 1.19, 95% CI 1.3 to 1.25) improved to >90%.

Results, "Adherence to IPC behaviours", first paragraph:

Patient handwashing, with or without HCW prompting, increased though not significantly from 82% to 99% (RR 1.21, 95% CI 0.95 to 1.71).

Results, "Adherence to IPC behaviours", second paragraph:

Virtually all HCWs stood 1.5 m from patients, increasing from 91% to 99% at follow-up (RR 1.11, 95% CI 0.83 to 1.48).

Table 4, screening block (columns: baseline correct, per cent, n = 90 | follow-up correct, per cent, n = 131 | RR | 95% CI):

Patient washed hands directly or washed on direction from HCW | 74 | 82 | 130 | 99 | 1.27 | 0.95 to 1.71†

Stood 1.5 m from patient | 82 | 91 | 130 | 99 | 1.11 | 0.83 to 1.48†

Table 4 footnote:

*Risk ratio using binomial regression (family: binomial, link: log) accounting for clustering at the health facility level (GEE).

†Indicates that a Poisson regression (family: Poisson, link: log) was used due to the failure of the binomial model to converge.

Methods, "Data analysis":

Generalised estimating equations (GEE) with robust SEs accounted for repeated measures among HCWs and clustering within PHUs.

For all statistical tests, a significance level of p<0.05 was chosen.

Discussion, first paragraph:

During workshops, HCWs focused on improving screening, maintaining physical distance and encouraging patient handwashing; changes in these domains were reflected in the improvements seen in these behaviours at follow-up. Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%).

What the paper's own numbers show

Method. I checked the abstract's handwashing interval for internal validity (lower limit no greater than upper limit, point estimate inside). From the Table 4 counts I recomputed each percentage and a crude follow-up versus baseline risk ratio with a log-scale Wald 95% CI, ignoring clustering. For each printed Table 4 interval I took the geometric centre, sqrt(lower × upper), which equals the point estimate of a symmetric log-scale interval, and the standard error it implies, ln(upper/lower)/3.92. I then compared the abstract's and the Results text's figures with both analyses. The GEE estimates themselves cannot be re-derived, because they need facility- and worker-level data. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Handwashing 95% CI, ordering Abstract 1.3 to 1.25 lower limit 1.3 > upper limit 1.25: not a valid interval
Handwashing RR inside its CI? Abstract RR 1.19 1.19 lies outside 1.25–1.3, so it is outside the interval even with the limits swapped
Patient handwashing, % correct Table 4; Results text 82% → 99% 74/90 = 82.2%, 130/131 = 99.2%
Handwashing RR, crude (ignoring clustering) from Table 4 counts — 1.21 (1.10 to 1.33); log SE 0.050
Handwashing RR, GEE Poisson Table 4 1.27 (0.95 to 1.71) centre of printed CI 1.275: internally consistent; implied robust log SE 0.150; includes 1
Handwashing RR Results text 1.21 (0.95 to 1.71) point estimate = crude 1.207; interval = the Table 4 GEE interval (centred on 1.275)
Stood 1.5 m at screening, % correct Table 4; Results text 91% → 99% 82/90 = 91.1%, 130/131 = 99.2%
Distance RR, crude (ignoring clustering) from Table 4 counts — 1.089 (1.019 to 1.164), i.e. 1.09 (1.02 to 1.16)
Distance RR Abstract 1.09 (1.02 to 1.16) identical to the crude, unclustered value above
Distance RR, GEE Poisson Table 4; Results text 1.11 (0.83 to 1.48) centre of printed CI 1.108: consistent; includes 1
Stood 1.5 m at consultation, for contrast Table 4 35/54 (65%) → 29/32 (91%); 1.18 (0.92 to 1.51) crude 1.40 (1.12 to 1.75); neither matches the abstract
Hypothetical "1.13 to 1.25" for handwashing — — geometric centre 1.188 (≈ 1.19), but it implies log SE 0.026: 0.52 × the crude SE and 0.17 × the GEE robust SE
Handwashing counts by district Supplement, Annex 1 Bo 34/45 → 58/58; Kenema 40/45 → 72/73 sum 74/90 → 130/131, matching Table 4

Three things follow. First, the abstract's handwashing interval is not a confidence interval for any estimate. Second, the abstract's distance result reproduces exactly a crude, unclustered analysis of the Table 4 counts, while the Methods and Table 4 describe a clustered GEE analysis whose interval for the same comparison includes 1. Third, the abstract's handwashing estimate (1.19) matches neither the crude ratio (1.21) nor the GEE estimate (1.27). Its pattern resembles the distance figure: both of the abstract's intervals exclude 1, and both of the corresponding GEE intervals include 1.

Figure

Patient handwashing and distance risk ratios as reported in the abstract, the Results text and Table 4, with crude ratios recomputed from the Table 4 counts

Risk ratios for patient handwashing (top) and for standing 1.5 m from the patient at screening (bottom), on a log scale with a reference line at 1 (no change). Orange circles are values printed in the paper; blue squares are crude, unclustered Wald estimates recomputed from the Table 4 counts. The abstract's handwashing "interval" is drawn at its printed limits, with the lower limit (1.3) to the right of the upper limit (1.25) and the point estimate (1.19) outside both. The abstract's distance interval coincides with the crude recomputation. The paper's GEE intervals for both behaviours, in Table 4 and in the Results text, cross 1.

Abstract and body statements side by side, with the crude recomputation below

The abstract's two effect estimates (left) set against the Results text and Table 4 (right) and against the crude recomputation from the Table 4 counts (bottom). The abstract's intervals lie entirely above 1; the body's intervals for the same two behaviours include 1. The abstract's distance figure reproduces the unclustered calculation; its handwashing figure reproduces nothing in the paper.

Why it matters

These are the only two quantitative effect estimates in the abstract, and its sentence reads as a statistically significant improvement in both behaviours. The analysis described in the Methods (GEE with robust standard errors, significance at p<0.05) does not support that reading. Patient handwashing, RR 1.27 (0.95 to 1.71), and distance at screening, RR 1.11 (0.83 to 1.48), are both non-significant in Table 4, and the Results text says so for handwashing. A reader of the abstract alone, or a reviewer extracting effect sizes from it, would record two significant improvements and, for handwashing, an interval that cannot be used. Clustering matters here: the GEE robust standard errors implied by Table 4 are about three times (handwashing) and four times (distance) the unclustered ones, so the unclustered intervals are much too narrow for observations clustered within eight facilities.

The overstatement of significance is confined to the abstract. The Discussion describes the percentage improvements, which did occur (82% to 99% and 91% to 99%), and its next sentence states that "Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%)". That agrees with Table 4, where, of the handwashing items, only the prompted ones (screener asked patient to wash hands, RR 1.45, 1.16 to 1.80; patient washed hands on direction, RR 1.49, 1.19 to 1.86) are significant. The abstract carries no such qualification. Its conclusions are qualitative (favourable attitudes towards IPC, gaps in practice, the need for socio-behavioural improvement and a secure supply chain), and they do not depend on these two estimates.

Explanations considered

Status

Suggested correction

The abstract's Results sentence should report the paper's primary, clustered analysis. Using Table 4 it would read, for example: "Maintenance of distance during screening (91% to 99%; RR 1.11, 95% CI 0.83 to 1.48) and patient handwashing (82% to 99%; RR 1.27, 95% CI 0.95 to 1.71) reached >90%, although neither change was statistically significant; HCW prompting of patient handwashing (RR 1.45, 95% CI 1.16 to 1.80) and handwashing on prompting (RR 1.49, 95% CI 1.19 to 1.86) increased significantly." If the authors intended to report crude estimates in the abstract, that should be stated, the handwashing figure should read 1.21 (1.10 to 1.33), and the abstract should note that the clustered analysis is not significant. Separately, the Results text's handwashing estimate should read RR 1.27 to match its interval and Table 4. The Discussion already notes that significant improvements were not consistent across behaviours and needs no change.

Recomputation script (python recompute.py.txt)
"""Recompute the patient-handwashing and distance risk ratios of PMC5321376
(BMJ Glob Health 2016;1:e000103, doi:10.1136/bmjgh-2016-000103).

Run:  python recompute.py.txt                      # numbers + fig1, fig2
      python recompute.py.txt --candidates DIR     # also write design1..5 to DIR
      python recompute.py.txt --fig1 3 --fig2 0    # use design 3 as fig1, no fig2
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png (and fig2.svg/png unless --fig2 0) next to this file. Five candidate
figure designs exist; fig1/fig2 are design1/design2 unless chosen otherwise.

Inputs are copied from the paper as archived in PMC (identical in the typeset PDF):
  Abstract, Results:      distance RR 1.09 (1.02 to 1.16); handwashing RR 1.19 (1.3 to 1.25)
  Results text:           handwashing RR 1.21 (0.95 to 1.71); stood 1.5 m RR 1.11 (0.83 to 1.48)
  Table 4:                counts, percentages, GEE risk ratios and 95% CIs
  Supplement (Annex 1):   handwashing counts by district (Bo, Kenema)
"""
import argparse
import math
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
ap = argparse.ArgumentParser()
ap.add_argument("--candidates", help="directory to write candidate designs 1-5 into")
ap.add_argument("--fig1", type=int, default=1, help="design number used as fig1")
ap.add_argument("--fig2", type=int, default=2, help="design number used as fig2 (0 = none)")
ARGS = ap.parse_args()
Z = 1.959964

# ---------------------------------------------------------------- inputs (from the paper)
# Table 4 rows: (baseline correct, baseline n, follow-up correct, follow-up n, RR, lo, hi)
# Screening block: baseline n = 90, follow-up n = 131. Consultation block: 54 and 32.
TABLE4 = {
    "Screener asked patient to wash hands":               (56, 90, 105, 131, 1.45, 1.16, 1.80),
    "Patient washed hands on direction from HCW":         (54, 90, 105, 131, 1.49, 1.19, 1.86),
    "Patient washed hands directly or on direction (dagger)": (74, 90, 130, 131, 1.27, 0.95, 1.71),
    "Stood 1.5 m from patient, screening (dagger)":       (82, 90, 130, 131, 1.11, 0.83, 1.48),
    "Stood 1.5 m from patient, consultation":             (35, 54, 29, 32, 1.18, 0.92, 1.51),
}
TABLE4_PCT = {  # printed per cent columns (baseline, follow-up)
    "Screener asked patient to wash hands": (62, 80),
    "Patient washed hands on direction from HCW": (60, 80),
    "Patient washed hands directly or on direction (dagger)": (82, 99),
    "Stood 1.5 m from patient, screening (dagger)": (91, 99),
    "Stood 1.5 m from patient, consultation": (65, 91),
}
ABSTRACT = {"handwashing": (1.19, 1.3, 1.25), "distance": (1.09, 1.02, 1.16)}
RESULTS_TEXT = {"handwashing": (1.21, 0.95, 1.71), "distance": (1.11, 0.83, 1.48)}
ANNEX_HANDWASH = {"Bo": ((34, 45), (58, 58)), "Kenema": ((40, 45), (72, 73))}

HW = "Patient washed hands directly or on direction (dagger)"
DS = "Stood 1.5 m from patient, screening (dagger)"
DC = "Stood 1.5 m from patient, consultation"


def crude_rr(a1, n1, a0, n0):
    """Unclustered risk ratio, follow-up vs baseline, with a log-scale Wald 95% CI."""
    rr = (a1 / n1) / (a0 / n0)
    se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
    return rr, rr * math.exp(-Z * se), rr * math.exp(Z * se), se


def geo_centre(lo, hi):
    return math.sqrt(lo * hi)


def log_se(lo, hi):
    return math.log(hi / lo) / (2 * Z)


R = {}  # recomputed numbers used by the figures

print("== 1. The abstract's handwashing interval ==")
rr, lo, hi = ABSTRACT["handwashing"]
print(f"abstract RR {rr}, 95% CI {lo} to {hi}")
print(f"  lower limit <= upper limit?   {lo <= hi}   ({lo} > {hi})")
print(f"  interval contains the RR?     {min(lo, hi) <= rr <= max(lo, hi)}")
print(f"  with the limits swapped (1.25 to 1.3), contains 1.19? {1.25 <= rr <= 1.3}")
print(f"  hypothetical '1.13 to 1.25': geometric centre {geo_centre(1.13, 1.25):.3f}, "
      f"implied log-scale SE {log_se(1.13, 1.25):.4f}")

print("\n== 2. Table 4 counts: percentages, crude RR, and the centre of each printed CI ==")
for k, (b, n0, f, n1, RR, L, H) in TABLE4.items():
    r, l, h, se = crude_rr(f, n1, b, n0)
    R[k] = (r, l, h, se)
    pb, pf = TABLE4_PCT[k]
    print(f"{k}:")
    print(f"  baseline {b}/{n0} = {100*b/n0:.1f}% (printed {pb}); follow-up {f}/{n1} = "
          f"{100*f/n1:.1f}% (printed {pf})")
    print(f"  crude unclustered RR {r:.3f} (Wald 95% CI {l:.3f} to {h:.3f}); log SE {se:.4f}")
    print(f"  Table 4 GEE RR {RR} ({L} to {H}); geometric centre of the printed CI "
          f"{geo_centre(L, H):.3f}; implied robust log SE {log_se(L, H):.4f}")

print("\n== 3. Where the abstract's numbers come from ==")
r, l, h, _ = R[DS]
print(f"distance: abstract 1.09 (1.02 to 1.16) vs crude screening distance "
      f"{r:.2f} ({l:.2f} to {h:.2f}) -> exact match after rounding: "
      f"{(round(r, 2), round(l, 2), round(h, 2)) == (1.09, 1.02, 1.16)}")
r2, l2, h2, _ = R[DC]
print(f"  consultation distance, for contrast: crude {r2:.2f} ({l2:.2f} to {h2:.2f}); "
      f"GEE 1.18 (0.92 to 1.51)")
r, l, h, se = R[HW]
print(f"handwashing: abstract 1.19 (1.3 to 1.25); crude {r:.2f} ({l:.2f} to {h:.2f}); "
      f"Table 4 GEE 1.27 (0.95 to 1.71)")
print(f"  a CI of 1.13 to 1.25 needs log SE {log_se(1.13, 1.25):.4f}, "
      f"{log_se(1.13, 1.25)/se:.2f} x the crude unclustered SE ({se:.4f}) and "
      f"{log_se(1.13, 1.25)/log_se(0.95, 1.71):.2f} x the GEE robust SE ({log_se(0.95, 1.71):.4f})")

for k, nm in ((HW, "handwashing"), (DS, "distance, screening")):
    L, H = TABLE4[k][5], TABLE4[k][6]
    print(f"  {nm}: GEE robust log SE / crude log SE = {log_se(L, H)/R[k][3]:.2f}")

print("\n== 4. Results text: handwashing RR 1.21 (0.95 to 1.71) ==")
print(f"  crude point estimate {(130/131)/(74/90):.3f}; geometric centre of 0.95-1.71 = "
      f"{geo_centre(0.95, 1.71):.3f} (Table 4 prints 1.27)")
print(f"  does 0.95-1.71 exclude 1? {0.95 > 1}; does the abstract's distance CI exclude 1? "
      f"{1.02 > 1}; Table 4 distance CI 0.83-1.48 excludes 1? {0.83 > 1}")

print("\n== 5. Supplement Annex 1, handwashing by district ==")
bt = sum(v[0][0] for v in ANNEX_HANDWASH.values()); bn = sum(v[0][1] for v in ANNEX_HANDWASH.values())
ft = sum(v[1][0] for v in ANNEX_HANDWASH.values()); fn = sum(v[1][1] for v in ANNEX_HANDWASH.values())
print(f"  baseline {bt}/{bn}, follow-up {ft}/{fn}  (Table 4: 74/90, 130/131)")

# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})
LW, MS = 2.0, 8


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight")
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


def log_axis(ax, lo=0.8, hi=1.9, ticks=(0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7)):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    ax.set_xticks(ticks)
    ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.axvline(1.0, color=INK2, lw=0.9, zorder=1)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


def interval(ax, y, est, lo, hi, color, marker, label=None, dashed=False):
    ax.plot([lo, hi], [y, y], color=color, lw=LW, ls="--" if dashed else "-",
            solid_capstyle="butt", zorder=2)
    ax.plot([lo, lo], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
    ax.plot([hi, hi], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
    ax.plot(est, y, marker, color=color, ms=MS, mec=BG, mew=1.5, zorder=3, label=label)


hw_c, ds_c = R[HW], R[DS]


# ---------------------------------------------------------------- design 1: forest plot
def design1():
    fig, ax = plt.subplots(figsize=(4.72, 4.4))
    rows = [
        # y, label, (est, lo, hi), kind
        (7, "Abstract", (1.19, 1.3, 1.25), "abs"),
        (6, "Results text", RESULTS_TEXT["handwashing"], "rep"),
        (5, "Table 4 (GEE)", (1.27, 0.95, 1.71), "rep"),
        (4, "Crude, ignoring clustering", hw_c[:3], "rec"),
        (2.5, "Abstract", ABSTRACT["distance"], "rep"),
        (1.5, "Table 4 / text (GEE)", (1.11, 0.83, 1.48), "rep"),
        (0.5, "Crude, ignoring clustering", ds_c[:3], "rec"),
    ]
    for y, lab, (e, l, h), kind in rows:
        if kind == "abs":
            # printed limits drawn where they sit; the 'interval' runs backwards
            ax.plot([1.25, 1.3], [y, y], color=ORANGE, lw=LW, ls=(0, (3, 2)), zorder=2)
            ax.plot(1.3, y, marker="|", color=ORANGE, ms=14, mew=LW)
            ax.plot(1.25, y, marker="|", color=ORANGE, ms=14, mew=LW)
            ax.plot(e, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=3)
            ax.annotate("lower 1.3", (1.3, y), (1.3, y + 0.42), ha="center", fontsize=10, color=INK2)
            ax.annotate("upper 1.25", (1.25, y), (1.235, y - 0.55), ha="center", fontsize=10, color=INK2)
            ax.text(1.02, y, "1.19 (1.3 to 1.25):\nlower limit > upper;\n1.19 lies outside", va="center",
                    ha="left", fontsize=10, color=INK,
                    transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
        else:
            c, m = (ORANGE, "o") if kind == "rep" else (BLUE, "s")
            interval(ax, y, e, l, h, c, m)
            txt = f"{e:.2f} ({l:.2f} to {h:.2f})"
            ax.text(1.02, y, txt, va="center", ha="left", fontsize=10, color=INK,
                    transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
    ax.set_yticks([r[0] for r in rows])
    ax.set_yticklabels([r[1] for r in rows], fontsize=10, color=INK)
    ax.text(0.0, 1.0, "Patient handwashing, 74/90 (82%) \u2192 130/131 (99%)", transform=ax.transAxes,
            fontsize=10.5, weight="bold", ha="left", va="bottom")
    ax.text(0.8, 3.15, "Stood 1.5 m at screening, 82/90 (91%) \u2192 130/131 (99%)", fontsize=10.5,
            weight="bold", ha="left", va="bottom", transform=ax.transData,
            bbox=dict(fc=BG, ec="none", pad=1.5), zorder=4)
    log_axis(ax, 0.8, 1.8)
    ax.set_ylim(-0.1, 7.7)
    ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
    h2, = ax.plot([], [], "s", color=BLUE, ms=MS,
                  label="Recomputed from Table 4 counts (unclustered Wald)")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.16), ncol=1,
              frameon=False, fontsize=10)
    return fig


# ---------------------------------------------------------------- design 2: two text panels
def box(ax, x, y, w, h, title, lines, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
                                fc="#ffffff", ec=edge, lw=1.5))
    ax.text(x + 0.02, y + h - 0.035, title, weight="bold", fontsize=10.5, va="top")
    ax.text(x + 0.02, y + h - 0.10, "\n".join(lines), fontsize=10, va="top", color=INK,
            linespacing=1.45)


def design2():
    fig = plt.figure(figsize=(7.2, 3.0))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(-0.015, 1.015); ax.set_ylim(-0.025, 1.0); ax.axis("off")
    box(ax, 0.0, 0.47, 0.41, 0.51, "Abstract, Results",
        ["\u201cmaintenance of distance", "(RR 1.09, 95% CI 1.02 to 1.16)",
         "and patient handwashing", "(RR 1.19, 95% CI 1.3 to 1.25)",
         "improved to >90%\u201d"], ORANGE)
    box(ax, 0.57, 0.47, 0.43, 0.51, "Results text / Table 4",
        ["Handwashing \u201cincreased", "though not significantly\u201d:",
         "RR 1.21 (text), 1.27 (Table 4),", "both 95% CI 0.95 to 1.71",
         "Stood 1.5 m: RR 1.11 (0.83 to 1.48)"], ORANGE)
    box(ax, 0.0, 0.0, 1.0, 0.29, "Recomputed from Table 4 counts, ignoring clustering",
        [f"Stood 1.5 m, screening: {ds_c[0]:.2f} ({ds_c[1]:.2f} to {ds_c[2]:.2f}), same as abstract",
         f"Patient handwashing: {hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f}), not the abstract's "
         "1.19 (1.3 to 1.25)"],
        BLUE)
    ax.annotate("", xy=(0.565, 0.74), xytext=(0.415, 0.74),
                arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
    ax.text(0.49, 0.78, "conflict", ha="center", fontsize=10, weight="bold")
    ax.text(0.49, 0.70, "CIs > 1\nvs\nCIs span 1", ha="center", va="top", fontsize=10,
            color=INK2)
    ax.annotate("", xy=(0.22, 0.30), xytext=(0.22, 0.46),
                arrowprops=dict(arrowstyle="<-", color=BLUE, lw=2))
    ax.text(0.235, 0.38, "distance matches crude", fontsize=10, color=INK2, va="center")
    return fig


# ---------------------------------------------------------------- design 3: zoomed number line
def design3():
    fig, ax = plt.subplots(figsize=(6.52, 2.9))
    y_abs, y_rec, y_gee = 2.0, 1.0, 0.0
    ax.plot([1.25, 1.3], [y_abs, y_abs], color=ORANGE, lw=LW, ls=(0, (3, 2)))
    for x, lab, dy in [(1.3, "printed lower limit 1.3", 0.32), (1.25, "printed upper limit 1.25", -0.45)]:
        ax.plot(x, y_abs, marker="|", color=ORANGE, ms=16, mew=LW)
        ax.text(x, y_abs + dy, lab, ha="center", fontsize=10, color=INK2)
    ax.plot(1.19, y_abs, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5)
    ax.text(1.175, y_abs, "RR 1.19", ha="right", va="center", fontsize=10, color=INK2)
    interval(ax, y_rec, *hw_c[:3], BLUE, "s")
    ax.text(hw_c[2] * 1.01, y_rec, f"{hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f})", va="center", fontsize=10)
    interval(ax, y_gee, 1.27, 0.95, 1.71, ORANGE, "o")
    ax.text(1.72, y_gee, "1.27 (0.95 to 1.71)", va="center", fontsize=10)
    ax.set_yticks([y_abs, y_rec, y_gee])
    ax.set_yticklabels(["Abstract (reported)", "Crude from Table 4 counts\n(recomputed)",
                        "Table 4 GEE (reported)"], color=INK)
    log_axis(ax, 0.9, 2.1, (0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7, 2.0))
    ax.set_ylim(-0.6, 2.7)
    ax.set_xlabel("Patient handwashing risk ratio (log scale); line at 1 = no change")
    return fig


# ---------------------------------------------------------------- design 4: provenance grid
def design4():
    fig = plt.figure(figsize=(7.2, 3.4))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 7.2); ax.set_ylim(0, 3.4); ax.axis("off")
    cx = (1.9, 3.42, 4.94)   # column centres, in inches
    cols = ["Abstract\n(reported)", "Crude from counts\n(recomputed)", "Table 4 GEE\n(reported)"]
    for j, c in enumerate(cols):
        ax.text(cx[j], 3.1, c, ha="center", va="center", fontsize=10, weight="bold",
                color=ORANGE if j != 1 else BLUE)
    rows = [("Stood 1.5 m,\nscreening", "1.09 (1.02\u20131.16)",
             f"{ds_c[0]:.2f} ({ds_c[1]:.2f}\u2013{ds_c[2]:.2f})", "1.11 (0.83\u20131.48)",
             "same", "differs"),
            ("Patient\nhandwashing", "1.19 (1.3\u20131.25)",
             f"{hw_c[0]:.2f} ({hw_c[1]:.2f}\u2013{hw_c[2]:.2f})", "1.27 (0.95\u20131.71)",
             "differs", "differs")]
    for i, (lab, a, c, g, m1, m2) in enumerate(rows):
        y = 2.2 - i * 1.0
        ax.text(0.05, y, lab, va="center", fontsize=10, weight="bold")
        for j, v in enumerate((a, c, g)):
            ax.add_patch(FancyBboxPatch((cx[j] - 0.725, y - 0.3), 1.45, 0.6,
                                        boxstyle="round,pad=0.0,rounding_size=0.06",
                                        fc="#ffffff", ec=BLUE if j == 1 else ORANGE, lw=1.5))
            ax.text(cx[j], y, v, ha="center", va="center", fontsize=10)
        ax.text(5.78, y, f"vs crude: {m1}\nvs GEE: {m2}", ha="left",
                va="center", fontsize=10, color=INK2)
    ax.text(0.05, 0.35, "Only the GEE intervals include 1 (no significant change).\n"
            "The abstract's handwashing interval has its lower limit above its upper limit.",
            fontsize=10, color=INK2, va="center")
    return fig


# ---------------------------------------------------------------- design 5: crude vs GEE, all rows
def design5():
    keys = [HW, DS, "Screener asked patient to wash hands",
            "Patient washed hands on direction from HCW", DC]
    labels = ["Patient washed hands\n(direct or prompted)", "Stood 1.5 m, screening",
              "Screener asked patient\nto wash hands", "Patient washed hands\non direction",
              "Stood 1.5 m, consultation"]
    fig, ax = plt.subplots(figsize=(6.62, 4.4))
    for i, k in enumerate(keys):
        y = len(keys) - 1 - i
        b, n0, f, n1, RR, L, H = TABLE4[k]
        interval(ax, y + 0.17, RR, L, H, ORANGE, "o")
        c = R[k]
        interval(ax, y - 0.17, c[0], c[1], c[2], BLUE, "s")
    # abstract values overlaid as open diamonds
    ax.plot(1.19, 4.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
    ax.plot(1.09, 3.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
    ax.text(1.19, 4.48, "abstract 1.19\n(CI printed 1.3 to 1.25)", ha="center", fontsize=10, color=INK2)
    ax.text(1.02, 2.62, "abstract 1.09 (1.02 to 1.16)", ha="left", fontsize=10, color=INK2, va="top")
    ax.set_yticks(range(len(keys)))
    ax.set_yticklabels(labels[::-1], color=INK)
    log_axis(ax, 0.8, 2.0, (0.8, 0.9, 1.0, 1.2, 1.5, 2.0))
    ax.set_ylim(-0.6, 5.15)
    ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Table 4 GEE (reported)")
    h2, = ax.plot([], [], "s", color=BLUE, ms=MS, label="Crude Wald from Table 4 counts (recomputed)")
    h3, = ax.plot([], [], "D", mfc="none", mec=ORANGE, ms=MS, mew=LW, label="Abstract (reported)")
    ax.legend(handles=[h1, h2, h3], loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=2,
              frameon=False, fontsize=10)
    return fig


designs = {1: design1, 2: design2, 3: design3, 4: design4, 5: design5}
if ARGS.candidates:
    os.makedirs(ARGS.candidates, exist_ok=True)
    for i, d in designs.items():
        save(d(), os.path.join(ARGS.candidates, f"design{i}"))
    print(f"\nwrote design1..5 (.svg, .png) to {ARGS.candidates}")
save(designs[ARGS.fig1](), os.path.join(HERE, "fig1"))
if ARGS.fig2:
    save(designs[ARGS.fig2](), os.path.join(HERE, "fig2"))
print(f"wrote fig1 (design {ARGS.fig1})" + (f" and fig2 (design {ARGS.fig2})" if ARGS.fig2 else ""))

Ebola review calls case-fatality change non-significant while reporting OR 1.6, 95% CI excluding 1, p<0.001

BMJ Glob Health (2020) · doi:10.1136/bmjgh-2019-001955 · PMC7061886

Summary

This review of 34 Ebola virus disease outbreaks (1976–2019) states in its Abstract, Results and Abstract Conclusions that the case fatality rate (CFR) did not change significantly over four decades. It supports that statement with "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001". That interval excludes 1 and the P value is below 0.001, so the statistic contradicts the claim it is cited for. The Results text gives a different interval (1.5 to 1.6) for the same OR. The OR also points the wrong way for the paper's own CFRs (71% falling to 62% gives an OR of about 0.67, not 1.6). The Table 3 CFRs for three of the four periods and the overall CFR of "66% (62 to 71)" cannot be reproduced from the outbreak counts in Table 1. Those counts give a pooled CFR that falls from 71.3% to 41.5% (crude OR 0.29, 95% CI 0.24 to 0.34) and an overall CFR of 43.1%. The CFR trend is the review's only quantitative inference, and the Abstract Conclusions use it to support the claim that control challenges persist. As published, the reported statistics do not support the sentence they accompany. Whether the real trend is significant depends on an analysis the paper does not describe: a patient-level comparison finds a large decline, while analyses that allow for variation between outbreaks find no clear trend.

What the paper reports

Abstract, Methods:

Analyses of trends in case fatality were performed by calculating ORs between times.

Abstract, Results:

In the past four decades, a total of 34 EVD outbreaks affecting 34 356 cases and causing 14 823 deaths were reported in 11 countries in Sub-Saharan Africa. The overall case fatality rate (95% CI) was 66% (62 to 71) and did not change substantially over time (OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001).

Abstract, Conclusions:

Sub-Saharan Africa continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates observed during the past four decades.

Methods, "Data extraction and analysis":

Additional analyses of trends in case fatality rates were performed by calculating ORs between periods using MedCalc statistical software (https://www.medcalc.org/calc/odds_ratio.php).

Results, "Context of EVD", fourth paragraph:

Case fatality rates varied from 71% during outbreaks in 1976–1986 to 62% during the later years. However, there was no statistically significant change over time (OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)). The overall trend from 1976 to 2019 outbreaks shows a linear rate of change over time (table 3, figure 3).

Results, "Context of EVD", third paragraph:

During the past four decades, approximately 14 853 people have died from EVD outbreaks.

Table 3, "OR for case fatality rate of Ebola virus disease according to period" (columns: 1976–1986 | 1987–1997 | 1998–2008 | 2009–2019):

Death (n) | 454 | 348 | 733 | 13 288

Case fatality rate (%) | 71 | 67 | 64 | 62

OR (95% CI) | 1.6 (1.4 to 1.8) | 1.7 (1.5 to 1.9) | 1.4 (1.3 to 1.6) | 1.6 (1.5 to 1.6)

P value | <0.001 | <0.001 | <0.001 | <0.001

Table 1 lists each outbreak's year, country, virus, cases, deaths and CFR. Its total row:

Total | | | 34 356 | 14 823 |

Table 2, sex rows (columns: case | death | CFR (%)):

Gender | Male | 15 803 | 6373 | 46

Gender | Female | 18 553 | 8449 | 54

Table 3 does not state the comparison group or the direction of its ORs, and the paper does not say whether the analysis treated individual patients or outbreaks as the unit.

What the paper's own numbers show

Method. I entered the 34 outbreaks of Table 1 (year, cases, deaths) and assigned each multi-year outbreak (for example "2000–2001") to its first year. Grouping them into the four Table 3 periods reproduces Table 3's death counts exactly (454, 348, 733, 13 288), which confirms the grouping. For each period I computed the pooled CFR (deaths ÷ cases). I then computed crude odds ratios of death with Woolf 95% CIs, the CFR odds ratio implied by Table 3's own CFRs, and a chi-square test of equal CFR across periods. For each reported OR I also derived the z statistic its CI implies. These patient-level calculations treat every case as independent, as a crude OR from a 2×2 table does. The cases are concentrated in a few outbreaks: the 2014 outbreaks in Liberia and Sierra Leone alone account for 72.2% of all cases. So I also ran three outbreak-level analyses, which the paper does not report: a Spearman correlation of outbreak CFR with year, a quasi-binomial logistic model, and a DerSimonian–Laird random-effects meta-regression. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Does the cited OR show "no change"? Abstract, Results 1.6 (95% CI 1.5 to 1.8), p<0.001; "did not change substantially" CI excludes 1; implied z = 10.1. By its own statistics the change is significant
Same OR in the body Results text 1.6 (95% CI 1.5 to 1.6); "no statistically significant change" CI excludes 1 (implied z ≈ 28); also a different CI from the Abstract's for the same quantity
Table 3 ORs Table 3 1.6, 1.7, 1.4, 1.6; all CIs above 1; all P < 0.001 every CI excludes 1; none is a "no change" result
Direction of the OR, using Table 3's CFRs Table 3; Results 62% vs 71%, reported as "OR in 2019 vs 1976" = 1.6 (0.62/0.38) ÷ (0.71/0.29) = 0.67 (0.56 to 0.79), p = 4.1 × 10⁻⁶; the inverse is 1.50 (1.26 to 1.78). A fall in CFR gives an OR below 1
Deaths per period Table 3 vs Table 1 454, 348, 733, 13 288 454, 348, 733, 13 288 (identical)
CFR, 1976–1986 Table 3 71% 454/637 = 71.3%
CFR, 1987–1997 Table 3 67% 348/467 = 74.5%
CFR, 1998–2008 Table 3 64% 733/1199 = 61.1%
CFR, 2009–2019 Table 3; Results 62% 13 288/32 053 = 41.5%
OR of death, 2009–2019 vs 1976–1986 from Table 1 — 0.285 (0.240 to 0.339), p ≈ 9 × 10⁻⁴⁶; the inverse is 3.50
Equal CFR across the four periods from Table 1 "no statistically significant change" χ² = 588, df = 3, p ≈ 4 × 10⁻¹²⁷ (patient level)
Overall CFR Abstract "66% (62 to 71)", labelled a 95% CI pooled 14 823/34 356 = 43.1% (binomial 95% CI 42.6 to 43.7); unweighted mean of the 34 outbreak CFRs 61.8%. 66.0% is the mean of the four Table 3 CFRs and 62–71 is their range
Total deaths Table 1; Abstract vs Results text 14 823 vs "approximately 14 853" Table 1 rows sum to 14 823; the text differs by 30
CFR by sex Table 2 male 46, female 54 male 6373/15 803 = 40.3%, female 8449/18 553 = 45.5%. 46.0% and 54.0% are each sex's share of all cases
Outbreak-level trend (not in the paper) from Table 1 — Spearman ρ = −0.16 (p = 0.35); quasi-binomial OR 2009–2019 vs 1976–1986 0.29 (0.04 to 2.09), p = 0.21, dispersion 122; random-effects meta-regression OR per decade 0.81 (0.62 to 1.05), p = 0.11

The table supports three findings.

  1. The reported inference contradicts the reported wording. Every OR the paper prints (Abstract, Results, the four Table 3 columns) has a CI that excludes 1 and a P value below 0.001. The sentences that cite them say there was no significant change.
  2. The reported OR does not match the reported CFRs. For CFRs of 71% and then 62%, the OR of death for the later period is 0.67. Its inverse, 1.50 (1.26 to 1.78 with the Table 1 period sizes), is close to the reported 1.6 but describes the earlier period against the later. Neither the Abstract's CI (1.5 to 1.8) nor the Results' CI (1.5 to 1.6) is reproduced.
  3. Table 3's CFRs, apart from the first period, do not follow from Table 1. Table 3's death counts match Table 1 exactly. Its CFRs for 1987–1997, 1998–2008 and 2009–2019 differ from deaths ÷ cases by −7.5, +2.9 and +20.5 percentage points. The largest gap is in the period that holds 93% of all cases (32 053 of 34 356). They do not match the unweighted means of the outbreak CFRs either (76.5%, 53.5%, 64.9%, 58.9%). The Abstract's "66% (62 to 71)" is reproduced exactly by the mean and range of the four Table 3 CFRs, not by any pooled estimate or confidence interval.

Figure

Reported and recomputed odds ratios of death for the later versus the earliest period, on one log axis

Figure 1. Odds ratios of death with 95% CIs on a log scale. The thin vertical line is OR = 1 (no change). Orange: every OR the paper reports for the CFR trend (Abstract, Results text, and the four columns of Table 3). All lie to the right of 1 with intervals that exclude it, and all carry P < 0.001, yet the paper says there was no significant change. Blue diamonds are recomputed from the paper's own numbers. The upper one is the OR implied by Table 3's CFRs (62% vs 71%). The lower one is the crude OR for 2009–2019 vs 1976–1986 from the Table 1 counts. Both lie to the left of 1, the opposite direction from the reported 1.6. The hollow blue square is the same Table 1 comparison with a CI widened for variation between outbreaks (quasi-binomial logistic model, dispersion 122). It is not an analysis the paper reports. It shows that the data are compatible with no change once outbreaks rather than patients are treated as the unit.

Period case fatality rates as reported and as computed from Table 1

Figure 2. Case fatality rate for each Table 3 period and overall, on an axis from 0 to 100%. Orange circles: Table 3's CFRs and the Abstract's overall 66%. Blue diamonds: deaths ÷ cases from the outbreak rows of Table 1, grouped so that the deaths equal Table 3's. The first period agrees (71% vs 71.3%). The others differ, most of all in 2009–2019 (62% vs 41.5%) and overall (66% vs 43.1%).

Why it matters

The CFR trend is the only quantitative inference in the review. It is stated in the Abstract Results, the Abstract Conclusions and the Results text, and the Abstract Conclusions present it as evidence for the review's framing ("continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates"). A reader who takes the Abstract at face value gets a claim (no significant change), a statistic that contradicts it (CI excluding 1, p<0.001), and an overall CFR (66%) that is about 23 percentage points above the pooled figure implied by the paper's own outbreak table (43.1%).

The paper's numbers do not settle what the correct conclusion is, and this write-up does not claim to. Two readings are defensible, and neither matches the paper.

Either way, the published statistics cannot support the published sentence, and the overall CFR and three of the four period CFRs are not what the paper's own table implies. The review's qualitative synthesis (epidemiological, health-system and sociocultural challenges) and its recommendations (one-health surveillance, community engagement, research on diagnostics, drugs and vaccines, health-system resilience) do not depend on these numbers.

Explanations considered

Status

Suggested correction

  1. Replace "did not change substantially over time" (Abstract), "there was no statistically significant change over time" (Results) and "there are no significant changes in case fatality rates" (Abstract Conclusions) with wording that matches whatever analysis is reported. The current ORs, with CIs excluding 1 and P < 0.001, indicate a significant difference.
  2. State the comparison and direction of every OR in Table 3 (which period against which reference), and the unit of analysis. Report a single, consistent CI for the "2019 vs 1976" OR; the Abstract gives 1.5 to 1.8 and the Results 1.5 to 1.6.
  3. Recompute the Table 3 CFRs as deaths ÷ cases from Table 1 (71.3%, 74.5%, 61.1%, 41.5%) and add the case counts to the table, or explain how the printed 67%, 64% and 62% were derived.
  4. Replace the overall "66% (62 to 71)" with the pooled CFR from Table 1 (43.1%) and a valid interval, or label it as the mean and range of the period CFRs if that is what it is.
  5. Because CFR varies widely between outbreaks and one region contributes most of the cases, consider a trend analysis that accounts for clustering by outbreak (for example random-effects or quasi-binomial models). Base the conclusion on its estimate and interval.
  6. Minor: correct "approximately 14 853" deaths to 14 823 (Table 1 total), and change the Table 2 "CFR (%)" entries for sex (46, 54) to 40.3 and 45.5, or relabel the column as the share of cases. The Table 2 age groups sum to 33 456 cases, 900 fewer than the 34 356 total, and the table does not say how the remaining cases are classified.
Recomputation script (python recompute.py.txt)
"""Recompute the case-fatality trend statistics of PMC7061886
(BMJ Glob Health 2020, doi:10.1136/bmjgh-2019-001955).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC:
  Table 1: year, cases and deaths of each of the 34 outbreaks (printed CFR column also kept)
  Table 2: cases and deaths by sex; cases by age group and source of infection
  Table 3: deaths, CFR (%), OR (95% CI) and P value for four periods
  Abstract and Results text: overall CFR, ORs "in 2019 vs 1976"
Multi-year outbreaks in Table 1 (e.g. "2000-2001") are assigned to their first year.
With that rule the period death counts equal Table 3's exactly.
"""
import itertools
import math
import os

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ------------------------------------------------------------------ inputs (from the paper)
# Table 1: (first year, country, cases, deaths, printed CFR %)
T1 = [
    (1976, "DRC", 318, 280, 88), (1976, "South Sudan", 284, 151, 53), (1977, "DRC", 1, 1, 100),
    (1979, "South Sudan", 34, 22, 65), (1994, "Gabon", 52, 31, 60), (1994, "Cote d'Ivoire", 1, 0, 0),
    (1995, "DRC", 315, 250, 79), (1996, "Gabon", 37, 21, 57), (1996, "Gabon", 60, 45, 75),
    (1996, "South Africa", 2, 1, 50), (2000, "Uganda", 425, 224, 53), (2001, "Gabon", 65, 53, 82),
    (2001, "Rep. Congo", 57, 43, 75), (2002, "Rep. Congo", 143, 128, 89), (2003, "Rep. Congo", 35, 29, 83),
    (2004, "South Sudan", 17, 7, 41), (2005, "Rep. Congo", 12, 10, 83), (2007, "DRC", 264, 187, 71),
    (2007, "Uganda", 149, 37, 25), (2008, "DRC", 32, 15, 47), (2011, "Uganda", 1, 1, 100),
    (2012, "Uganda", 11, 4, 36), (2012, "DRC", 36, 13, 36), (2012, "Uganda", 6, 3, 50),
    (2014, "DRC", 69, 49, 71), (2014, "Guinea", 3814, 2544, 67), (2014, "Liberia", 10678, 4810, 45),
    (2014, "Sierra Leone", 14124, 3659, 26), (2014, "Nigeria", 20, 8, 40), (2014, "Mali", 8, 6, 75),
    (2017, "DRC", 8, 4, 50), (2018, "DRC", 54, 33, 61), (2018, "DRC", 3220, 2150, 67),
    (2019, "Uganda", 4, 4, 100),
]
T1_TOTAL = (34356, 14823)            # Table 1 "Total" row; also Abstract
TEXT_DEATHS = 14853                  # Results, "Context of EVD": "approximately 14 853 people have died"
PERIODS = [(1976, 1986), (1987, 1997), (1998, 2008), (2009, 2019)]
LABELS = ["1976–1986", "1987–1997", "1998–2008", "2009–2019"]
T3_DEATHS = [454, 348, 733, 13288]
T3_CFR = [71, 67, 64, 62]
T3_OR = [(1.6, 1.4, 1.8), (1.7, 1.5, 1.9), (1.4, 1.3, 1.6), (1.6, 1.5, 1.6)]   # all P < 0.001
ABS_OR = (1.6, 1.5, 1.8)             # Abstract: "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001"
RES_OR = (1.6, 1.5, 1.6)             # Results:  "OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)"
ABS_CFR = (66, 62, 71)               # Abstract: "overall case fatality rate (95% CI) was 66% (62 to 71)"
T2_SEX = {"Male": (15803, 6373, 46), "Female": (18553, 8449, 54)}   # cases, deaths, printed CFR
T2_AGE = {"0–15": 7215, "15–60": 24049, "60+": 2192}
T2_SOURCE = {"Health facility": 13742, "Household": 20614}


def orci(a, b, c, d):
    """OR of a/b vs c/d with Woolf (log) 95% CI and two-sided Wald P."""
    o = (a * d) / (b * c)
    se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    lo, hi = math.exp(math.log(o) - Z * se), math.exp(math.log(o) + Z * se)
    return o, lo, hi, 2 * stats.norm.sf(abs(math.log(o) / se))


yr = np.array([r[0] for r in T1], float)
n = np.array([r[2] for r in T1], float)
dth = np.array([r[3] for r in T1], float)
C, D = int(n.sum()), int(dth.sum())

print("=== 1. Totals (Table 1) ===")
print(f"sum of 34 outbreaks: cases {C} (Table 1 total {T1_TOTAL[0]}), deaths {D} (Table 1 total {T1_TOTAL[1]}; "
      f"Results text {TEXT_DEATHS}, difference {TEXT_DEATHS - D})")
print(f"pooled overall CFR = {D}/{C} = {100 * D / C:.1f}%")
print("Table 1 printed CFR column vs deaths/cases, where the rounding differs:")
for r in T1:
    if int(100 * r[3] / r[2] + 0.5) != r[4]:
        print(f"  {r[0]} {r[1]}: {r[3]}/{r[2]} = {100 * r[3] / r[2]:.2f}% printed {r[4]}")

print("\n=== 2. Periods of Table 3, rebuilt from Table 1 ===")
P = []
for (a, b), lab, rd, rc in zip(PERIODS, LABELS, T3_DEATHS, T3_CFR):
    m = (yr >= a) & (yr <= b)
    c, d = int(n[m].sum()), int(dth[m].sum())
    P.append((c, d))
    print(f"{lab}: {m.sum():2d} outbreaks, cases {c:5d}, deaths {d:5d} (Table 3: {rd}; equal: {d == rd}), "
          f"pooled CFR {100 * d / c:.1f}% (Table 3: {rc}%), difference {rc - 100 * d / c:+.1f} points; "
          f"unweighted mean of outbreak CFRs {np.mean(100 * dth[m] / n[m]):.1f}%")
pooled_cfr = [100 * d / c for c, d in P]

print("\n=== 3. Overall CFR, Abstract '66% (62 to 71)' ===")
se_p = math.sqrt((D / C) * (1 - D / C) / C)
print(f"pooled: {100 * D / C:.1f}% (binomial 95% CI {100 * (D / C - Z * se_p):.1f}–{100 * (D / C + Z * se_p):.1f})")
print(f"difference reported - pooled: {ABS_CFR[0] - 100 * D / C:.1f} points; "
      f"share of all cases in 2009–2019: {100 * P[3][0] / C:.1f}%; "
      f"pooled fall 1976–1986 to 2009–2019: {pooled_cfr[0] - pooled_cfr[3]:.1f} points")
print(f"unweighted mean of the 34 outbreak CFRs: {np.mean(100 * dth / n):.1f}%")
print(f"unweighted mean of the four Table 3 CFRs: {np.mean(T3_CFR):.1f}%; their range: {min(T3_CFR)}–{max(T3_CFR)}")

print("\n=== 4. What the reported ORs themselves imply ===")
for lab, (o, lo, hi) in [("Abstract", ABS_OR), ("Results", RES_OR)] + \
        [(f"Table 3 {l}", t) for l, t in zip(LABELS, T3_OR)]:
    se = (math.log(hi) - math.log(lo)) / (2 * Z)
    zz = math.log(o) / se if se > 0 else float("inf")
    print(f"{lab:22s} OR {o} ({lo} to {hi}): CI excludes 1: {lo > 1}; implied z = {zz:.1f}")
print("Abstract and Results print different CIs for the same 'OR in 2019 vs 1976':", ABS_OR[1:], RES_OR[1:])

print("\n=== 5. Direction: OR of death, later vs earlier ===")
c1, d1 = P[0]
c4, d4 = P[3]
o_rep = orci(0.62 * c4, c4 - 0.62 * c4, 0.71 * c1, c1 - 0.71 * c1)
print(f"from the reported CFRs 62% vs 71% (Table 1 period sizes): OR {o_rep[0]:.3f} "
      f"({o_rep[1]:.3f}–{o_rep[2]:.3f}), p = {o_rep[3]:.1e}; inverse {1 / o_rep[0]:.2f} "
      f"({1 / o_rep[2]:.2f}–{1 / o_rep[1]:.2f})")
print(f"odds at reported CFRs: {[round(p / (100 - p), 2) for p in T3_CFR]}; "
      f"0.62/0.38 / (0.71/0.29) = {(0.62 / 0.38) / (0.71 / 0.29):.3f}")
o_crude = orci(d4, c4 - d4, d1, c1 - d1)
print(f"from Table 1 counts, 2009–2019 vs 1976–1986: OR {o_crude[0]:.3f} ({o_crude[1]:.3f}–{o_crude[2]:.3f}), "
      f"p = {o_crude[3]:.1e}; inverse {1 / o_crude[0]:.2f}")
print("pairwise crude ORs (row period vs column period) from Table 1:")
for i, j in itertools.combinations(range(4), 2):
    (ci, di), (cj, dj) = P[i], P[j]
    o = orci(dj, cj - dj, di, ci - di)
    print(f"  {LABELS[j]} vs {LABELS[i]}: {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}) p = {o[3]:.1e}")
tab = np.array([[d, c - d] for c, d in P])
chi2, pchi, dfchi, _ = stats.chi2_contingency(tab)
print(f"chi-square test of equal CFR across the 4 periods (patient-level): chi2 = {chi2:.0f}, df = {dfchi}, p = {pchi:.1e}")

print("\n=== 6. Allowing for between-outbreak variation (not in the paper) ===")
rho = stats.spearmanr(yr, dth / n)
print(f"Spearman correlation, outbreak CFR vs year (34 outbreaks): rho = {rho.statistic:.2f}, p = {rho.pvalue:.2f}")


def logit_fit(X):
    b = np.zeros(X.shape[1])
    for _ in range(100):
        p = 1 / (1 + np.exp(-X @ b))
        W = n * p * (1 - p)
        b = np.linalg.solve(X.T @ (W[:, None] * X), X.T @ (W * (X @ b) + dth - n * p))
    p = 1 / (1 + np.exp(-X @ b))
    cov = np.linalg.inv(X.T @ ((n * p * (1 - p))[:, None] * X))
    phi = np.sum((dth - n * p) ** 2 / (n * p * (1 - p))) / (len(n) - X.shape[1])
    return b, cov, phi


Xp = np.column_stack([np.ones_like(yr)] + [((yr >= a) & (yr <= b)).astype(float) for a, b in PERIODS[1:]])
b, cov, phi = logit_fit(Xp)
se_b, se_q = math.sqrt(cov[3, 3]), math.sqrt(cov[3, 3] * phi)
dfq = len(n) - Xp.shape[1]
q_or = (math.exp(b[3]), math.exp(b[3] - stats.t.ppf(.975, dfq) * se_q), math.exp(b[3] + stats.t.ppf(.975, dfq) * se_q),
        2 * stats.t.sf(abs(b[3] / se_q), dfq))
print(f"logistic model with period factors: OR 2009–2019 vs 1976–1986 = {math.exp(b[3]):.3f} "
      f"(binomial 95% CI {math.exp(b[3] - Z * se_b):.3f}–{math.exp(b[3] + Z * se_b):.3f}); "
      f"Pearson dispersion = {phi:.0f}")
print(f"  quasi-binomial (dispersion-scaled, t on {dfq} df) 95% CI {q_or[1]:.3f}–{q_or[2]:.3f}, p = {q_or[3]:.3f}")
Xt = np.column_stack([np.ones_like(yr), (yr - 1976) / 10])
bt, covt, phit = logit_fit(Xt)
set_q = math.sqrt(covt[1, 1] * phit)
dft = len(n) - 2
print(f"linear trend: OR per decade {math.exp(bt[1]):.3f}; binomial p = {2 * stats.norm.sf(abs(bt[1] / math.sqrt(covt[1, 1]))):.1e}; "
      f"quasi-binomial 95% CI {math.exp(bt[1] - stats.t.ppf(.975, dft) * set_q):.3f}–"
      f"{math.exp(bt[1] + stats.t.ppf(.975, dft) * set_q):.3f}, p = {2 * stats.t.sf(abs(bt[1] / set_q), dft):.3f}")
# DerSimonian-Laird random-effects meta-regression of outbreak logit CFR on decade (0.5 continuity correction)
dd, nn = dth + 0.5, n + 1
lg, v = np.log(dd / (nn - dd)), 1 / dd + 1 / (nn - dd)
w = 1 / v
B = np.linalg.solve(Xt.T @ (w[:, None] * Xt), Xt.T @ (w * lg))
Q = np.sum(w * (lg - Xt @ B) ** 2)
Pm = np.diag(w) - (w[:, None] * Xt) @ np.linalg.inv(Xt.T @ (w[:, None] * Xt)) @ (Xt.T * w)
tau2 = max(0.0, (Q - dft) / np.trace(Pm))
w = 1 / (v + tau2)
cvr = np.linalg.inv(Xt.T @ (w[:, None] * Xt))
B = cvr @ (Xt.T @ (w * lg))
sr = math.sqrt(cvr[1, 1])
print(f"random-effects meta-regression: tau^2 = {tau2:.2f}; OR per decade {math.exp(B[1]):.3f} "
      f"({math.exp(B[1] - Z * sr):.3f}–{math.exp(B[1] + Z * sr):.3f}), p = {2 * stats.norm.sf(abs(B[1] / sr)):.3f}")
share = {k: 100 * float(n[[r[1] == k and r[0] == 2014 for r in T1]].sum()) / C for k in ("Liberia", "Sierra Leone")}
print(f"share of all cases from the 2014 Liberia and Sierra Leone outbreaks: {share['Liberia'] + share['Sierra Leone']:.1f}%")

print("\n=== 7. Single-year reading of 'OR in 2019 vs 1976' ===")
m76, m19 = yr == 1976, yr == 2019
print(f"1976 outbreaks: {int(dth[m76].sum())}/{int(n[m76].sum())} = {100 * dth[m76].sum() / n[m76].sum():.1f}%; "
      f"2019 outbreak (Uganda): {int(dth[m19].sum())}/{int(n[m19].sum())} = 100% (OR undefined: no survivors)")
o = orci(2150, 3220 - 2150, dth[m76].sum(), n[m76].sum() - dth[m76].sum())
print(f"2018–2019 DRC outbreak (2150/3220 = {100 * 2150 / 3220:.1f}%) vs 1976: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}), p = {o[3]:.1e}")

print("\n=== 8. Attempts to reproduce the Table 3 ORs ===")
for i, (c, d) in enumerate(P):
    oc, od = C - c, D - d
    o = orci(d, c - d, od, oc - od)
    rr = (d / c) / (od / oc)
    se = math.sqrt(1 / d - 1 / c + 1 / od - 1 / oc)
    print(f"{LABELS[i]} vs all other periods: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}); risk ratio {rr:.2f} "
          f"({rr * math.exp(-Z * se):.2f}–{rr * math.exp(Z * se):.2f}); inverse risk ratio {1 / rr:.2f}; "
          f"period CFR / overall CFR {(d / c) / (D / C):.2f}   [Table 3: {T3_OR[i]}]")

print("\n=== 9. Other quantities (Table 2, Results) ===")
for s, (c, d, pr) in T2_SEX.items():
    print(f"{s}: CFR {d}/{c} = {100 * d / c:.1f}% (printed {pr}); share of cases {100 * c / C:.1f}%; share of deaths {100 * d / D:.1f}%")
print(f"male + female cases {sum(v[0] for v in T2_SEX.values())}, deaths {sum(v[1] for v in T2_SEX.values())}")
print(f"age 15–60 share of cases: {100 * T2_AGE['15–60'] / C:.1f}% (Results: 70%); age groups sum {sum(T2_AGE.values())}")
print(f"health facility + household cases: {sum(T2_SOURCE.values())}")

# ------------------------------------------------------------------ figures
BG, INK, MUTED, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
    "xtick.color": INK, "ytick.color": INK, "axes.edgecolor": MUTED,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: ORs on one log axis
rows = [
    ("Abstract: “OR in 2019 vs 1976”", ABS_OR, ORANGE, "o", "1.6 (1.5–1.8)"),
    ("Results text: “OR in 2019 vs 1976”", RES_OR, ORANGE, "o", "1.6 (1.5–1.6)"),
] + [(f"Table 3, column {l}", t, ORANGE, "o", f"{t[0]} ({t[1]}–{t[2]})") for l, t in zip(LABELS, T3_OR)] + [
    ("Table 3 CFRs 62% vs 71%", o_rep[:3], BLUE, "D", f"{o_rep[0]:.2f} ({o_rep[1]:.2f}–{o_rep[2]:.2f})"),
    ("Table 1 counts, 2009–19 vs 1976–86", o_crude[:3], BLUE, "D",
     f"{o_crude[0]:.2f} ({o_crude[1]:.2f}–{o_crude[2]:.2f})"),
    ("Same, outbreak-level CI", q_or[:3], BLUE, "s",
     f"{q_or[0]:.2f} ({q_or[1]:.2f}–{q_or[2]:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ypos = np.arange(len(rows))[::-1].astype(float)
ypos[-3:] -= 0.5
for y, (lab, (o, lo, hi), col, mk, txt) in zip(ypos, rows):
    hollow = mk == "s"
    ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
    ax.plot(o, y, mk, ms=8, color=col, mfc=BG if hollow else col, mew=2)
    ax.text(4.3, y, txt, va="center", ha="left", fontsize=10, color=col if col == BLUE else "#b8481c")
ax.axvline(1, color=MUTED, lw=0.8, zorder=0)
ax.text(1.1, ypos[0] + 0.85, "OR = 1, no change", ha="left", va="bottom", fontsize=10, color=MUTED)
ax.set_xscale("log")
ax.set_xlim(0.03, 4.2)
ax.set_xticks([0.05, 0.1, 0.2, 0.5, 1, 2])
ax.set_xticklabels(["0.05", "0.1", "0.2", "0.5", "1", "2"])
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(ypos[-1] - 0.7, ypos[0] + 1.5)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio of death (95% CI), log scale")
ax.text(0.9, ypos[0] + 0.85, "← lower CFR later", ha="right", va="bottom", fontsize=10, color=MUTED)
fig.legend(handles=[
    Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper (all P < 0.001)"),
    Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper’s numbers, patient-level CI"),
    Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
           label="Recomputed, CI allowing for variation between outbreaks (quasi-binomial)"),
], loc="lower left", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.02, 0.0))
fig.subplots_adjust(left=0.40, right=0.80, top=0.97, bottom=0.30)
save(fig, "fig1")

# Figure 2: period CFRs, reported vs pooled from Table 1
fig, ax = plt.subplots(figsize=(7.2, 3.6))
cats = LABELS + ["Overall"]
rep = T3_CFR + [ABS_CFR[0]]
rec = pooled_cfr + [100 * D / C]
x = np.arange(len(cats), dtype=float) * 1.4
x[-1] += 0.35
for xi, r, c in zip(x, rep, rec):
    ax.plot([xi - 0.07, xi + 0.07], [r, c], color=GRID, lw=2, zorder=1)
    ax.plot(xi - 0.07, r, "o", ms=9, color=ORANGE, zorder=3)
    ax.plot(xi + 0.07, c, "D", ms=8, color=BLUE, zorder=3)
    ax.text(xi - 0.19, r, f"{r}%", va="center", ha="right", fontsize=10, color="#b8481c")
    ax.text(xi + 0.19, c, f"{c:.1f}%", va="center", ha="left", fontsize=10, color=BLUE)
ax.set_xticks(x)
ax.set_xticklabels([f"{l}\n{c:,} cases".replace(",", " ") for l, (c, _) in zip(LABELS, P)] + [f"Overall\n{C:,} cases".replace(",", " ")])
ax.set_ylim(0, 100)
ax.set_ylabel("Case fatality rate (%)")
ax.set_xlim(-0.6, x[-1] + 0.6)
ax.axvline((x[-2] + x[-1]) / 2, color=GRID, lw=0.8)
ax.grid(axis="y", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
fig.legend(handles=[
    Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported (Table 3; Abstract for overall)"),
    Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="Deaths ÷ cases from Table 1"),
], loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
fig.subplots_adjust(left=0.1, right=0.97, top=0.86, bottom=0.2)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Abstract reports p=0.002 for an odds ratio whose own 95% CI (0.86 to 7.36) includes 1

BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-009400 · PMC9494011

Summary

The abstract reports that identifying as "gay or queer" was associated with discrimination from healthcare providers since COVID-19 ("OR=2.51, 95% CI 0.86 to 7.36, p=0.002"), and Table 3 marks the same estimate, which belongs to the queer/asexual/pansexual/questioning group (aOR 2.52), as ** (p≤0.01). The interval includes 1: reconstructing the p-value from it on the log-odds scale gives p ≈ 0.09, and p=0.002 would need an interval of about 1.40 to 4.50; the bisexual healthcare estimate (aOR 2.74, CI 0.96 to 7.81, also **) likewise gives p ≈ 0.06. These two estimates are the paper's only basis for saying that queer and bisexual respondents had higher odds of discrimination from healthcare providers, so those statements in the abstract, Results and Discussion are not supported by the paper's own intervals. The gay respondents' healthcare estimate (aOR 3.45, CI 1.23 to 9.72) does exclude 1, at p ≈ 0.02 rather than the p≤0.01 its marker indicates, so the broader finding of elevated odds for some subgroups survives.

What the paper reports

Abstract, Results:

Identifying as gay or queer was associated with increased odds of experiencing the same or more discrimination from government representatives (OR=1.89, 95% CI 1.04 to 3.45, p=0.045) and healthcare providers (OR=2.51, 95% CI 0.86 to 7.36, p=0.002) due to COVID-19.

Results, paragraph on the final (healthcare provider) adjusted model:

Sexual orientation was associated with increased odds of experiencing discrimination or violence from healthcare providers since COVID-19, with gay (OR=3.45, 95% CI 1.22 to 9.71), bisexual (OR=2.74, 95% CI 0.96 to 7.81) or queer/asexual/pansexual (OR=2.51, 95% CI 0.86 to 7.36) orientation being significantly associated.

Table 3, multivariable logistic regression. Each outcome has five columns: OR | aOR | significance marker | 95% CI lower | 95% CI upper. The outcomes are, in order, discrimination based on sexual orientation and/or gender identity, from police and/or military, from government representatives, and from health providers. Heterosexual respondents are the reference category.

Gay | 1.83 | 1.82 | * | 1.05 | 3.17 | 0.98 | 1.27 | * | 1.05 | 1.80 | 1.73 | 1.89 | * | 1.04 | 3.45 | 3.07 | 3.45 | ** | 1.23 | 9.72

Bisexual | 1.36 | 1.30 | | 0.73 | 2.28 | 0.77 | 0.72 | | 0.39 | 1.32 | 1.07 | 1.13 | | 0.61 | 2.09 | 2.58 | 2.74 | ** | 0.96 | 7.81

Queer/asexual/pansexual/questioning | 1.93 | 2.00 | * | 1.07 | 3.72 | 0.78 | 0.62 | | 0.32 | 1.19 | 1.53 | 1.30 | * | 1.21 | 2.47 | 2.91 | 2.52 | ** | 0.86 | 7.36

Table 3 footer:

*P≤0.05; **p≤0.01; ***p=0.0.

Results, paragraph on the government-representatives model:

though only a gay (OR=1.89, 95% CI 1.04 to 3.45) and queer/asexual/pansexual (OR=1.30, 95% CI 0.68 to 2.47) orientation were significant when compared with their heterosexual counterparts.

Results, paragraph on the police/military model:

Sexual orientation nor gender identity was not significantly associated with a change in odds of experiencing violence or discrimination from police and/or military.

Discussion, second paragraph:

Specifically, gay and queer/asexual/pansexual/questioning individuals had higher odds of experiencing violence and discrimination from police, government representatives and healthcare providers when compared with their heterosexual peers. Bisexual individuals also had higher odds of experiencing violence and discrimination from healthcare providers.

Methods, Statistical analyses. No other inference method, test direction or significance threshold is described:

multivariable logistic regression analyses were conducted to examine the factors associated with experiences of stigma and discrimination

Table 1. The reference category for all sexual-orientation contrasts is small:

Heterosexual | 74 | 0.95

What the paper's own numbers show

Method. For a logistic-regression odds ratio with a Wald 95% interval, the interval is symmetric on the log scale, so SE(log OR) = [ln(upper) − ln(lower)] / (2 × 1.96). Then z = ln(OR) / SE, and the two-sided p = 2·[1 − Φ(|z|)]. The geometric midpoint √(lower × upper) should reproduce the point estimate, and this checks that the OR and CI belong together. To allow for rounding, I also recomputed p over every combination of the OR and both bounds shifted by ±0.005, which is the "p range" below. The conclusion does not depend on the reconstruction method in any case: a two-sided 95% interval that contains 1 cannot go with p < 0.05.

Quantity Location Reported Recomputed from the paper's numbers
Healthcare providers, "gay or queer": OR 2.51 (0.86–7.36) Abstract p=0.002 CI midpoint 2.52; SE 0.548; z 1.68; p = 0.093 (rounding range 0.091–0.095)
Same estimate, queer/asexual/pansexual/questioning: aOR 2.52 (0.86–7.36) Table 3 ** (p≤0.01) p = 0.091 (0.090–0.093)
Same estimate, 2.51 (0.86–7.36) Results text "significantly associated" p = 0.093; CI includes 1
95% CI that OR 2.51 with p=0.002 would require — — z 3.09; SE 0.298; 1.40 to 4.50 (the reported interval is 1.84 times wider on the log scale)
Same interval, read as a 90% / 99% CI — stated as 95% CI p = 0.158 / 0.027 (neither is 0.002)
One-sided p for the same estimate — — 0.046
Healthcare providers, bisexual: aOR 2.74 (0.96–7.81) Table 3; Results text ** (p≤0.01); "significantly associated" p = 0.059 (0.058–0.061); CI includes 1
Healthcare providers, gay: aOR 3.45 (1.23–9.72) Table 3 ** (p≤0.01) p = 0.019 (0.018–0.019): significant at 0.05, but not at the p≤0.01 the marker indicates
Healthcare providers, gay: OR 3.45 (1.22–9.71) Results text "significantly associated" p = 0.019 (0.019–0.020)
Healthcare providers, lesbian: aOR 2.52 (0.50–12.87) Table 3 no marker p = 0.265 (shown for comparison; consistent)
Government representatives, gay: aOR 1.89 (1.04–3.45) Abstract; Table 3 p=0.045; * p = 0.037 (0.036–0.039); broadly consistent
Government representatives, queer: aOR 1.30 (1.21–2.47) Table 3 * CI midpoint 1.73, not 1.30: the row's OR and CI cannot both be right
Government representatives, queer: OR 1.30 (0.68–2.47) Results text "significant" CI midpoint 1.30 (coherent); p = 0.425 (0.415–0.435); CI includes 1
Police/military, queer: aOR 0.62 (0.32–1.19) Table 3 no marker p = 0.154; OR below 1 and not significant
Healthcare providers, disability: aOR 1.35 (1.07–1.71) Abstract; Table 3 p=0.009; ** p = 0.012 (0.010–0.015); consistent within rounding
Heterosexual reference group Table 1 n = 74 7758 − 6125 − 22 − 1158 − 379 = 74 (consistent)

The healthcare-provider interval for the queer/asexual/pansexual/questioning group is internally coherent: its geometric midpoint (2.516) matches the reported point estimate (2.51 in the text, 2.52 in Table 3). What disagrees with the other numbers is the abstract's p-value and the table's ** marker. The same holds for the bisexual row (midpoint 2.738 against 2.74).

Figure

Reported odds ratios and 95% CIs for discrimination from healthcare providers, with the p-values they imply

Figure 1. Adjusted odds ratios for discrimination from healthcare providers, by sexual orientation (reference: heterosexual, n = 74), on a log scale. The thin vertical line marks OR = 1. Orange dots and lines show each odds ratio and 95% CI as the paper reports it, and the orange text gives the significance the paper attaches to it (abstract p-value or Table 3 marker). Blue text gives the two-sided p recomputed from the same OR and CI with a log-scale Wald reconstruction. The dashed blue interval under the abstract row is the 95% CI (1.40–4.50) that an OR of 2.51 would need for p = 0.002 to be correct. The reported interval (0.86–7.36) is much wider and crosses 1. For the queer/asexual/pansexual/questioning and bisexual groups, the reported intervals include 1 (recomputed p = 0.09 and 0.06), although the abstract, Results text and Table 3 present them as significant at p = 0.002 or p ≤ 0.01. The gay-group interval excludes 1 (p = 0.019). All numbers come from the paper or from recompute.py.txt.

Why it matters

The abstract is the part of the paper most readers see, and it is where this estimate appears with p=0.002. At the conventional 0.05 level, the paper's own interval does not support a significant association between queer/asexual/pansexual/questioning identity and discrimination from healthcare providers. The same applies to bisexual identity (CI 0.96 to 7.81). The Discussion's statements that queer/asexual/pansexual/questioning and bisexual individuals "had higher odds" of discrimination from healthcare providers rest on these two estimates.

The abstract's wording, "gay or queer", also joins two estimates. The government figure (1.89, 1.04 to 3.45) is the gay estimate. The healthcare figure (2.51, 0.86 to 7.36) is the queer/asexual/pansexual/questioning estimate. The gay healthcare estimate is a different number (aOR 3.45, 1.23 to 9.72). A reader of the abstract cannot tell that the p=0.002 healthcare figure belongs to the 379-person queer/asexual/pansexual/questioning group rather than to gay respondents.

The same Discussion sentence also extends to police and government representatives, but the paper's other numbers do not support it for the queer group either. Its police/military aOR is 0.62 (0.32 to 1.19), and the Results state that sexual orientation was not significantly associated with police/military discrimination. Its government-representatives estimate appears as 1.30 with CI 1.21 to 2.47 in Table 3 (an interval whose midpoint is 1.73, not 1.30) and as 1.30 with CI 0.68 to 2.47 in the Results text. The text version is internally coherent and gives p ≈ 0.43.

What survives: the gay healthcare-provider estimate excludes 1 (p ≈ 0.02). So do the disability, immigrant and ethnic-minority estimates for healthcare providers in Table 3. The paper's broad finding that some marginalised subgroups reported higher odds of discrimination from healthcare providers therefore does not depend on the erroneous values. What fails is the claim of significance for the queer (and bisexual) healthcare-provider contrasts, which the abstract reports with a p-value that is over 40 times smaller than its own interval implies.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. Abstract, Results: report the healthcare-provider estimate with a p-value consistent with its interval (about p=0.09 if the interval 0.86 to 7.36 is correct), and do not describe the association as significant. Alternatively, if the interval is the wrong value, give the correct one. Attribute each estimate to the right group: the government estimate (1.89, 1.04 to 3.45) is for gay respondents, and the healthcare estimate (2.51, 0.86 to 7.36) is for queer/asexual/pansexual/questioning respondents. If the abstract is meant to describe gay respondents' healthcare-provider estimate, that is aOR 3.45 (95% CI 1.23 to 9.72), p ≈ 0.02.
  2. Table 3, health-provider columns: remove the ** markers from the queer/asexual/pansexual/questioning row (CI 0.86 to 7.36) and the bisexual row (CI 0.96 to 7.81), and change the gay row's ** to * (CI 1.23 to 9.72, p ≈ 0.02), unless the authors' output shows different intervals.
  3. Table 3, government-representatives columns, queer/asexual/pansexual/questioning row: reconcile the lower bound (1.21 in the table vs 0.68 in the text) and the * marker with the OR of 1.30.
  4. Results (healthcare-provider and government-representatives paragraphs) and Discussion: restate which sexual-orientation groups had intervals excluding 1, specifically that the queer/asexual/pansexual/questioning and bisexual healthcare-provider estimates, and the queer police and government estimates, were not statistically significant at the 0.05 level.
  5. Methods: state how p-values and confidence intervals were computed (Wald or likelihood-based, two-sided) and the significance threshold used.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the p-values implied by the odds ratios and 95% CIs reported in
PMC9494011 (BMJ Glob Health 2022, doi:10.1136/bmjgh-2022-009400), and redraw
the figure for the write-up.  Run:  python recompute.py.txt
Every input number below is copied from the article (abstract, Results text,
Table 1 or Table 3).  Outputs fig1.svg / fig1.png next to this file."""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.ppf(0.975)


def wald(or_, lo, hi, level=0.95):
    """Log-scale Wald reconstruction: SE from the CI width, p from OR/SE."""
    z = norm.ppf(1 - (1 - level) / 2)
    se = (np.log(hi) - np.log(lo)) / (2 * z)
    gm = np.sqrt(lo * hi)                     # point estimate the CI implies
    p = 2 * norm.sf(abs(np.log(or_)) / se)    # two-sided, from stated OR
    p_gm = 2 * norm.sf(abs(np.log(gm)) / se)  # two-sided, from CI midpoint
    p_one = norm.sf(abs(np.log(or_)) / se)    # one-sided
    return gm, se, p, p_gm, p_one


def p_range(or_, lo, hi, d_or=0.005, d_ci=0.005):
    """Smallest and largest two-sided p compatible with 2-decimal rounding."""
    ps = []
    for o in (or_ - d_or, or_ + d_or):
        for l in (lo - d_ci, lo + d_ci):
            for h in (hi - d_ci, hi + d_ci):
                ps.append(wald(o, l, h)[2])
    return min(ps), max(ps)


# label, OR, lower, upper, what the paper reports about significance
ROWS = [
    ("Abstract: healthcare, 'gay or queer'",         2.51, 0.86, 7.36, "p=0.002"),
    ("Results text: healthcare, queer/asex./pans.",  2.51, 0.86, 7.36, "'significantly associated'"),
    ("Table 3: healthcare, queer/asex./pans./quest.", 2.52, 0.86, 7.36, "** (p<=0.01)"),
    ("Table 3: healthcare, bisexual",                2.74, 0.96, 7.81, "** (p<=0.01)"),
    ("Results text: healthcare, bisexual",           2.74, 0.96, 7.81, "'significantly associated'"),
    ("Table 3: healthcare, gay",                     3.45, 1.23, 9.72, "** (p<=0.01)"),
    ("Results text: healthcare, gay",                3.45, 1.22, 9.71, "'significantly associated'"),
    ("Table 3: healthcare, lesbian",                 2.52, 0.50, 12.87, "(no marker)"),
    ("Abstract/Table 3: government, gay",            1.89, 1.04, 3.45, "p=0.045 / *"),
    ("Table 3: government, queer/asex./pans./quest.", 1.30, 1.21, 2.47, "*"),
    ("Results text: government, queer/asex./pans.",  1.30, 0.68, 2.47, "'significant'"),
    ("Table 3: police, queer/asex./pans./quest.",    0.62, 0.32, 1.19, "(no marker)"),
    ("Abstract/Table 3: healthcare, disability",     1.35, 1.07, 1.71, "p=0.009 / **"),
]

print("Log-scale Wald reconstruction from each reported OR and 95% CI")
print(f"{'quantity':47s} {'OR':>5} {'95% CI':>12} {'CI mid':>6} {'SE':>6} "
      f"{'p(2-s)':>7} {'p range (rounding)':>18} {'p(1-s)':>7}  reported")
for lab, o, l, h, rep in ROWS:
    gm, se, p, p_gm, p1 = wald(o, l, h)
    lo_p, hi_p = p_range(o, l, h)
    print(f"{lab:47s} {o:5.2f} {l:5.2f}-{h:<6.2f} {gm:6.2f} {se:6.3f} "
          f"{p:7.3f} {lo_p:8.3f}-{hi_p:<9.3f} {p1:7.3f}  {rep}")

# What would p=0.002 require for OR=2.51?
z_req = norm.isf(0.002 / 2)
se_req = np.log(2.51) / z_req
lo_req, hi_req = 2.51 * np.exp(-Z95 * se_req), 2.51 * np.exp(Z95 * se_req)
print(f"\nFor OR=2.51 with two-sided p=0.002: z={z_req:.2f}, SE={se_req:.3f}, "
      f"required 95% CI {lo_req:.2f} to {hi_req:.2f}")
_, se_rep, _, _, _ = wald(2.51, 0.86, 7.36)
p_rep = wald(2.51, 0.86, 7.36)[2]
print(f"Reported CI implies SE={se_rep:.3f}, z={np.log(2.51)/se_rep:.2f}; "
      f"ratio of SEs = {se_rep/se_req:.2f}")
print(f"Recomputed p / reported p = {p_rep/0.002:.0f}; "
      f"one-sided p / 0.002 = {wald(2.51, 0.86, 7.36)[4]/0.002:.0f}")
for lab, l, h in (("queer healthcare", 0.86, 7.36), ("bisexual healthcare", 0.96, 7.81),
                  ("queer government (Table 3)", 1.21, 2.47)):
    print(f"CI geometric midpoint, {lab}: {np.sqrt(l*h):.3f}")

# Alternative readings of the interval level
print("\nIf the interval 0.86-7.36 were read at another confidence level:")
for lvl in (0.90, 0.95, 0.99):
    print(f"  {int(lvl*100)}% CI -> two-sided p = {wald(2.51, 0.86, 7.36, lvl)[2]:.3f}")

# Reference group size (Table 1) and the identity check
n, gay, les, bi, q, het = 7758, 6125, 22, 1158, 379, 74
print(f"\nTable 1: heterosexual reference n = {het}; "
      f"7758 - (gay+lesbian+bisexual+queer) = {n - gay - les - bi - q}")

# ---------------------------------------------------------------- figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
                     "text.color": INK, "axes.labelcolor": INK,
                     "xtick.color": INK2, "ytick.color": INK})

# Healthcare-provider model, sexual-orientation rows (Table 3), plus the
# abstract's sentence; reported marker vs recomputed p.
fig_rows = [
    ("Abstract (healthcare,\n'gay or queer')", 2.51, 0.86, 7.36, "p = 0.002"),
    ("Table 3: queer/asexual/\npansexual/questioning", 2.52, 0.86, 7.36, "** (p ≤ 0.01)"),
    ("Table 3: bisexual", 2.74, 0.96, 7.81, "** (p ≤ 0.01)"),
    ("Table 3: gay", 3.45, 1.23, 9.72, "** (p ≤ 0.01)"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.3), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.64, top=0.89, bottom=0.13)
ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(0.4, 16)
ticks = [0.5, 1, 2, 4, 8, 16]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1, color=INK2, lw=1)
ax.text(1.06, -0.75, "OR = 1 (no association)", color=INK2, fontsize=10,
        ha="left", va="center")

ys = np.array([4.9, 3.1, 1.6, 0.1])  # extra room under the abstract row
tr = ax.get_yaxis_transform()  # x in axes fraction, y in data units
for y, (lab, o, l, h, rep) in zip(ys, fig_rows):
    p = wald(o, l, h)[2]
    ax.plot([l, h], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
    ax.plot(o, y, "o", ms=9, color=ORANGE, mec=BG, mew=1)
    ax.text(1.03, y + 0.30, f"{o:.2f} ({l:.2f}–{h:.2f})", transform=tr,
            color=INK2, fontsize=10, va="center", ha="left")
    ax.text(1.03, y, f"reported: {rep}", transform=tr, color=ORANGE,
            fontsize=10, va="center", ha="left")
    ax.text(1.03, y - 0.30, f"recomputed: p = {p:.3f}", transform=tr,
            color=BLUE, fontsize=10, va="center", ha="left", fontweight="bold")

# interval that p = 0.002 would require (abstract row)
y0 = ys[0] - 0.75
ax.plot([lo_req, hi_req], [y0, y0], color=BLUE, lw=2, ls=(0, (4, 2)))
ax.plot(2.51, y0, "D", ms=8, color=BLUE, mec=BG, mew=1)
ax.text(1.03, y0, f"CI that p = 0.002 needs:\n{lo_req:.2f}–{hi_req:.2f}",
        transform=tr, color=BLUE, fontsize=10, va="center", ha="left")

ax.set_yticks(list(ys) + [y0])
ax.set_yticklabels([r[0] for r in fig_rows] + ["abstract OR, if\np = 0.002 were right"])
for t in ax.get_yticklabels()[-1:]:
    t.set_color(BLUE)
ax.set_ylim(-1.05, ys[0] + 0.6)
ax.set_xlabel("Adjusted OR, discrimination from healthcare\nproviders (log scale), with 95% CI")
for s_ in ("top", "right", "left"):
    ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)

from matplotlib.lines import Line2D
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8,
                  label="Reported OR and 95% CI (paper)"),
           Line2D([], [], color=BLUE, lw=2, ls=(0, (4, 2)), marker="D", ms=7,
                  label="Recomputed from the paper's numbers")]
fig.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, 0.99),
           ncol=2, frameon=False, fontsize=10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png")

Abstract reports whole-cohort estimates as sexual-minority results, making two non-significant subgroup findings appear significant

BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-010231 · PMC9791420

Summary

The paper's second aim is to compare the mental health of sexual minority adolescents with and without HIV, and the abstract reports that "Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013)", along with three significant estimates for sexual minority adolescents with unknown HIV status. All four of these estimates, with both CI limits and the p-value, match the HIV-status rows of Table 1 exactly. Table 1 is the whole-cohort model (N=1247, heterosexual and sexual minority adolescents together), and the main text itself labels 2.42 as the result "In the overall cohort". The sexual-minority-only model in Table 2 (N=497) gives different values, and two of the four claims are not significant there: HIV-positive status and high anxiety, AOR 2.09 (0.93 to 4.69), p=0.073; unknown status and suicidal attempt/ideation, AOR 1.47 (0.98 to 2.22), p=0.064. The paper's headline comparison of sexual minority and heterosexual adolescents is unaffected, but the abstract's Results sentence on the second aim misstates which findings reached significance and contradicts the abstract's own Conclusion.

What the paper reports

Abstract, Background (the second aim):

and to determine associations between the mental health status of sexual minority adolescents living with and without HIV.

Abstract, Methods:

A second multivariate regression model was developed to establish associations between HIV status among sexual minority individuals and the dependent variables.

Abstract, Results:

Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.82, 95% CI: 2.78 to 5.27; p<0.001), high general anxiety (AOR: 3.09; 95% CI: 2.29 to 4.17; p<0.001) and suicidal attempt/ideation (AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).

Abstract, Conclusion:

Although there was no significant difference in the mental health status of sexual minority adolescents living and not living with HIV, sexual minority adolescents with unknown HIV status reported worse mental health than their HIV-negative peers.

Methods, Data analysis:

The second set of multivariate regression analyses included data from only sexual minority individuals.

Significance was set at 5%.

Results, third paragraph (describing Table 1):

In the overall cohort, self-reported HIV-positive status among adolescents was associated with increased odds of high general anxiety (AOR: 2.42; 95% CI:1.21 to 4.84; p=0.013), but not depression or suicidal attempt/ideation.

Results, fourth paragraph (describing Table 2):

Having an unknown HIV status was associated with significantly higher odds of experiencing depressive symptoms (AOR: 3.11, 95% CI: 1.76 to 5.49, p<0.001) and high general anxiety (AOR: 2.32, 95% CI: 1.55 to 3.48, p<0.001).

Table 1, "Multivariate logistic regression analysis to determine association between sexual orientation, HIV status, mental health status of adolescents 13–19 years old in Nigeria (N=1247)", HIV-positive row (columns: total | depressive symptoms yes | no | AOR (95% CI) p | high general anxiety yes | no | AOR (95% CI) p):

Positive | 41(3.3) | 29(70.7) | 12(29.3) | 1.81(0.85 to 3.88)0.125 | 17(41.5) | 24(58.5) | 2.42(1.21 to 4.84)0.013

Table 2, "Multivariate logistic regression analysis to determine association between HIV status and mental health status among sexual minority adolescents 13–19 years old in Nigeria (N=497)", HIV-positive row (columns as above, then suicidal ideation yes | no | AOR (95% CI) p):

HIV positive | 29 (5.8) | 24(82.8) | 5(17.2) | 2.25(0.78 to 6.48)0.132 | 14(48.3) | 15(51.7) | 2.09(0.93 to 4.69)0.073 | 12(41.4) | 17(58.6) | 1.47(0.66 to 3.28)0.352

Table 2, unknown-status row:

Unknown HIV status | 222 (44.7) | 201(90.5) | 21(9.5) | 3.11(1.76 to 5.49)<0.001 | 130(58.6) | 92(41.4) | 2.32(1.55 to 3.48)<0.001 | 109(49.1) | 113(50.9) | 1.47(0.98 to 2.22)0.064

Results, fifth paragraph (exploratory analyses, describing supplementary Table S1, heterosexual adolescents only):

Exploratory analyses among heterosexual participants showed that, like sexual minority individuals, unknown HIV status was associated with an increased likelihood of reporting depressive and anxiety symptoms and self-harm/suicidal ideation in this study. The adjusted ORs were comparable in these latter multivariate models among heterosexual and sexual minority participants, though these were relatively larger in sexual minority individuals (see online supplemental table 1).

Supplementary Table S1 (N=750), unknown-status estimates for depressive symptoms, high general anxiety and suicidal ideation:

3.92 (2.66-5.78) <0.001 | 4.16 (2.63-6.56) <0.001 | 2.06 (1.28-3.30) 0.003

Discussion, first paragraph:

Among sexual minority adolescents, those with unknown HIV status had significantly higher odds of reporting depressive symptoms and generalised anxiety disorder. These findings partially support the hypotheses.

What the paper's own numbers show

Method. I matched each of the four estimates in the abstract's Results sentence on HIV status against the HIV-status rows of Table 1 (whole cohort) and Table 2 (sexual minorities only), comparing the AOR, both CI limits and the p-value. For every estimate I recomputed the Wald z and two-sided p implied by the printed 95% CI (SE = [ln(upper) − ln(lower)] / 3.92; z = ln(AOR)/SE), to confirm that each table is internally consistent and that the printed p-values are two-sided. As a model-free check on the key subgroup estimate, I computed crude odds ratios from the Table 2 counts. Finally, I checked that Table 2 and supplementary Table S1 (heterosexual adolescents only, N=750) partition the Table 1 cohort, which confirms which table covers which population. I also compared the unknown-status estimates in Table S1 with those in Table 2, as the Results' exploratory-analysis paragraph does. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
HIV-positive vs negative, high general anxiety, given for sexual minority adolescents Abstract, Results 2.42 (1.21 to 4.84), p=0.013 Identical to Table 1 (whole cohort); Wald p from CI 0.0125
Same estimate, whole cohort Table 1; Results ¶3 ("In the overall cohort") 2.42 (1.21 to 4.84), p=0.013 SE 0.354, z 2.50, p 0.0125; CI excludes 1
Same estimate, sexual minorities only Table 2 2.09 (0.93 to 4.69), p=0.073 SE 0.413, z 1.79, p 0.074; CI includes 1, not significant
Crude OR, same comparison, sexual minorities Table 2 counts, 14/29 vs 79/246 — 1.97 (0.91 to 4.29), p=0.086
Unknown vs negative, depressive symptoms, given for sexual minority adolescents Abstract, Results 3.82 (2.78 to 5.27), p<0.001 Identical to Table 1; Table 2 value is 3.11 (1.76 to 5.49), p<0.001
Unknown vs negative, high general anxiety, given for sexual minority adolescents Abstract, Results 3.09 (2.29 to 4.17), p<0.001 Identical to Table 1; Table 2 value is 2.32 (1.55 to 3.48), p<0.001
Unknown vs negative, suicidal attempt/ideation, given for sexual minority adolescents Abstract, Results 1.65 (1.22 to 2.24), p=0.001 Identical to Table 1 (Wald p 0.0012); Table 2 value is 1.47 (0.98 to 2.22), p=0.064 (Wald p 0.065); not significant
Abstract estimates that match Table 1 / Table 2 exactly Abstract vs Tables 1 and 2 — 4 of 4 / 0 of 4
Abstract estimates with CI excluding 1 Abstract vs Table 2 4 of 4 2 of 4 for the same estimates in Table 2
Ratio of abstract AOR to Table 2 AOR Abstract vs Table 2 — 1.16, 1.23, 1.33, 1.12 (the abstract's values are larger than the subgroup estimates for all four)
HIV-positive vs negative, depressive symptoms / suicidal attempt/ideation, sexual minorities Table 2 2.25 (0.78 to 6.48), p=0.132 / 1.47 (0.66 to 3.28), p=0.352 Wald p 0.133 / 0.346; neither significant (abstract does not report these)
Unknown vs negative, depressive symptoms / high general anxiety / suicidal attempt/ideation, heterosexuals vs sexual minorities Table S1 vs Table 2; Results ¶5 ("like sexual minority individuals ... self-harm/suicidal ideation"; AORs "relatively larger in sexual minority individuals") S1: 3.92 / 4.16 / 2.06 (p=0.003); Table 2: 3.11 / 2.32 / 1.47 (p=0.064) The Table S1 AOR is larger than the Table 2 AOR for 3 of 3 outcomes; the Table 2 suicidal-ideation CI includes 1
Sample sizes of the HIV-status rows Tables 1, 2 and S1 N=1247, 497, 750 Rows sum to 1247, 497 and 750; Table 2 + Table S1 = Table 1 for each status (246+549=795; 29+12=41; 222+189=411)

The Wald p-values recomputed from the printed CIs agree with the printed p-values for every estimate in both tables (largest difference 0.009, for a non-significant Table 1 estimate, 0.703 vs 0.712), so both tables are internally consistent, the printed p-values are two-sided, and the discrepancy lies only in the abstract's attribution of Table 1 values to the sexual-minority subgroup.

A second, smaller inconsistency in Table 1. In the depressive-symptom and high-anxiety columns of Table 1, the yes/no counts in the "Negative" and "Unknown" HIV rows do not add up to those rows' totals: "Negative" (n=795) shows 332/79 and 195/216, which sum to 411, and "Unknown" (n=411) shows 330/465 and 126/669, which sum to 795. Adding the Table 2 and Table S1 counts gives 330/465 and 126/669 for HIV-negative adolescents and 332/79 and 195/216 for those with unknown status, so these count cells (and their percentages, e.g. 332/411 = 80.8%) appear to have been placed in each other's rows. The suicidal-ideation column and the HIV-positive row add up correctly, and the column totals (691, 338, 309) match the Results text. This does not change any AOR, but it reverses the descriptive picture: as printed, Table 1 shows 80.8% of HIV-negative and 41.5% of unknown-status adolescents with depressive symptoms, when the subgroup tables imply the reverse.

Figure

The four estimates the abstract gives for sexual minority adolescents are the whole-cohort Table 1 estimates; in the sexual-minority model (Table 2) two of them are not significant.

Figure 1. Left: the abstract's Results sentence on sexual minority adolescents by HIV status, with two of its four estimates (orange). Top right: the same two rows of Table 1, the whole-cohort model (N=1247), which carry identical numbers; the other two abstract estimates (3.82 and 3.09) also match Table 1 exactly. Bottom right: the corresponding rows of Table 2, the model restricted to sexual minority adolescents (N=497), which is the population the abstract names (blue). Both Table 2 confidence intervals include 1, so neither association is significant at the paper's stated 5% level. All numbers are printed in the paper; recompute.py.txt checks each p-value against its CI.

Dot-and-interval plot of the four abstract estimates against the Table 2 sexual-minority estimates.

Figure 2. Each of the four estimates the abstract reports for sexual minority adolescents (orange circles, identical to Table 1, whole cohort) next to the same estimate from the sexual-minority-only model in Table 2 (blue squares), on a log odds-ratio axis with the null value (OR = 1) as a thin vertical reference line. All four Table 2 estimates are smaller, and the intervals for HIV-positive status with high general anxiety and for unknown status with suicidal attempt/ideation cross 1.

Why it matters

The comparison of sexual minority adolescents living with and without HIV is one of the paper's two stated aims and has its own model (Table 2). The abstract's Results sentence on that aim reports four significant associations, but all four come from the whole-cohort model, where heterosexual adolescents make up 60% of the sample. In the subgroup the abstract names, HIV-positive status is not significantly associated with any of the three outcomes, and unknown status is not significantly associated with suicidal attempt/ideation; the two effects that do remain significant are 19% to 25% smaller than the abstract states (3.11 vs 3.82 and 2.32 vs 3.09).

The abstract is the part of the paper most often read, indexed and quoted. A reader who relies on it would conclude that HIV-positive sexual minority adolescents in this sample had significantly higher odds of high anxiety, and that unknown HIV status was associated with suicidal attempt/ideation in sexual minority adolescents. Neither is what the paper's subgroup analysis found. The abstract also contradicts itself: its Conclusion ("no significant difference in the mental health status of sexual minority adolescents living and not living with HIV") matches Table 2, while its Results sentence does not. Results paragraphs 3 and 4, the Discussion's summary of findings (its first paragraph) and the tables report the subgroup findings correctly; the Discussion's later paragraph on HIV status agrees with Table 2 that HIV-positive status had no significant effect, although it describes the comparison as one with "heterosexual peers" rather than with HIV-negative sexual minority peers, which is the Table 2 comparison. One sentence in the exploratory-analysis paragraph (Results, fifth paragraph) repeats the suicidal-ideation claim ("like sexual minority individuals, unknown HIV status was associated with ... self-harm/suicidal ideation"), and the same paragraph says the adjusted ORs were "relatively larger in sexual minority individuals", although the heterosexual unknown-status AORs in Table S1 (3.92, 4.16, 2.06) are larger than the sexual-minority ones in Table 2 (3.11, 2.32, 1.47) for all three outcomes. The headline comparison of sexual minority with heterosexual adolescents (AORs 5.54, 3.56 and 2.95) is not affected.

Explanations considered

Status

Suggested correction

Replace the abstract's Results sentence on HIV status with the estimates from the sexual-minority model (Table 2), for example:

Among sexual minority adolescents, HIV-positive status was not significantly associated with depressive symptoms, high general anxiety (AOR: 2.09; 95% CI: 0.93 to 4.69; p=0.073) or suicidal attempt/ideation, while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.11; 95% CI: 1.76 to 5.49; p<0.001) and high general anxiety (AOR: 2.32; 95% CI: 1.55 to 3.48; p<0.001), but not of suicidal attempt/ideation (AOR: 1.47; 95% CI: 0.98 to 2.22; p=0.064).

If the authors prefer to keep the whole-cohort estimates in the abstract, the sentence should say that they come from the whole cohort (sexual minority and heterosexual adolescents together, the model reported in Table 1), and the Table 2 results should be added for the sexual-minority comparison that the second aim and the Conclusion refer to.

In the exploratory-analysis paragraph of the Results, the phrase "like sexual minority individuals" should not extend to self-harm/suicidal ideation, which was not significantly associated with unknown status among sexual minority adolescents (Table 2: AOR 1.47; 95% CI: 0.98 to 2.22; p=0.064), and the statement that the adjusted ORs were "relatively larger in sexual minority individuals" should be checked against Table S1, whose unknown-status AORs are the larger ones.

In Table 1, the depressive-symptom and high-general-anxiety counts and percentages in the "Negative" and "Unknown" HIV-status rows should be checked against the data; the subgroup tables imply that they belong in each other's rows. Two CIs in the education rows also cannot contain their point estimates and look like typographical errors: Table 1, primary education, suicidal ideation, "0.28(1.13 to 0.64)", and Table 2, secondary education, depressive symptoms, "0.58(1.19 to 1.76)".

Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC9791420 write-up and regenerate the figures.

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Figures are written next to this file:
  fig1.svg/png, fig2.svg/png        figures used in writeup.md (provisional: fig1 = cand1, fig2 = cand2
                                    until a design is chosen from the candidates)
  candidates/cand1..cand5_*.svg/png five candidate figure designs, kept out of the deliverable files

All inputs are typed in from the article as archived in PubMed Central (PMC9791420):
Abstract, Results text, Table 1 (whole cohort, N=1247), Table 2 (sexual-minority
adolescents only, N=497), and the online supplement bmjgh-2022-010231supp001.pdf,
Table S1 (heterosexual adolescents only, N=750).
"""
import os
import shutil
import textwrap

import numpy as np
from scipy.stats import norm

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z975 = norm.ppf(0.975)

# ---------------------------------------------------------------------------
# 1. Inputs, as printed in the paper
# ---------------------------------------------------------------------------
# (AOR, lower, upper, printed p)
ABSTRACT_SM = {  # Abstract, Results: attributed to "Sexual minority adolescents"
    "HIV-positive: high anxiety":       (2.42, 1.21, 4.84, "0.013"),
    "Unknown status: depression":       (3.82, 2.78, 5.27, "<0.001"),
    "Unknown status: high anxiety":     (3.09, 2.29, 4.17, "<0.001"),
    "Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE1 = {  # Table 1, whole cohort (N=1247), HIV status rows vs HIV-negative
    "HIV-positive: depression":         (1.81, 0.85, 3.88, "0.125"),
    "HIV-positive: high anxiety":       (2.42, 1.21, 4.84, "0.013"),
    "HIV-positive: suicidal att./id.":   (1.15, 0.56, 2.36, "0.712"),
    "Unknown status: depression":       (3.82, 2.78, 5.27, "<0.001"),
    "Unknown status: high anxiety":     (3.09, 2.29, 4.17, "<0.001"),
    "Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE2 = {  # Table 2, sexual-minority adolescents only (N=497)
    "HIV-positive: depression":         (2.25, 0.78, 6.48, "0.132"),
    "HIV-positive: high anxiety":       (2.09, 0.93, 4.69, "0.073"),
    "HIV-positive: suicidal att./id.":   (1.47, 0.66, 3.28, "0.352"),
    "Unknown status: depression":       (3.11, 1.76, 5.49, "<0.001"),
    "Unknown status: high anxiety":     (2.32, 1.55, 3.48, "<0.001"),
    "Unknown status: suicidal att./id.": (1.47, 0.98, 2.22, "0.064"),
}
TABLES1_HIVPOS_ANX = (3.12, 0.80, 12.28, "0.103")  # Table S1, heterosexuals only (N=750)
TABLES1_UNKNOWN = {  # Table S1, heterosexuals only (N=750), unknown status vs HIV-negative
    "Unknown status: depression":       (3.92, 2.66, 5.78, "<0.001"),
    "Unknown status: high anxiety":     (4.16, 2.63, 6.56, "<0.001"),
    "Unknown status: suicidal att./id.": (2.06, 1.28, 3.30, "0.003"),
}

# Outcome counts (yes, no) by HIV status
OUTCOMES = ["depression", "high anxiety", "suicidal att./id."]
T2_COUNTS = {  # Table 2, sexual minorities (N=497)
    "negative": {"n": 246, "depression": (174, 72), "high anxiety": (79, 167), "suicidal att./id.": (77, 169)},
    "positive": {"n": 29,  "depression": (24, 5),   "high anxiety": (14, 15),  "suicidal att./id.": (12, 17)},
    "unknown":  {"n": 222, "depression": (201, 21), "high anxiety": (130, 92), "suicidal att./id.": (109, 113)},
}
S1_COUNTS = {  # Table S1, heterosexuals (N=750)
    "negative": {"n": 549, "depression": (156, 393), "high anxiety": (47, 502), "suicidal att./id.": (72, 477)},
    "positive": {"n": 12,  "depression": (5, 7),     "high anxiety": (3, 9),    "suicidal att./id.": (0, 12)},
    "unknown":  {"n": 189, "depression": (131, 58),  "high anxiety": (65, 124), "suicidal att./id.": (39, 150)},
}
T1_COUNTS = {  # Table 1, whole cohort (N=1247), rows exactly as printed
    "negative": {"n": 795, "depression": (332, 79),  "high anxiety": (195, 216), "suicidal att./id.": (149, 646)},
    "positive": {"n": 41,  "depression": (29, 12),   "high anxiety": (17, 24),   "suicidal att./id.": (12, 29)},
    "unknown":  {"n": 411, "depression": (330, 465), "high anxiety": (126, 669), "suicidal att./id.": (148, 263)},
}
T1_TOTAL_YES = {"depression": 691, "high anxiety": 338, "suicidal att./id.": 309}  # Results, paragraph 1


# ---------------------------------------------------------------------------
# 2. Calculations
# ---------------------------------------------------------------------------
def wald_from_ci(aor, lo, hi):
    """SE of log(AOR), Wald z and two-sided p implied by a symmetric 95% CI on the log scale."""
    se = (np.log(hi) - np.log(lo)) / (2 * Z975)
    z = np.log(aor) / se
    return se, z, 2 * norm.sf(abs(z)), float(np.exp((np.log(lo) + np.log(hi)) / 2))


def crude_or(a, b, c, d):
    """Crude OR (exposed yes/no = a/b, reference yes/no = c/d), Woolf 95% CI and Wald p."""
    orr = (a * d) / (b * c)
    se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    return orr, np.exp(np.log(orr) - Z975 * se), np.exp(np.log(orr) + Z975 * se), 2 * norm.sf(abs(np.log(orr) / se))


def fmt_p(p):
    return "<0.0001" if p < 1e-4 else f"{p:.4f}"


WALD = {}
print("=" * 100)
print("A. Wald check of every estimate in the HIV-status rows: p implied by the printed 95% CI")
print("=" * 100)
print(f"{'source':10s} {'estimate':36s} {'AOR (95% CI)':>20s} {'CI mid':>7s} {'SE':>6s} {'z':>6s} "
      f"{'p from CI':>10s} {'printed p':>9s} {'CI excl. 1':>10s}")
for src, table in [("Abstract", ABSTRACT_SM), ("Table 1", TABLE1), ("Table 2", TABLE2),
                   ("Table S1", {"HIV-positive: high anxiety": TABLES1_HIVPOS_ANX, **TABLES1_UNKNOWN})]:
    for k, (a, lo, hi, rep) in table.items():
        se, z, p, mid = wald_from_ci(a, lo, hi)
        WALD[(src, k)] = (se, z, p)
        print(f"{src:10s} {k:36s} {a:5.2f} ({lo:4.2f} to {hi:5.2f}) {mid:7.2f} {se:6.3f} {z:6.2f} "
              f"{fmt_p(p):>10s} {rep:>9s} {str(lo > 1 or hi < 1):>10s}")

print()
print("=" * 100)
print("B. Which model do the abstract's four 'sexual minority' figures come from?")
print("=" * 100)
for k, v in ABSTRACT_SM.items():
    in_t1 = TABLE1[k] == v
    in_t2 = TABLE2[k] == v
    t2 = TABLE2[k]
    print(f"{k:36s} abstract {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | identical to Table 1: {in_t1} | "
          f"identical to Table 2: {in_t2} | Table 2 value {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]}")
n_match_t1 = sum(TABLE1[k] == v for k, v in ABSTRACT_SM.items())
print(f"-> {n_match_t1} of {len(ABSTRACT_SM)} abstract estimates match Table 1 exactly (AOR, both CI limits, p); "
      f"{sum(TABLE2[k] == v for k, v in ABSTRACT_SM.items())} match Table 2.")
for k in ["Unknown status: depression", "Unknown status: high anxiety"]:
    print(f"   Table 2 AOR is {100*(1-TABLE2[k][0]/ABSTRACT_SM[k][0]):.1f}% smaller than the abstract's, {k}")
print(f"   heterosexual share of the whole cohort: 750/1247 = {100*750/1247:.1f}%")
diffs = [abs(WALD[(src, k)][2] - float(v[3])) for src, t in [("Table 1", TABLE1), ("Table 2", TABLE2)]
         for k, v in t.items() if not v[3].startswith("<")]
print(f"   largest |Wald p - printed p| over Tables 1 and 2: {max(diffs):.4f}")
sig_abs = sum(v[1] > 1 for v in ABSTRACT_SM.values())
sig_t2 = sum(TABLE2[k][1] > 1 for k in ABSTRACT_SM)
print(f"-> CI excludes 1: {sig_abs} of 4 in the abstract; {sig_t2} of 4 for the same estimates in Table 2.")
for k in ABSTRACT_SM:
    print(f"   ratio abstract/Table 2 AOR, {k:36s}: {ABSTRACT_SM[k][0] / TABLE2[k][0]:.2f}")

print()
print("=" * 100)
print("B2. Results paragraph 5 (exploratory analyses): unknown status, heterosexuals (Table S1) vs sexual minorities (Table 2)")
print("=" * 100)
for k, v in TABLES1_UNKNOWN.items():
    t2 = TABLE2[k]
    print(f"{k:36s} Table S1 {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | Table 2 {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]:7s} "
          f"| Table 2 CI excludes 1: {t2[1] > 1} | larger AOR: {'Table 2 (sexual minorities)' if t2[0] > v[0] else 'Table S1 (heterosexuals)'}")
n_s1_larger = sum(TABLES1_UNKNOWN[k][0] > TABLE2[k][0] for k in TABLES1_UNKNOWN)
print(f"-> Table S1 AOR larger than Table 2 AOR for {n_s1_larger} of 3 unknown-status outcomes.")

print()
print("=" * 100)
print("C. Crude ORs from the Table 2 counts (sexual minorities), vs HIV-negative")
print("=" * 100)
CRUDE_T2 = {}
for grp in ["positive", "unknown"]:
    for o in OUTCOMES:
        a, b = T2_COUNTS[grp][o]
        c, d = T2_COUNTS["negative"][o]
        orr, lo, hi, p = crude_or(a, b, c, d)
        key = ("HIV-positive: " if grp == "positive" else "Unknown status: ") + o
        CRUDE_T2[key] = (orr, lo, hi, p)
        print(f"{key:36s} {a}/{a+b} vs {c}/{c+d}: crude OR {orr:.2f} (95% CI {lo:.2f} to {hi:.2f}), p={'<0.001' if p < 0.001 else format(p, '.3f')} "
              f"| Table 2 AOR {TABLE2[key][0]}")

print()
print("=" * 100)
print("D. Arithmetic of the HIV-status rows")
print("=" * 100)
for name, cnt, N in [("Table 2", T2_COUNTS, 497), ("Table S1", S1_COUNTS, 750), ("Table 1", T1_COUNTS, 1247)]:
    print(f"{name}: HIV rows sum to {sum(v['n'] for v in cnt.values())} (stated N={N})")
for g in ["negative", "positive", "unknown"]:
    s = T2_COUNTS[g]["n"] + S1_COUNTS[g]["n"]
    print(f"  {g:8s}: Table 2 n + Table S1 n = {T2_COUNTS[g]['n']} + {S1_COUNTS[g]['n']} = {s}; "
          f"Table 1 n = {T1_COUNTS[g]['n']}  -> equal: {s == T1_COUNTS[g]['n']}")
print("Row check, Table 1 (yes + no should equal the row total):")
for g in ["negative", "positive", "unknown"]:
    for o in OUTCOMES:
        y, n = T1_COUNTS[g][o]
        exp = (T2_COUNTS[g][o][0] + S1_COUNTS[g][o][0], T2_COUNTS[g][o][1] + S1_COUNTS[g][o][1])
        print(f"  {g:8s} {o:18s}: printed {y}/{n} (sum {y+n}, row total {T1_COUNTS[g]['n']}, "
              f"{'ok' if y + n == T1_COUNTS[g]['n'] else 'MISMATCH'}); Table 2 + Table S1 give {exp[0]}/{exp[1]}")
for o in OUTCOMES:
    s = sum(T1_COUNTS[g][o][0] for g in T1_COUNTS)
    print(f"  Table 1 'yes' column sum, {o:18s}: {s} (Results text: {T1_TOTAL_YES[o]})")
print("  Table 1 printed % for 'Negative' depression: 332/411 = "
      f"{100*332/411:.1f}% (printed 80.8); 332/795 = {100*332/795:.1f}%")

# ---------------------------------------------------------------------------
# 3. Figures
# ---------------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
from matplotlib.transforms import Bbox

BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, PANEL = "#eb6834", "#2a78d6", "#e4e3df", "#f3f2ef"
plt.rcParams.update({
    "font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "path",  # text as outlines: no dependence on installed fonts
    "axes.linewidth": 0.8,
})
LW = 2.0          # 2 px lines
MS = 9            # marker size (pt), >= 8 px
W = 7.2           # figure width (in)
SHORT = {  # row labels for the four abstract claims
    "HIV-positive: high anxiety": "HIV-positive vs negative\nhigh general anxiety",
    "Unknown status: depression": "Unknown vs negative\ndepressive symptoms",
    "Unknown status: high anxiety": "Unknown vs negative\nhigh general anxiety",
    "Unknown status: suicidal att./id.": "Unknown vs negative\nsuicidal attempt/ideation",
}
KEYS = list(ABSTRACT_SM)


def save(fig, stem):
    """Save at exactly W inches wide: crop height to the content, never widen past the figure."""
    os.makedirs(CAND, exist_ok=True)
    fig.canvas.draw()
    bb = fig.get_tightbbox(fig.canvas.get_renderer())
    if bb.x0 < -0.01 or bb.x1 > W + 0.01:
        print(f"WARNING {stem}: content spans x={bb.x0:.2f}..{bb.x1:.2f} in, beyond the {W} in figure")
    box = Bbox.from_extents(0, bb.y0 - 0.12, W, bb.y1 + 0.12)
    for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
        fig.savefig(os.path.join(CAND, f"{stem}.{ext}"), bbox_inches=box, **kw)
    plt.close(fig)
    print(f"wrote {stem}.svg / {stem}.png")


def ci_str(v):
    return f"{v[0]:.2f} ({v[1]:.2f} to {v[2]:.2f}), p={v[3]}"


def p_print(v):  # printed p in "p=0.013" / "p<0.001" form
    return f"p{v[3]}" if v[3].startswith("<") else f"p={v[3]}"


# --- Candidate 1: text-panel diagram (abstract vs Table 1 vs Table 2) --------------------------
def cand1():
    fig = plt.figure(figsize=(W, 5.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 74); ax.axis("off")

    def panel(x, y, w, h, title, sub):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
                                    fc=PANEL, ec="#c9c7c1", lw=1))
        ax.text(x + 1.8, y + h - 2.6, title, fontsize=11.5, weight="bold", va="top")
        ax.text(x + 1.8, y + h - 6.6, sub, fontsize=10, color=INK2, va="top")

    # Abstract panel
    panel(0.5, 6, 41, 60, "Abstract, Results", "sentence on sexual minority adolescents")
    quote = ("“Sexual minority adolescents living with HIV had significantly higher odds of "
             "high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown "
             "HIV status had significantly higher odds of … suicidal attempt/ideation "
             "(AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).”")
    ax.text(2.8, 52, textwrap.fill(quote, 41), fontsize=10, va="top", style="italic", linespacing=1.35)
    a1, a4 = ABSTRACT_SM[KEYS[0]], ABSTRACT_SM[KEYS[3]]
    ax.text(2.8, 21, "HIV-positive, high anxiety", fontsize=10, color=INK2)
    ax.text(2.8, 17.2, ci_str(a1), fontsize=10.5, color=ORANGE, weight="bold")
    ax.text(2.8, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
    ax.text(2.8, 8.2, ci_str(a4), fontsize=10.5, color=ORANGE, weight="bold")

    # Table 1 panel
    t1a, t1d = TABLE1[KEYS[0]], TABLE1[KEYS[3]]
    panel(58.5, 38, 41, 28, "Table 1: whole cohort", "N=1247, sexual minority and heterosexual")
    ax.text(60.3, 52, "HIV-positive, high anxiety", fontsize=10, color=INK2)
    ax.text(60.3, 48.2, ci_str(t1a), fontsize=10.5, color=INK, weight="bold")
    ax.text(60.3, 44, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
    ax.text(60.3, 40.2, ci_str(t1d), fontsize=10.5, color=INK, weight="bold")

    # Table 2 panel
    t2a, t2d = TABLE2[KEYS[0]], TABLE2[KEYS[3]]
    panel(58.5, 6, 41, 28, "Table 2: sexual minorities only", "N=497, the group the abstract names")
    ax.text(60.3, 20, "HIV-positive, high anxiety", fontsize=10, color=INK2)
    ax.text(60.3, 16.2, ci_str(t2a), fontsize=10.5, color=BLUE, weight="bold")
    ax.text(60.3, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
    ax.text(60.3, 8.2, ci_str(t2d), fontsize=10.5, color=BLUE, weight="bold")

    # connectors
    ax.add_patch(FancyArrowPatch((42, 44), (58.2, 50), arrowstyle="-|>", mutation_scale=14,
                                 lw=LW, color=INK2, connectionstyle="arc3,rad=-0.15"))
    ax.text(50.2, 53.5, "identical\nnumbers\n(all 4\nestimates)", ha="center", fontsize=10, color=INK, linespacing=1.15)
    ax.add_patch(FancyArrowPatch((42, 28), (58.2, 22), arrowstyle="-|>", mutation_scale=14,
                                 lw=LW, color=INK2, ls=(0, (4, 3)), connectionstyle="arc3,rad=0.15"))
    ax.text(50.2, 10, "population\nnamed in\nthe abstract", ha="center", fontsize=10, color=INK, linespacing=1.15)
    ax.text(99.5, 2.2, "Table 2: both CIs include 1, not significant at the paper's 5% level",
            ha="right", fontsize=10, color=INK2)
    # legend
    h = [Line2D([], [], marker="s", ls="", ms=MS, color=ORANGE, label="value printed in the abstract"),
         Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="paper's own sexual-minority model (Table 2)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
    save(fig, "cand1_text_panels")


# --- Candidate 2: dot-and-interval plot, log OR axis -------------------------------------------
def cand2():
    fig, ax = plt.subplots(figsize=(W, 4.3))
    fig.subplots_adjust(left=0.235, right=0.725, bottom=0.14, top=0.80)
    tr = ax.get_yaxis_transform()  # x in axes fraction, y in data
    off = 0.17
    for i, k in enumerate(KEYS):
        y = len(KEYS) - 1 - i
        for dy, v, col, mk in [(off, ABSTRACT_SM[k], ORANGE, "o"), (-off, TABLE2[k], BLUE, "s")]:
            ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="butt")
            ax.plot(v[0], y + dy, marker=mk, ms=MS, color=col, mec=BG, mew=1.5, ls="")
            ax.text(1.03, y + dy, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {p_print(v)}",
                    va="center", fontsize=10, color=INK, transform=tr)
    ax.axvline(1, color=INK, lw=1.0, zorder=2)
    ax.text(1.03, len(KEYS) - 0.45, "no association (OR = 1)", fontsize=10, color=INK, va="bottom")
    ax.set_xscale("log"); ax.set_xlim(0.5, 8)
    ticks = [0.5, 1, 2, 4, 8]
    ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
    ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
    ax.set_ylim(-0.6, len(KEYS) - 0.2)
    ax.set_xlabel("Adjusted odds ratio (log scale) with 95% CI")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.text(1.03, len(KEYS) - 0.45, "AOR (95% CI), printed p", fontsize=10, color=INK2, va="bottom", transform=tr)
    h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS,
                label="Abstract, given for sexual minorities (= Table 1, whole cohort, N=1247)"),
         Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS,
                label="Table 2, sexual-minority model (N=497)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
              ncol=1, frameon=False, fontsize=10)
    save(fig, "cand2_forest")


# --- Candidate 3: Wald z recomputed from each printed CI, with the 5% threshold ---------------
def cand3():
    fig, ax = plt.subplots(figsize=(W, 3.9))
    fig.subplots_adjust(left=0.235, right=0.97, bottom=0.17, top=0.86)
    for i, k in enumerate(KEYS):
        y = len(KEYS) - 1 - i
        za = WALD[("Abstract", k)][1]; zt = WALD[("Table 2", k)][1]
        ax.plot([zt, za], [y, y], color="#b9b7b0", lw=LW, zorder=1)
        ax.plot(za, y, "o", ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.plot(zt, y, "s", ms=MS + 1, color=BLUE, mec=BG, mew=1.5, zorder=3)
        ax.text(za + 0.25, y + 0.16, f"z={za:.2f}", fontsize=10, color=INK, va="bottom")
        ax.text(zt - 0.25, y + 0.16, f"z={zt:.2f}", fontsize=10, color=INK, va="bottom", ha="right")
    ax.axvline(Z975, color=INK, lw=1.0, zorder=2)
    ax.text(Z975 + 0.08, -0.62, "z = 1.96 (two-sided p = 0.05)", fontsize=10, color=INK, va="bottom")
    ax.set_xlim(0, 9.5); ax.set_ylim(-0.7, len(KEYS) - 0.3)
    ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
    ax.set_xlabel("Wald z = ln(AOR) / SE, SE recomputed from the printed 95% CI")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="from the abstract's CI (= Table 1)"),
         Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="from Table 2's CI (sexual minorities)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
              ncol=2, frameon=False, fontsize=10)
    save(fig, "cand3_wald_z")


# --- Candidate 4: comparison grid (abstract | Table 1 | Table 2) ------------------------------
def cand4():
    fig = plt.figure(figsize=(W, 4.3))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-2, 58); ax.axis("off")
    cols = [(0.5, "Estimate"), (26, "Abstract\n(for sexual minorities)"),
            (51.5, "Table 1\n(whole cohort,\nN=1247)"), (76, "Table 2\n(sexual minorities,\nN=497)")]
    for x, t in cols:
        ax.text(x, 53, t, fontsize=10, weight="bold", va="top")
    ax.plot([0.5, 99.5], [45, 45], color=INK2, lw=0.8)
    for i, k in enumerate(KEYS):
        y = 40 - i * 10
        if i % 2 == 0:
            ax.add_patch(FancyBboxPatch((0.5, y - 4.6), 99, 9.2, boxstyle="square,pad=0", fc=PANEL, ec="none"))
        ax.text(0.5, y, SHORT[k], fontsize=10, va="center", linespacing=1.2)
        a, t1, t2 = ABSTRACT_SM[k], TABLE1[k], TABLE2[k]
        for x, v, col in [(26, a, ORANGE), (51.5, t1, INK), (76, t2, BLUE)]:
            ax.text(x, y + 1.6, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f})", fontsize=10.5, color=col,
                    weight="bold", va="center")
            sig = v[1] > 1 or v[2] < 1
            ax.text(x, y - 2.0, f"{p_print(v)} · {'CI excludes 1' if sig else 'CI includes 1'}",
                    fontsize=10, color=INK2, va="center")
        ax.text(48.9, y + 1.6, "=", fontsize=12, color=INK, ha="center", va="center", weight="bold")
    ax.text(0.5, 2.0, "'=' marks an exact match of AOR, both CI limits and p.\nOrange: value printed in the abstract; "
            "blue: paper's own sexual-minority model.", fontsize=10, color=INK2, va="center", linespacing=1.3)
    save(fig, "cand4_grid")


# --- Candidate 5: one estimate, all the paper's versions of it --------------------------------
def cand5():
    k = KEYS[0]
    rows = [
        ("Abstract: “sexual minority\nadolescents”", ABSTRACT_SM[k], ORANGE, "o", True),
        ("Table 1: whole cohort\n(N=1247)", TABLE1[k], INK2, "D", False),
        ("Table 2: sexual minorities\n(N=497)", TABLE2[k], BLUE, "s", True),
        ("Crude OR, Table 2 counts\n(14/29 vs 79/246)", CRUDE_T2[k], BLUE, "s", False),
        ("Table S1: heterosexuals\nonly (N=750)", TABLES1_HIVPOS_ANX, INK2, "D", False),
    ]
    fig, ax = plt.subplots(figsize=(W, 4.4))
    fig.subplots_adjust(left=0.29, right=0.725, bottom=0.14, top=0.74)
    tr = ax.get_yaxis_transform()
    for i, (lab, v, col, mk, filled) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="butt")
        ax.plot(v[0], y, marker=mk, ms=MS, color=col if filled else BG, mec=col, mew=2, ls="")
        p = v[3] if isinstance(v[3], str) else f"{v[3]:.3f}"
        ptxt = f"p{p}" if p.startswith("<") else f"p={p}"
        ax.text(1.03, y, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {ptxt}", va="center", fontsize=10, transform=tr)
    ax.axvline(1, color=INK, lw=1.0, zorder=2)
    ax.text(1.03, len(rows) - 0.55, "OR = 1", fontsize=10, color=INK, va="bottom")
    ax.set_xscale("log"); ax.set_xlim(0.5, 13)
    ticks = [0.5, 1, 2, 4, 8]
    ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
    ax.set_ylim(-0.6, len(rows) - 0.4)
    ax.set_xlabel("OR, high general anxiety, HIV-positive vs negative\n(log scale, 95% CI)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="printed in the abstract"),
         Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="sexual-minority data (filled: Table 2 AOR; open: crude OR recomputed)"),
         Line2D([], [], marker="D", color=INK2, lw=LW, ms=MS, mfc=BG, label="other populations in the paper")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
              ncol=1, frameon=False, fontsize=10)
    save(fig, "cand5_one_estimate")


cand1(); cand2(); cand3(); cand4(); cand5()
for src, dst in [("cand1_text_panels", "fig1"), ("cand2_forest", "fig2")]:
    for ext in ("svg", "png"):
        shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("fig1 = cand1_text_panels, fig2 = cand2_forest (provisional, pending the choice of design)")

Abstract names G3244A where the body names G3242A as the second termination-abolishing pathogenic mutation

Cell (2010) · doi:10.1016/j.cell.2010.05.018 · PMC2887341

Summary

The Summary (abstract) of this crystal-structure paper on the human mitochondrial transcription terminator MTERF1 ends its results with a disease link: "the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination." The body of the paper names a different second mutation, G3242A, in each of the three places where it states this result: the Introduction, the Results section on pathogenic mutations, and the Conclusions. The body also describes G3244A in terms that cannot fit the abstract sentence: its base pair makes "no specific interaction" with MTERF1 in the crystal structure, the mutation does not affect DNA binding, and its effect on termination is "moderate". A reader of the abstract alone is therefore told that the wrong variant loses a recognition contact and abolishes termination, and is never told about G3242A, the variant the paper's own data support.

What the paper reports

Abstract (Summary), final results sentence:

Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination.

Introduction, last paragraph:

We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., 2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a consequence eliminate its transcriptional termination activity.

Results and discussion, "Pathogenic mutations in the leu-tRNA", first paragraph (structure and binding):

All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the R387A mutation.

Same section, second paragraph (termination assays, Figure 6C/D):

As expected, only residual termination is observed in both orientations with the G3249A mutation. In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not substantially altering binding, results nevertheless in a strong decrease in termination.

Unexpectedly, one additional mutation appears to moderately affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible effect on termination. Interestingly, no specific interaction is observed in the crystal structure with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base pair and displays a high base pair buckle (see Experimental Procedures).

Results and discussion, "Conclusions":

An important conclusion derived from the crystal structure is that two pathogenic mutations that interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of MTERF1 at the leu-tRNA site.

Further experiments will be needed to address whether the G3249A and G3242A mutations result in transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.

What the paper's own numbers show

Method. No arithmetic is involved. The error concerns which mutation is named, so I checked the paper's internal consistency. I parsed the PMC-archived full text and counted every mention of the five leu-tRNA variants the paper discusses (G3242A, A3243G, A3243T, G3244A, G3249A) in each section. I then extracted the pair of mutations that each section names as disrupting sequence recognition and eliminating or strongly impairing termination. Finally, I tabulated what the body says about each candidate's structural contact, its effect on binding and its effect on termination. recompute.py.txt holds the excerpts verbatim, checks all 33 excerpt sentences against the archived text (all found), and prints every value below.

Quantity Location Reported Recomputed from the paper's own text
Second mutation said to disrupt recognition and eliminate termination Abstract (Summary) G3244A (paired with G3249A) Contradicted by all three body sections below
The same pair Introduction, last paragraph G3249A and G3242A G3242A
Mutations expected to conflict with sequence recognition Results, Pathogenic mutations, first paragraph "except for G3242A and G3249A" G3242A
The same pair Conclusions (pair named twice; G3242A mentioned 4 times) G3249A and G3242A G3242A
Agreement of the three body sections with each other Introduction, Results, Conclusions — All three name G3249A + G3242A
Agreement of the abstract with the body Abstract vs body — No. The pairs differ in one mutation, G3244A vs G3242A, a single-character difference
Mentions of G3242A / G3244A / G3249A Abstract 0 / 1 / 1 G3242A, the body's second mutation, never appears in the abstract
Mentions of G3242A / G3244A / G3249A Introduction 1 / 0 / 1 —
Mentions of G3242A / G3244A / G3249A Results, Pathogenic mutations 3 / 2 / 3 G3244A appears only here, as "one additional mutation"
Mentions of G3242A / G3244A / G3249A Conclusions 4 / 0 / 4 —
Mentions in the whole article All sections G3242A 8, G3244A 3, G3249A 9 —
Structural contact of the mutated base pair Results, Pathogenic mutations G3242 contacts Arg251; G3249 is double hydrogen-bonded to Arg387; G3244 base pair: "no specific interaction is observed" Only G3242A and G3249A can remove a recognition contact
Effect on DNA binding Results, Pathogenic mutations G3242A "weak or no effect"; G3244A "not affecting DNA binding"; G3249A "completely eliminate specific DNA binding" —
Effect on termination Results, Pathogenic mutations; Conclusions G3249A "only residual termination"; G3242A "strong decrease" / "very strongly reduces"; G3244A "moderate but reproducible effect" G3244A's effect is moderate, not elimination

By the paper's own account, G3244A meets neither condition in the abstract sentence. It removes no recognition contact, and it does not eliminate termination. G3242A meets the first condition and, in the body's words, comes close to meeting the second ("very strongly reduces").

Figure

The abstract's final results sentence names G3249A and G3244A. The body names G3242A as the second mutation and describes G3244A as making no contact and having a moderate effect.

The top panel holds the abstract's only sentence about the pathogenic mutations, with the second mutation it names (G3244A) in orange. The two lower panels hold verbatim excerpts from the body of the same article. The left panel collects what the Introduction, Results and Conclusions say about G3242A, the mutation they name alongside G3249A: it contacts Arg251, and it strongly decreases termination while leaving binding largely intact. G3242A does not appear in the abstract (dashed connector). The right panel collects everything the body says about G3244A. It appears only in the Results, as "one additional mutation" whose base pair makes no specific contact with MTERF1, which does not affect binding and which has a moderate effect on termination. The solid connector marks the conflict between the abstract's claim about G3244A and the body's description of it. All text is quoted from the PMC-archived article, and the section counts come from recompute.py.txt.

Why it matters

Explanations considered

Status

Suggested correction

  1. In the Summary (abstract), replace "G3249A and G3244A" with "G3249A and G3242A", so that the sentence reads: "Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3242A mutations interfere with key interactions for sequence recognition, eliminating termination."
  2. Consider aligning the strength of the wording with the Results and Conclusions. G3249A abolishes specific binding and leaves only residual termination, while G3242A barely affects binding but strongly reduces termination. "Severely impairing termination" would match both. The same applies to "eliminate" for G3242A in the Introduction.
  3. Optionally, mention in the abstract that G3244A, which makes no contact with MTERF1, has a moderate effect on termination, because the body presents this as an unexpected finding.
  4. Because the abstract propagates through PubMed and indexing services, a published correction notice linked to the article record would be more effective than a silent change to the online text.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC2887341
(Cell 2010; doi:10.1016/j.cell.2010.05.018), and redraw fig1 in this file's folder.

Run:  python recompute.py.txt      (needs matplotlib)

There is no arithmetic to redo: the error is a mismatch between the Summary
(abstract) and the body text in which pathogenic mtDNA mutation is named as the
second one that disrupts sequence recognition and eliminates termination. This
script therefore
  1. counts every mention of the leu-tRNA mutations (G3242A, A3243G, A3243T,
     G3244A, G3249A) in each section,
  2. extracts the pair of mutations each section names as eliminating or
     strongly impairing termination,
  3. tabulates what the body says about G3242A and G3244A (structural contact,
     effect on binding, effect on termination), and
  4. draws fig1.
The paragraphs used are copied verbatim below from the article as archived in
PubMed Central (author manuscript). If the repository tool src/show.py is
reachable, the script also re-reads the archived full text, checks that every
excerpt occurs in it verbatim, and repeats the counts on the whole article.
"""
import os
import re
import subprocess
import collections
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs
SECTIONS = collections.OrderedDict()
SECTIONS["Abstract (Summary)"] = (
    "Defects in mitochondrial gene expression are associated with aging and disease. Mterf proteins "
    "have been implicated in modulating transcription, replication and protein synthesis. We have solved "
    "the structure of a member of this family, the human mitochondrial transcriptional terminator MTERF1, "
    "bound to dsDNA containing the termination sequence. The structure indicates that upon sequence "
    "recognition MTERF1 unwinds the DNA molecule, promoting eversion of three nucleotides. Base flipping "
    "is critical for stable binding and transcriptional termination. Additional structural and biochemical "
    "results provide insight into the DNA binding mechanism and explain how MTERF1 recognizes its target "
    "sequence. Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations "
    "interfere with key interactions for sequence recognition, eliminating termination. Our results provide "
    "insight into the role of mterf proteins and suggest a link between mitochondrial disease and the "
    "regulation of mitochondrial transcription.")
SECTIONS["Introduction"] = (
    "Interestingly, the pathogenic A3243G MELAS mutation was shown to decrease the ability of MTERF1 to "
    "terminate transcription in vitro (Hess et al., 1991) and yet no obvious transcriptional alterations "
    "are observed in cells carrying this MELAS mutation (Chomyn et al. 1992). "
    "We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., "
    "2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), "
    "interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a "
    "consequence eliminate its transcriptional termination activity.")
SECTIONS["Results: Pathogenic mutations (binding)"] = (
    "It is important to note that the pathogenic effects of these mutations might be simply due to an "
    "alteration on the leu-tRNA structure, and this appears indeed to be the case for the A3243G mutation "
    "(Sasarman et al., 2008). A3243G is frequently identified in MELAS patients and has been previously "
    "shown to slightly reduce MTERF1 binding (Hess et al., 1991) and proposed to interfere in vitro with "
    "transcriptional termination (Hess et al., 1991; Chomyn et al., 1992). "
    "Our measurements confirm that A3243G results in slightly decreased binding (Figure 6A and 3SE). "
    "A similar binding affinity was observed for the A3243T mutation (Figure 6A). "
    "All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, "
    "except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double "
    "hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine "
    "interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could "
    "not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A "
    "mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to "
    "completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the "
    "R387A mutation.")
SECTIONS["Results: Pathogenic mutations (termination)"] = (
    "As expected, only residual termination is observed in both orientations with the G3249A mutation. "
    "In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not "
    "substantially altering binding, results nevertheless in a strong decrease in termination. The A3243G "
    "mutation moderately reduces termination, while termination on A3243T is only marginally weaker than "
    "for the wt sequence. The reduction in termination observed for A3243G is consistent with what was "
    "previously reported (Hess et al., 1991). Unexpectedly, one additional mutation appears to moderately "
    "affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible "
    "effect on termination. Interestingly, no specific interaction is observed in the crystal structure "
    "with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base "
    "pair and displays a high base pair buckle (see Experimental Procedures). It is possible that the "
    "strength of the G:C base pair in the wt sequence is important to maintain this conformation and that "
    "the G3244A mutation thus leads to a slight destabilization of the bound MTERF1 that is sufficient to "
    "make it more readily displaced by the RNA polymerase.")
SECTIONS["Conclusions"] = (
    "An important conclusion derived from the crystal structure is that two pathogenic mutations that "
    "interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a "
    "strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially "
    "affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while "
    "G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs "
    "termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A "
    "and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of "
    "MTERF1 at the leu-tRNA site. The G3249A mutation leads to the development of a variant of "
    "Kearns-Sayre syndrome (Seneca et al., 2001), a mitochondrial myopathy, while the G3242A mutation has "
    "been associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009). "
    "Further experiments will be needed to address whether the G3249A and G3242A mutations result in "
    "transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.")
FIG6_LEGEND = ("The DNA residues involved in arginine guanine interactions with MTERF1 are indicated "
               "by an asterisk.")

MUT = re.compile(r"\b[GA]32(?:42|43|44|49)[AGT]\b")
ORDER = ["G3242A", "A3243G", "A3243T", "G3244A", "G3249A"]


def counts(text):
    return collections.Counter(MUT.findall(text))


def sentences(text):
    return [s.strip() for s in re.split(r"(?<=\.)\s+(?=[A-Z])", text) if s.strip()]


# ------------------------------------------------- 1. mentions per section
print("=== 1. Mentions of each leu-tRNA mutation, by section (verbatim excerpts) ===")
print(f"  {'section':45s} " + " ".join(f"{m:>7s}" for m in ORDER))
EXC = {}
for sec, txt in SECTIONS.items():
    c = counts(txt)
    EXC[sec] = c
    print(f"  {sec:45s} " + " ".join(f"{c.get(m, 0):7d}" for m in ORDER))

# ------------------------------------- 2. which pair each section names
print("\n=== 2. The pair each section names as disrupting recognition / eliminating termination ===")
PAIR_PATTERNS = [
    ("Abstract (Summary)", r"pathogenic (G\d{4}A) and (G\d{4}A) mutations interfere"),
    ("Introduction", r"two mutations, (G\d{4}A) that causes.*?and (G\d{4}A) that is associated"),
    ("Results: Pathogenic mutations (binding)", r"except for (G\d{4}A) and (G\d{4}A)\."),
    ("Conclusions", r"pathogenic effects of the (G\d{4}A) and (G\d{4}A) MtDNA mutations"),
]
PAIRS = {}
for sec, pat in PAIR_PATTERNS:
    m = re.search(pat, SECTIONS[sec])
    PAIRS[sec] = frozenset(m.groups())
    print(f"  {sec:45s} {' + '.join(sorted(PAIRS[sec], reverse=True))}")
body = {p for s, p in PAIRS.items() if s != "Abstract (Summary)"}
print(f"  body sections agree with each other: {len(body) == 1}")
print(f"  abstract pair equals body pair:      {PAIRS['Abstract (Summary)'] in body}")
diff = PAIRS["Abstract (Summary)"] ^ next(iter(body))
print(f"  mutations that differ: {sorted(diff)}  (one digit apart: "
      f"{sum(a != b for a, b in zip(*sorted(diff)))} character differs)")
print(f"  G3242A mentions in the abstract: {EXC['Abstract (Summary)'].get('G3242A', 0)}; "
      f"G3244A mentions in Introduction + Conclusions: "
      f"{EXC['Introduction'].get('G3244A', 0) + EXC['Conclusions'].get('G3244A', 0)}")

# ------------------------------------- 3. what the body says about each
print("\n=== 3. What the body says about G3242A, G3244A and G3249A ===")
ALL = " ".join(v for k, v in SECTIONS.items() if k != "Abstract (Summary)")
KEYS = {
    "G3242A": ["G3242 interacts", "G3242A resulted", "the G3242A mutation, while", "G3242A does not"],
    "G3244A": ["G3244A, while", "no specific interaction", "the G3244A mutation thus"],
    "G3249A": ["G3249 forms", "the G3249A mutation appears", "residual termination", "G3249A appears"],
}
for mut, keys in KEYS.items():
    print(f"  {mut}:")
    for s in sentences(ALL):
        if any(k in s for k in keys):
            print("    -", textwrap.shorten(s, 230, placeholder=" ..."))
SUMMARY = {  # attribute -> (G3242A, G3244A, G3249A), each a verbatim phrase from the body
    "contact in the crystal structure": ("G3242 interacts in the structure with Arg251",
                                         "no specific interaction is observed",
                                         "G3249 forms a double hydrogen bond with Arg387"),
    "effect on DNA binding": ("weak or no effect on binding", "not affecting DNA binding",
                              "completely eliminate specific DNA binding"),
    "effect on termination": ("a strong decrease in termination",
                              "a moderate but reproducible effect on termination",
                              "only residual termination"),
}
print("\n  attribute                         | G3242A | G3244A | G3249A")
for a, vals in SUMMARY.items():
    for v in vals:
        assert v in ALL, v
    print(f"  {a:33s} | " + " | ".join(vals))

# ------------------------- 4. optional cross-check against the archived full text
print("\n=== 4. Cross-check against the archived full text (src/show.py) ===")
FULLCOUNTS = None
show = os.path.join(HERE, "..", "..", "..", "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if os.path.exists(show) and os.path.exists(py):
    out = subprocess.run([py, show, "PMC2887341", "--full"], capture_output=True, text=True,
                         cwd=os.path.abspath(os.path.join(HERE, "..", "..", ".."))).stdout
    flat = re.sub(r"\s+", " ", out)
    excerpts = [s for t in SECTIONS.values() for s in sentences(t)] + [FIG6_LEGEND]
    missing = [s for s in excerpts if s not in flat]
    print(f"  excerpt sentences found verbatim: {len(excerpts) - len(missing)} of {len(excerpts)}")
    for s in missing:
        print("    NOT FOUND:", s[:120])
    sec = "header"
    FULLCOUNTS = collections.defaultdict(collections.Counter)
    for line in out.splitlines():
        if line.startswith("ABSTRACT:"):
            sec = "Abstract"
        elif line.startswith("## "):
            sec = line[3:].strip()
        elif line.startswith("BACK MATTER"):
            sec = "Back matter (figure legends)"
        FULLCOUNTS[sec].update(MUT.findall(line))
    print("  mentions by section in the whole article:")
    for s, c in FULLCOUNTS.items():
        if c:
            print(f"    {s[:60]:60s} " + ", ".join(f"{m} x{c[m]}" for m in ORDER if c[m]))
    tot = sum(FULLCOUNTS.values(), collections.Counter())
    print("  whole article: " + ", ".join(f"{m} x{tot[m]}" for m in ORDER))
else:
    print("  src/show.py not reachable; skipped (the verbatim excerpts above were used).")

# ================================================================ figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#c9c7c1"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "path",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)
    print(f"  wrote {stem}.svg and {stem}.png")


print("\n=== Figure ===")
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
R = fig.canvas.get_renderer()


def panel(x, y, w, h, title, accent):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
                                fc="white", ec=EDGE, lw=1.2))
    ax.add_patch(FancyBboxPatch((x, y + h - 0.9), w, 0.9, boxstyle="square,pad=0",
                                fc=accent, ec="none"))
    ax.text(x + 1.8, y + h - 2.2, title, fontsize=11, fontweight="bold", va="top")


def runs(x, y, parts, size=10.5):
    """Draw text segments on one line; return the extent of each segment in data units.
    A part is (text, kwargs) or (text, kwargs, gap), gap being an x shift before it."""
    ext = []
    for part in parts:
        txt, kw = part[0], part[1]
        x += part[2] if len(part) > 2 else 0.0
        t = ax.text(x, y, txt, fontsize=size, va="top", **kw)
        bb = t.get_window_extent(R).transformed(ax.transData.inverted())
        ext.append(bb)
        x = bb.x1
    return ext


def lines(x, y, rows, step=3.3, size=10.5):
    out = []
    for i, r in enumerate(rows):
        out.append(runs(x, y - i * step, r if isinstance(r, list) else [(r, {})], size))
    return out


HO = dict(color=ORANGE, fontweight="bold")
HB = dict(color=BLUE, fontweight="bold")
SEC = dict(color=INK2)

# --- Abstract panel (top)
panel(2, 76, 96, 22, "Abstract (Summary), final results sentence", ORANGE)
lines(3.8, 92.2, [
    "\u201cFinally, we have demonstrated that the mitochondrial pathogenic",
    [("G3249A and ", {}), ("G3244A", HO, -0.55), (" mutations interfere with key interactions for", {})],
    "sequence recognition, eliminating termination.\u201d",
])
miss = runs(3.8, 81.6, [("G3242A: not mentioned in the abstract", SEC)], size=10)[0]
pair = runs(55.0, 81.6, [("Pair named: G3249A + ", SEC), ("G3244A", HO, -0.2)], size=10)
g3244_abs = pair[1]

# --- Body panel, G3242A (bottom left)
panel(2, 8, 47, 48, "Body on G3242A", BLUE)
lines(3.8, 50.5, [
    [("Introduction", dict(fontweight="bold"))],
    "“two mutations, G3249A … and",
    [("G3242A", HB), (" … eliminate its", {})],
    "transcriptional termination activity”",
    [("Results", dict(fontweight="bold"))],
    "“G3242 interacts in the structure",
    "with Arg251”",
    "“the G3242A mutation, while not",
    "substantially altering binding, results",
    "nevertheless in a strong decrease in",
    "termination”",
    [("Conclusions", dict(fontweight="bold"))],
    "“the G3249A and G3242A MtDNA",
    "mutations”; G3242A named 4 times",
], step=2.95, size=10)

# --- Body panel, G3244A (bottom right)
panel(51, 8, 47, 48, "Body on G3244A", BLUE)
lines(52.8, 50.5, [
    [("Results (its only section)", dict(fontweight="bold"))],
    "“Unexpectedly, one additional",
    "mutation appears to moderately affect",
    [("termination: ", {}), ("G3244A", HB), (", while not", {})],
    "affecting DNA binding, results in a",
    "moderate but reproducible effect on",
    "termination.”",
    "“no specific interaction is observed",
    "in the crystal structure with any of",
    "the nucleotides in this base pair”",
    [("Introduction, Conclusions", dict(fontweight="bold"))],
    "not mentioned",
], step=2.95, size=10)

# --- connectors
cx = (g3244_abs.x0 + g3244_abs.x1) / 2
ax.add_patch(FancyArrowPatch((cx, 75.6), (cx, 56.4), arrowstyle="-|>",
                             mutation_scale=14, lw=2, color=INK))
ax.text(cx - 2.0, 66.0, "Conflict: the abstract says a recognition\n"
        "contact is lost and termination is\neliminated; the body reports no contact\n"
        "and a moderate effect", fontsize=10, ha="right", va="center", color=INK)
mx = 12.0
ax.add_patch(FancyArrowPatch((mx, 75.6), (mx, 56.4), arrowstyle="-|>", mutation_scale=14, lw=2,
                             color=INK2, linestyle=(0, (3, 2))))
ax.text(mx + 1.5, 66.0, "the body's second\nmutation, G3242A", fontsize=10,
        ha="left", va="center", color=INK2)

ax.text(2.0, 4.6, "Orange: as printed in the abstract.  Blue: the body of the same article "
        "(Introduction,", fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 1.8, "Results, Conclusions). All text quoted verbatim from the PMC-archived article.",
        fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")

Abstract says metformin upregulates intestinal G6PC; the paper's own Results report a decrease

Cell Rep (2026) · doi:10.1016/j.celrep.2026.117791 · PMC13560472

Summary

The Abstract, the "In brief" summary and one Results subsection heading state that high metformin concentrations "upregulate" G6PC expression in intestinal epithelial cells (IECs), in the Abstract's words "to prevent glucose release". Every data statement in the paper reports the opposite: G6PC protein is reduced in IECs of metformin-treated HFD-fed mice (Figures 5E/5G) and in metformin-treated or AMPK-overexpressing primary IECs (Figures 5M, 6A–6C), G6pc mRNA is suppressed (Figure 6D), and the Figure 6 title, the Results conclusion and the Discussion all say G6PC goes down; of 13 statements giving a direction for this effect, the 3 summary-level ones say "up" and the 10 in the body say "down". The paper's own metabolite data (glucose 6-phosphate up 5-fold, glucose down) and its G6PC overexpression experiment fit a decrease, not an increase. The data and conclusions stand, but the most-read parts of the article give the first step of its intestinal mechanism the wrong sign.

What the paper reports

Statements that give an increase.

Abstract (Summary):

Here, we showed that high metformin concentrations upregulate G6PC expression through AMP-activated protein kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs).

In brief (the opening clause, which names the authors, is omitted here):

in intestinal epithelial cells (IECs), high metformin concentrations upregulate G6PC expression to prevent glucose release and inhibit mitochondrial activity, leading to increased glucose utilization and lactate overproduction

Results, subsection heading:

Upregulation of G6PC expression by metformin increases glycolysis in IECs

Statements that give a decrease.

Results, subsection "Inhibition of mitochondrial respiration leads to increased glucose uptake and glycolysis in IECs":

However, it is important to note that we found that metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G).

Results, the paragraph directly under the "Upregulation of G6PC expression..." heading:

Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G), and treatment with high metformin concentration (1 mM) activated AMPK or overexpression of functional AMPKα1β1γ1 complex significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C). Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D).

The same paragraph, concluding:

Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose 6-phosphate and augmentation of glycolysis in IECs.

Figure 6 legend, title:

Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells

Discussion, paragraph 3:

we found that metformin treatment significantly decreased G6PC protein levels in the IECs of HFD-fed mice and in primary IECs; this will decrease glucose release from IECs because G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from the cells with the bidirectional function of glucose transporters.

These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs.

Discussion, summary paragraph:

High metformin concentrations in the intestine downregulate G6PC expression to stimulate glycolysis inside IECs, and with inhibited mitochondrial complex 1 activity, leading to increased glucose utilization through glycolysis and lactate release.

Supporting experiments described in the Results.

Compared to IECs with expression of adenoviral GFP controls, overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G). Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H).

found that metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose 6-phosphate/glucose (Figure 6K).

All quotes above were checked verbatim against the PMC-archived article text.

What the paper's own numbers show

Method. This is a direction-of-effect contradiction, so there are no test statistics to recompute. I made a complete inventory instead. I searched the full archived text (abstract, In brief, headings, body and figure legends) for every mention of G6PC or G6pc. I kept each statement that gives a direction for metformin's effect on G6PC in IECs and recorded where it sits and which way it points. I first did this with a regex scan, then read each hit by hand; the hand count adds the two mRNA statements ("suppressed", "inhibition of the expression") that the regex missed and drops one regex false positive. I then checked whether the paper's own reported metabolite changes (Figures 6I–6K) and its G6PC overexpression results (Figures 6E–6H) fit an increase or a decrease. G6PC catalyses glucose 6-phosphate + H₂O → glucose + Pᵢ, the last step before glucose can leave the cell. All of this is in recompute.py.txt, which also re-checks every quote against the archived text.

Quantity Location Reported Recomputed
Direction, metformin → G6PC in IECs Abstract "upregulate G6PC expression ... to prevent glucose release" Up; contradicts every data statement
Direction, metformin → G6PC in IECs In brief "upregulate G6PC expression to prevent glucose release" Up; contradicts every data statement
Direction, metformin → G6PC in IECs Results subsection heading "Upregulation of G6PC expression by metformin increases glycolysis" Up; the paragraph beneath it reports a decrease
G6PC protein, IECs of HFD-fed mice Results; Figs 5E, 5G "significantly reduced"; "decreased" Down
G6PC protein, primary IECs, 1 mM metformin or AMPKα1β1γ1 overexpression Results; Figs 5M, 6A–6C "significantly decreased" Down
G6pc mRNA, primary IECs Results; Fig 6D "suppressed" Down
Mechanistic conclusion Results; Fig 6 title; Discussion (×3) "decreased G6PC expression", "Reduction of G6PC expression", "a decrease in G6PC protein", "downregulate G6PC expression" Down
Tally of directional statements Whole archived text n/a 13 statements: 3 up (all in Abstract, In brief, one heading), 10 down (all in Results text, Fig 6 legend, Discussion)
Statements citing a data figure Results n/a 4 of 4 point down; none points up
G6P/glucose ratio Results; Figs 6I–6K G6P 5-fold up, glucose decreased, ratio "drastically increased" Ratio fold = 5 / (glucose fold) > 5 for any fall in glucose: substrate up, product down, which fits lower G6PC activity
G6PC overexpression with metformin Results; Figs 6F–6H 2-NDBG accumulation down > 6-fold; lactate down 75% Down to < 16.7% and 25% of control; restoring G6PC blunts metformin's effect, as a metformin-induced decrease predicts

No experiment in the paper reports metformin raising G6PC in IECs. The one increase in G6pc the paper reports is in the liver of intestinal AMPKα1/2 knockout mice (Figure 4F), a different tissue and a genetic manipulation, not a metformin effect in IECs.

Figure

The three summary statements give an increase in G6PC; the ten body statements give a decrease

Figure 1. Every statement in the archived article that gives a direction for metformin's effect on G6PC in IECs, grouped by where it appears. Orange, left: the three statements that say "upregulate"/"Upregulation" (Abstract, In brief, one Results subsection heading). Blue, right: the statements that say G6PC is reduced, decreased, suppressed or downregulated (five are shown; five more in the Results and Discussion are listed in the table above and in recompute.py.txt). All of the data-backed statements (Figures 5E/5G, 5M, 6A–6D) are on the right. The connector marks the conflict: the two groups give opposite directions for the same effect, in the same cells, at the same high metformin concentrations.

What an increase and a decrease in G6PC would each predict, compared with what the paper reports

Figure 2. Why the data fit a decrease. The top row shows the reaction G6PC catalyses (glucose 6-phosphate to free glucose, which can then leave the cell through bidirectional glucose transporters) and the branch to glycolysis and lactate. The orange box lists what an increase in G6PC, as the Abstract says, would predict. The blue box lists what a decrease would predict. The grey box lists what the Results text reports: glucose 6-phosphate up 5-fold with glucose down (Figures 6I–6K), G6PC overexpression cutting metformin-stimulated 2-NDBG accumulation more than 6-fold and lactate by 75% (Figures 6F–6H), and G6PC protein and mRNA down. Each reported finding matches the decrease. The effect sizes are as stated in the Results text; nothing is read from the paper's images.

Why it matters

The change in G6PC is the first step of the intestinal half of the paper's model: high metformin concentrations in IECs lower G6PC through AMPK, glucose 6-phosphate builds up and is routed into glycolysis, and the extra lactate goes through the portal vein to the liver. The Abstract and the In brief are the parts of the article that are most widely read, indexed and quoted, and they give this step the wrong sign. A reader who sees only them learns that metformin raises intestinal G6PC, the opposite of what the paper measured. Such a reader might cite the paper as evidence for G6PC induction, or design experiments on that premise. The Abstract sentence is also internally inconsistent: G6PC frees glucose for export, so raising it would promote glucose release, not "prevent" it. A careful reader may notice that, but a reader skimming the abstract or an automated summary may not. The same wrong word in a Results subsection heading means even a reader who opens the Results may first meet "Upregulation" before the text beneath it reports a decrease.

The underlying data, figures and conclusions are unaffected. Every data statement, the Figure 6 title and the Discussion consistently report a decrease, and the overall claim that metformin's actions in the intestine and liver together improve hyperglycemia does not depend on this wording. The problem is confined to the three summary-level statements.

Explanations considered

A different model, tissue, dose or time point. The Abstract specifies "high metformin concentrations" in IECs. That matches the conditions under which the body reports a decrease: 1–2 mM metformin in primary IECs (Figures 5M, 6A–6D) and oral metformin in IECs of HFD-fed mice (Figures 5E/5G). The body reports no IEC condition in which G6PC rises. This does not account for the contradiction.

The liver G6pc result. The Results report "significantly higher mRNA levels of rate-limited gluconeogenic genes G6pc and Pck1 in the liver" of intestinal AMPKα1/2 knockout mice (Figure 4F). That is a knockout effect in the liver, compared with floxed controls, not metformin acting on IECs. It cannot be the source of a statement about high metformin concentrations in IECs. Does not account for it.

G6PC overexpression. Figures 6E–6H use adenoviral overexpression of G6PC, an increase made by the investigators, not by metformin. In those cells metformin's effects on 2-NDBG accumulation and lactate were reduced, which supports the decrease model. Does not account for it.

"Upregulate" in some other sense (e.g. upregulated activity, or regulation in general). The Abstract says "upregulate G6PC expression", and the body reports expression (protein and mRNA) going down. No passage reports increased G6PC activity; the G6P/glucose data (Figures 6I–6K) point to lower activity. Does not account for it.

Text-conversion or manuscript-stage artefact. The PMC record is the NIH author manuscript, not the typeset version. However, the publisher-supplied PubMed record (PMID 42555357) carries the same "upregulate G6PC expression" Abstract sentence verbatim, so the wording is in the publisher's own abstract metadata as well as the manuscript. A single-word conversion error would not change the same verb in three separate places while leaving ten others intact. Does not account for it.

A wording slip rather than a data error. This is the most plausible reading of the evidence: the data and the detailed text agree with each other, and only three summary statements disagree. It does not reconcile the two directions, but it does mean the correction needed is to wording, not to results.

Status

As of 2026-10-23:

Sources: PMC-archived article text (via src/show.py PMC13560472); PubMed efetch for PMID 42555357; Crossref REST API for doi:10.1016/j.celrep.2026.117791; Europe PMC metadata; src/known.py.

Suggested correction

Change "upregulate" to "downregulate" in the Abstract and the In brief, and "Upregulation" to "Downregulation" in the Results subsection heading, so that they match the Results, Figure 6 and the Discussion:

The same changes would need to be made to the abstract held by PubMed and other indexers. No data, figure or statistical result needs to change.

Recomputation script (python recompute.py.txt)
"""Inventory of every statement in PMC13560472 (Cell Rep 2026; doi:10.1016/j.celrep.2026.117791)
that gives the direction of metformin's effect on G6PC in intestinal epithelial cells (IECs),
plus the arithmetic implied by the paper's own Figure 6I-6K description.
Redraws fig1/fig2 (SVG + PNG at 200 dpi) in this file's folder.

Run:  python recompute.py.txt          (needs matplotlib)
If run from the repository root (/home/ubuntu/papers) with src/show.py available, it also
re-checks every quoted passage verbatim against the PMC-archived article text.
"""
import os
import subprocess
import sys
import textwrap
from collections import Counter

HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13560472"

# ---------------------------------------------------------------- inputs (verbatim quotes)
# (id, location, part of paper, direction stated for metformin -> G6PC, verbatim excerpt)
STATEMENTS = [
    ("A1", "Abstract (Summary)", "summary", "UP",
     "high metformin concentrations upregulate G6PC expression through AMP-activated protein "
     "kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs)"),
    ("A2", "In brief", "summary", "UP",
     "high metformin concentrations upregulate G6PC expression to prevent glucose release"),
    ("A3", "Results, subsection heading", "summary", "UP",
     "Upregulation of G6PC expression by metformin increases glycolysis in IECs"),
    ("B1", "Results, 'Inhibition of mitochondrial respiration...' (Figs 5E, 5G)", "body", "DOWN",
     "metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice "
     "(Figures 5E and 5G)"),
    ("B2", "Results, under the 'Upregulation of G6PC...' heading (Figs 5E, 5G)", "body", "DOWN",
     "Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G)"),
    ("B3", "Results, same paragraph (Figs 5M, 6A-6C)", "body", "DOWN",
     "significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C)"),
    ("B4", "Results, same paragraph (Fig 6D)", "body", "DOWN",
     "Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D)"),
    ("B5", "Results, same paragraph (conclusion)", "body", "DOWN",
     "Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose "
     "6-phosphate and augmentation of glycolysis in IECs."),
    ("B6", "Results, same paragraph", "body", "DOWN",
     "the inhibition of the expression of G6pc and Pck1 genes by metformin would block the use of "
     "lactate/pyruvate as substrates for gluconeogenesis"),
    ("B7", "Figure 6 legend, title", "body", "DOWN",
     "Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells"),
    ("B8", "Discussion, paragraph 3", "body", "DOWN",
     "we found that metformin treatment significantly decreased G6PC protein levels in the IECs of "
     "HFD-fed mice and in primary IECs; this will decrease glucose release from IECs"),
    ("B9", "Discussion, paragraph 3", "body", "DOWN",
     "These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs."),
    ("B10", "Discussion, summary paragraph", "body", "DOWN",
     "High metformin concentrations in the intestine downregulate G6PC expression to stimulate "
     "glycolysis inside IECs"),
]

# Other passages used in the write-up (verbatim), checked the same way
OTHER_QUOTES = [
    "overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation "
    "of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G)",
    "Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H)",
    "metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased "
    "glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose "
    "6-phosphate/glucose (Figure 6K)",
    "G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from "
    "the cells with the bidirectional function of glucose transporters",
    "The above data demonstrate that high metformin concentrations can inhibit glucose production and "
    "release in the IECs.",
    "Our results show that high concentrations of metformin in the intestine can prevent glucose release "
    "and stimulate glycolysis",
    "The mRNA levels of G6pc, Pck1, and Gapdh in the liver of HFD-fed male mice",
]

# Figure 6I-6K, as described in the Results text
G6P_FOLD = 5.0          # "a 5-fold increase in glucose 6-phosphate levels"
GLUCOSE_DIRECTION = "decreased"   # "with decreased glucose levels in IECs"
G6PC_OE_2NDBG_FOLD = 6.0          # "more than 6-fold reduction" of 2-NDBG accumulation
G6PC_OE_LACTATE_RED = 0.75        # "a 75% reduction of lactate levels"


# ---------------------------------------------------------------- 0. verbatim check (optional)
def norm(s):
    return " ".join(s.replace("’", "'").split())


def archived_text():
    repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
    show = os.path.join(repo, "src", "show.py")
    if not os.path.exists(show):
        return None
    r = subprocess.run([sys.executable, show, PMCID, "--full"], capture_output=True, text=True, cwd=repo)
    return r.stdout if r.returncode == 0 else None


txt = archived_text()
print("=== 0. Verbatim check against the PMC-archived text ===")
if txt is None:
    print("src/show.py not available; skipping (quotes were checked when the write-up was made)")
else:
    t = norm(txt)
    allq = [s[4] for s in STATEMENTS] + OTHER_QUOTES
    ok = [norm(q) in t for q in allq]
    for q, k in zip(allq, ok):
        if not k:
            print("  NOT FOUND:", q[:90])
    print(f"{sum(ok)}/{len(ok)} quoted passages found verbatim")

# ---------------------------------------------------------------- 1. tally
print("\n=== 1. Direction stated for metformin's effect on G6PC in IECs ===")
for sid, loc, part, d, q in STATEMENTS:
    print(f"  {sid:4s} {d:4s} [{part:7s}] {loc}")
tally = Counter((s[2], s[3]) for s in STATEMENTS)
n_up = sum(1 for s in STATEMENTS if s[3] == "UP")
n_down = sum(1 for s in STATEMENTS if s[3] == "DOWN")
print(f"Total directional statements: {len(STATEMENTS)}  (UP {n_up}, DOWN {n_down})")
print(f"UP statements located in Abstract / In brief / heading: {tally[('summary', 'UP')]} of {n_up}")
print(f"DOWN statements located in Results text / Fig 6 legend / Discussion: {tally[('body', 'DOWN')]} of {n_down}")
data_backed = [s for s in STATEMENTS if "Figure" in s[4]]
print(f"Quoted statements that cite a data figure panel: {len(data_backed)}; directions: "
      f"{sorted(set(s[3] for s in data_backed))}")

# ---------------------------------------------------------------- 2. biochemistry consistency
print("\n=== 2. Consistency of Figs 6I-6K with the direction of G6PC ===")
print("G6PC reaction: glucose 6-phosphate + H2O -> glucose + Pi (last step before glucose exit)")
# ratio fold = (G6P fold) / (glucose fold); any glucose fold < 1 gives a ratio fold > G6P fold
print(f"  ratio fold = {G6P_FOLD:.0f} / (glucose fold); glucose fold < 1 -> ratio fold > {G6P_FOLD:.0f}: "
      f"{all(G6P_FOLD / g > G6P_FOLD for g in (0.999, 0.5, 0.1))}")
print(f"With glucose {GLUCOSE_DIRECTION}, the G6P/glucose ratio rises by more than {G6P_FOLD:.0f}-fold: "
      "substrate builds up while product falls, the pattern expected when G6PC activity is LOWER.")
print(f"G6PC overexpression: 2-NDBG accumulation reduced >{G6PC_OE_2NDBG_FOLD:.0f}-fold "
      f"(to <{100 / G6PC_OE_2NDBG_FOLD:.1f}% of GFP control); lactate reduced {G6PC_OE_LACTATE_RED:.0%} "
      f"(to {100 * (1 - G6PC_OE_LACTATE_RED):.0f}% of control).")
print("Restoring G6PC blunting metformin's effect is what a G6PC decrease predicts; an increase predicts the opposite.")

# ---------------------------------------------------------------- 3. figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
                     "figure.facecolor": BG, "savefig.facecolor": BG})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
    plt.close(fig)
    print(f"wrote {name}.svg and {name}.png")


def box(ax, x, y, w, h, color, lw=2, ls="-", fc=BG):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                linewidth=lw, edgecolor=color, facecolor=fc, linestyle=ls))


def entries(ax, x, y_top, width_chars, items, color, gap=0.016, line_h=0.0265):
    y = y_top
    for loc, quote in items:
        ax.text(x, y, loc, color=color, fontsize=10, fontweight="bold", va="top")
        y -= line_h
        for ln in textwrap.wrap(quote, width_chars):
            ax.text(x, y, ln, color=INK, fontsize=10, va="top")
            y -= line_h
        y -= gap
    return y


# ----- Figure 1: summary statements vs body statements
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")

ax.text(0.03, 0.975, "Direction of metformin's effect on G6PC in intestinal epithelial cells",
        fontsize=12, fontweight="bold", color=INK, va="top")
ax.text(0.03, 0.942, "Every statement in the archived article that gives a direction, grouped by where it appears",
        fontsize=10, color=INK2, va="top")

L = (0.03, 0.14, 0.405, 0.765)    # x, y, w, h
R = (0.555, 0.14, 0.415, 0.765)
box(ax, *L, ORANGE); box(ax, *R, BLUE)
ax.text(L[0] + 0.02, L[1] + L[3] - 0.02, f"INCREASE: {n_up} statements", color=ORANGE,
        fontsize=11, fontweight="bold", va="top")
ax.text(L[0] + 0.02, L[1] + L[3] - 0.05, "Abstract, In brief, one Results heading", color=INK2,
        fontsize=10, va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.02, f"DECREASE: {n_down} statements", color=BLUE,
        fontsize=11, fontweight="bold", va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.05, "Results text, Figure 6 title, Discussion", color=INK2,
        fontsize=10, va="top")

left_items = [
    ("Abstract", "“…high metformin concentrations upregulate G6PC expression through "
     "… AMPK activation to prevent glucose release in … IECs”"),
    ("In brief", "“…high metformin concentrations upregulate G6PC expression to "
     "prevent glucose release…”"),
    ("Results, subsection heading", "“Upregulation of G6PC expression by metformin "
     "increases glycolysis in IECs”"),
]
right_items = [
    ("Results (Figs 5E, 5G)", "“metformin treatment significantly reduced G6PC protein "
     "levels in IECs of HFD-fed mice”"),
    ("Results (Figs 5M, 6A–6C)", "“significantly decreased G6PC protein levels in "
     "primary IECs”"),
    ("Results (Fig 6D)", "“Metformin treatment suppressed the mRNA levels of G6pc”"),
    ("Figure 6 title", "“Reduction of G6PC expression by metformin increases lactate "
     "production”"),
    ("Discussion, summary", "“High metformin concentrations in the intestine "
     "downregulate G6PC expression”"),
]
yl = entries(ax, L[0] + 0.02, L[1] + L[3] - 0.095, 37, left_items, ORANGE, gap=0.03)
for i, ln in enumerate(textwrap.wrap("The heading sits directly above a paragraph that reports a decrease "
                                     "(Figs 5E/5G, 5M, 6A\u20136D).", 40)):
    ax.text(L[0] + 0.02, yl - 0.01 - 0.0265 * i, ln, color=INK2, fontsize=10, va="top")
yend = entries(ax, R[0] + 0.02, R[1] + R[3] - 0.095, 36, right_items, BLUE, gap=0.014)
ax.text(R[0] + 0.02, yend - 0.002, f"+ {n_down - len(right_items)} more in Results and Discussion",
        color=INK2, fontsize=10, va="top")

arr = FancyArrowPatch((L[0] + L[2] + 0.005, 0.60), (R[0] - 0.005, 0.60), arrowstyle="<->",
                      mutation_scale=14, linewidth=2, color=INK)
ax.add_patch(arr)
ax.text(0.495, 0.615, "opposite", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(0.495, 0.585, "direction", ha="center", va="top", fontsize=10, fontweight="bold", color=INK)

ax.text(0.03, 0.115, "Every data statement (Figs 5E/5G, 5M, 6A–6D) reports a decrease. No experiment in the "
        "paper", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.087, "reports metformin increasing G6PC.", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.045, "Orange: summary wording (increase).  Blue: wording that matches the paper's data (decrease).",
        fontsize=10, color=INK2, va="top")
save(fig, "fig1")

# ----- Figure 2: what each direction predicts vs what Figs 6I-6K and 6E-6H report
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.975, "What each direction predicts, and what the paper measured",
        fontsize=12, fontweight="bold", color=INK, va="top")

# reaction strip
y0 = 0.80
for x, lab in ((0.05, "Glucose\n6-phosphate"), (0.40, "Glucose\n(in IEC)"), (0.73, "Glucose\nreleased")):
    box(ax, x, y0 - 0.045, 0.21, 0.10, INK2, lw=1.5)
    ax.text(x + 0.105, y0 + 0.005, lab, ha="center", va="center", fontsize=10, color=INK)
for (xa, xb, lab) in ((0.265, 0.395, "G6PC"), (0.615, 0.725, "GLUT")):
    ax.add_patch(FancyArrowPatch((xa, y0), (xb, y0), arrowstyle="-|>", mutation_scale=14,
                                 linewidth=2, color=INK))
    ax.text((xa + xb) / 2, y0 + 0.015, lab, ha="center", va="bottom", fontsize=10,
            fontweight="bold", color=INK)
ax.add_patch(FancyArrowPatch((0.155, y0 - 0.05), (0.155, y0 - 0.115), arrowstyle="-|>",
                             mutation_scale=14, linewidth=2, color=INK))
ax.text(0.17, y0 - 0.085, "glycolysis → lactate", fontsize=10, color=INK, va="center")

rows = [
    ("If metformin INCREASED G6PC (Abstract)", ORANGE, [
        "more G6P made into glucose; G6P/glucose falls",
        "more glucose free to leave the cell, not less",
        "extra G6PC should mimic metformin, not oppose it",
    ]),
    ("If metformin DECREASED G6PC (Results)", BLUE, [
        "G6P builds up, feeds glycolysis; G6P/glucose rises",
        "less glucose released (\u201cprevent glucose release\u201d)",
        "extra G6PC should blunt metformin\u2019s effect",
    ]),
]
measured = [
    f"G6P \u00d7{G6P_FOLD:.0f}, glucose down (Figs 6I, 6J)",
    f"G6P/glucose up > {G6P_FOLD:.0f}-fold (Fig 6K)",
    f"Extra G6PC: 2-NDBG down > {G6PC_OE_2NDBG_FOLD:.0f}-fold, lactate down "
    f"{G6PC_OE_LACTATE_RED:.0%} (Figs 6F\u20136H)",
    "G6PC protein down (Figs 5E/5G, 5M, 6A\u20136C)",
    "G6pc mRNA down (Fig 6D)",
]
ax.text(0.03, 0.60, "Prediction", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.625, 0.60, "Reported in the paper", fontsize=10, color=INK2, fontweight="bold")
yy = 0.555
for title, col, lines in rows:
    box(ax, 0.03, yy - 0.215, 0.565, 0.235, col)
    ax.text(0.045, yy, title, color=col, fontsize=10, fontweight="bold", va="top")
    for i, ln in enumerate(lines):
        ax.text(0.045, yy - 0.045 - 0.05 * i, "\u2022 " + ln, color=INK, fontsize=10, va="top")
    yy -= 0.27
box(ax, 0.625, 0.065, 0.345, 0.51, INK2, lw=1.5)
ym = 0.545
for item in measured:
    for ln in textwrap.wrap(item, 26):
        ax.text(0.64, ym, ln, color=INK, fontsize=10, va="top")
        ym -= 0.033
    ym -= 0.014
ax.text(0.64, ym - 0.01, "Matches: DECREASE", color=BLUE, fontsize=10, fontweight="bold", va="top")
ax.add_patch(FancyArrowPatch((0.623, 0.17), (0.598, 0.17), arrowstyle="-|>", mutation_scale=16,
                             linewidth=2, color=BLUE))
ax.text(0.03, 0.035, "Effect sizes are as stated in the Results text; nothing is read from the paper's images.",
        fontsize=10, color=INK2, va="top")
save(fig, "fig2")

Oral cancer test's accuracy table counts 101 subjects in a cohort the paper gives as 92

Cell Rep Med (2024) · doi:10.1016/j.xcrm.2024.101447 · PMC10983043

Summary

The paper reports that the beta-defensin index (BDI) detects oral squamous cell carcinoma (OSCC) with 98.2% sensitivity (95% CI 90.3–99.9) and 82.6% specificity (95% CI 68.6–92.2) in "a multi-center validation study". These figures come from Table 1, whose four cells (TP 54, TN 38, FP 8, FN 1) add up to 101, while the text twice says the analysis covers 92 subjects: 54 cancers and 38 benign lesions in the discovery and validation cohorts. The printed TP and TN equal the cancer and benign totals, so the 8 false positives and 1 false negative are counted twice, and the denominators become 55 and 46 instead of 54 and 38. With the paper's own counts, which the subject-level labels in supplementary Table S1 confirm, specificity is 30/38 = 78.9% (exact 95% CI 62.7–90.4), not 82.6%. Sensitivity barely changes (53/54 = 98.1%). Because the abstract presents these as validation results, it also matters that they pool the discovery cohort in which the 1.25 cutoff was chosen. The validation cohorts alone give 96.6% sensitivity and 78.3% specificity.

What the paper reports

Abstract (Summary):

A multi-center validation study shows sensitivity and specificity values of 98.2% (95% confidence interval [CI] 90.3–99.9) and 82.6% (95% CI 68.6–92.2), respectively.

Highlights:

A multi-center validation study yields sensitivity and specificity of 98% and 83%

Results, "Beta-defensin index: Proof-of-concept, discovery, and validation studies" (study design paragraph):

We recruited a total of 92 subjects for the discovery and validation study

Results, discovery-phase study:

Of the 40 oral lesions, 25 were diagnosed by pathology review as being cancerous while 15 were benign (see Table S1B diagnoses).

We arrived at a BDI cutoff value of 1.25 and found 100% sensitivity with an acceptable specificity value of 80% (Figures 4B and 4C).

Figure 4 legend, panel D:

(D) BDI values for 23 non-cancerous and 29 OSCC lesions from the three validation cohorts (blue: internal validation cohort [n = 21]; green: external validation cohort 1 [n = 19]; orange: external validation cohort 2 [n = 12]).

Results, "Sample size and power calculation":

In the validation cohort (total 52 subjects; 29 OSCC: mean = 6.1, SD = 6.9; 23 benign: mean = 0.8, SD = 0.64)

Results, validation study:

We then calculated the sensitivity and specificity of the BDI, based on the threshold value of 1.25 from the discovery cohort, and found overall sensitivity of 98.2% and specficity of 82.6%. For all the subjects (n = 92), sensitivity/specificity and positive/negative predictive values, with a confidence interval of 95%, are shown in Table 1. Among 92 subjects, we recorded eight false positives (FPs) and one FN (Tables 1 and S1).

Table 1, "Sensitivity, specificity, and positive and negative predictive values of BDI in OSCC detection" (columns: values (BDI vs. biopsy) | 95% confidence interval):

True positive, TP | 54 | –

True negative, TN | 38 | –

False positive, FP | 8 | –

False negative, FN | 1 | –

Sensitivity, TP/(TP + FN) | 98.2 | 90.28%–99.95%

Specificity, TN/(TN + FP) | 82.6 | 68.58%–92.18%

Positive predictive value | 87.10 | 78.22%–92.69%

Negative predictive value | 97.44 | 84.44%–99.63%

Results, "Benchmarking of BDI against other OSCC biomarkers/platforms":

Figure S5 shows that the BDI is the most sensitive (98%) in detecting OSCC

Discussion:

In the discovery and validation studies, we reported eight FP cases.

Additionally, one of the benefits of using the BDI would be the reduction of unnecessary biopsies conducted by dentists in the United States from >90% to 17.4% when considering the FP rate of 82.6%.

what we believe happened in our one and only FN result obtained through the validation study at CMC (where the BDI score was reported as 0.23 [Figure 4D and Table S1])

What the paper's own numbers show

Method. I added up the cancer and benign counts that the paper gives for the discovery cohort (Results) and the three validation cohorts (Figure 4D legend; power-calculation paragraph), and checked them against the site sizes in the STAR Methods (40, 21, 19 and 12 subjects). I then summed the four Table 1 cells and derived the disease-positive (TP + FN) and disease-negative (TN + FP) totals they imply. I recomputed every Table 1 estimate from its printed cells to identify the computation behind it: exact Clopper–Pearson intervals for sensitivity and specificity, and the logit interval of Mercaldo et al. (2007), as used by MedCalc, for the predictive values. Finally I rebuilt the 2×2 table from the stated cohort (54 cancers, 38 benign) and the stated 8 FP and 1 FN, and checked it against the TP/TN/FP/FN label for each subject in supplementary Table S1 (mmc2.xlsx), cohort by cohort. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Subjects, discovery + validation Results; STAR Methods (40 + 21 + 19 + 12) 92 92
Cancers Results (discovery 25); Fig 4D legend (validation 29) 25 + 29 54
Benign lesions Results (discovery 15); Fig 4D legend (validation 23) 15 + 23 38
Sum of Table 1 cells Table 1 TP 54, TN 38, FP 8, FN 1 (text: n = 92) 54 + 38 + 8 + 1 = 101; the excess of 9 equals FP + FN
Cancers implied by Table 1 (TP + FN) Table 1 — 55 (cohort has 54)
Benign lesions implied by Table 1 (TN + FP) Table 1 — 46 (cohort has 38)
Sensitivity Table 1; Abstract; Results 98.2 (90.28–99.95) 54/55 = 98.18% (90.28–99.95): reproduces the printed value from the printed cells. Stated cohort: 53/54 = 98.15% (90.11–99.95)
Specificity Table 1; Abstract; Results 82.6 (68.58–92.18) 38/46 = 82.61% (68.58–92.18): reproduces the printed value. Stated cohort: 30/38 = 78.95% (62.68–90.45)
Positive predictive value Table 1 87.10 (78.22–92.69) 54/62 = 87.10% (78.22–92.69): reproduces the printed value. Stated cohort: 53/61 = 86.89% (78.14–92.47)
Negative predictive value Table 1 97.44 (84.44–99.63) 38/39 = 97.44% (84.44–99.63): reproduces the printed value. Stated cohort: 30/31 = 96.77% (81.04–99.53)
Discovery cohort counts Results (100% sensitivity, 80% specificity in 25 + 15); Table S1B 100%, 80% TP 25, FN 0, TN 12, FP 3 (text and Table S1B agree)
Validation cohorts, pooled Table S1C–S1E (21 + 12 + 19 subjects) — TP 28, FN 1, TN 18, FP 5. Sensitivity 28/29 = 96.55% (82.24–99.91); specificity 18/23 = 78.26% (56.30–92.54)
Discovery + validation, from Table S1 labels Table S1B–S1E — TP 53, TN 30, FP 8, FN 1, N = 92: identical to the table rebuilt from the stated counts
Specificity by cohort Table S1B–S1E — discovery 12/15; internal 4/6; external (12 subjects) 10/12; external (19 subjects) 4/5
Location of the single FN Discussion; Table S1E "validation study at CMC", BDI 0.23 the only FN label in Table S1 is in the 19-subject external cohort, BDI 0.23
Share of benign lesions classed positive Discussion ("17.4%") 17.4% 17.4% = 100 − 82.6 = 8/46; stated cohort: 8/38 = 21.1%
Highlights Highlights 98% and 83% stated cohort: 98% and 79%

All Table 1 estimates and confidence intervals follow exactly from the printed cells, so the table is arithmetically consistent with itself. What is wrong is the cells: they describe 55 cancers and 46 benign lesions, whereas the study had 54 and 38. The 8 FP and 1 FN given in the text (and listed one by one in Table S1) match the stated cohort; only TP and TN are off, by exactly FN (1) and FP (8). All 92 Table S1 labels agree with the BDI value and the 1.25 cutoff.

Figure

Table 1 as printed compared with the 2×2 table rebuilt from the paper's stated cohort

Figure 1. Left (orange): the 2×2 table as printed in Table 1, with its row and column sums. Right (blue): the same table rebuilt from the paper's stated cohort of 54 cancers and 38 benign lesions and its stated 8 false positives and 1 false negative; the subject-level labels in Table S1 give the same four counts. The shaded TP and TN cells are the only ones that differ. The printed table's columns sum to 55 and 46 (N = 101), one and eight more than the cohort contains, and its specificity is therefore 38/46 = 82.6% instead of 30/38 = 78.9%. Intervals are exact Clopper–Pearson 95% CIs.

Reported accuracy estimates compared with those recomputed from the stated cohort

Figure 2. The four accuracy estimates in Table 1 with their 95% CIs: as reported (orange circles, from the printed cells) and as recomputed from the stated 92-subject cohort (blue squares). Sensitivity, PPV and NPV move by at most 0.7 points. Specificity falls by 3.7 points to 78.9%, and the lower limit of its interval falls from 68.6% to 62.7%. Sensitivity and specificity use Clopper–Pearson intervals, PPV and NPV the logit interval, the same methods that reproduce the printed CIs exactly. The horizontal axis starts at 50%.

Why it matters

Sensitivity and specificity are the paper's main quantitative result. They appear in the abstract, the highlights, Table 1, the Results and the Discussion, and the specificity feeds the Discussion's estimate of how far the test would cut unnecessary biopsies.

  1. Specificity is overstated. With the cohort the paper describes, specificity is 78.9% (62.7–90.4), not 82.6% (68.6–92.2). The highlights' "83%" becomes 79%. PPV (86.9% vs 87.1%) and NPV (96.8% vs 97.4%) are slightly overstated too. Sensitivity is essentially unchanged (98.1% vs 98.2%), so the benchmarking statement that the BDI is "the most sensitive (98%)" still holds as rounded.
  2. The unnecessary-biopsy estimate changes. The Discussion's 17.4% is 100 − 82.6, the proportion of benign lesions the test would call positive. (The text calls 82.6% the "FP rate"; it is the specificity, and 17.4% is the false-positive rate.) With the stated cohort that proportion is 8/38 = 21.1%.
  3. The headline figures are not validation-only figures. The abstract and highlights call them results of "a multi-center validation study", but the Results state that Table 1 covers "all the subjects (n = 92)", which includes the 40-subject discovery cohort in which the 1.25 cutoff was selected to maximise sensitivity. In the 52 validation subjects alone, sensitivity is 28/29 = 96.6% (82.2–99.9) and specificity 18/23 = 78.3% (56.3–92.5).

The overall conclusion, that the BDI separates OSCC from benign lesions with high sensitivity and moderate specificity, survives. What changes is the size of the headline specificity and its interval, and the description of which subjects those numbers come from.

Explanations considered

Status

Suggested correction

  1. Replace the Table 1 cells with the counts from Table S1 (TP 53, TN 30, FP 8, FN 1; N = 92) and recompute the four estimates and their CIs: sensitivity 98.1% (90.1–99.95), specificity 78.9% (62.7–90.4), PPV 86.9% (78.1–92.5), NPV 96.8% (81.0–99.5).
  2. Update the Abstract ("82.6% (95% CI 68.6–92.2)"), the Highlights ("83%"), the Results text ("specficity of 82.6%") and the Discussion's biopsy estimate (17.4% becomes about 21.1%). In the Discussion, describe 82.6% (corrected, 78.9%) as specificity rather than "FP rate".
  3. Either describe the pooled figures as discovery-plus-validation results, or report the validation-only estimates in the Abstract and Highlights (sensitivity 28/29 = 96.6%, specificity 18/23 = 78.3%), since the cutoff was selected in the discovery cohort.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy table of PMC10983043
(Cell Rep Med 2024;5:101447, doi:10.1016/j.xcrm.2024.101447).

Run:  python recompute.py.txt [path/to/mmc2.xlsx]
Needs numpy, scipy and matplotlib (openpyxl only if mmc2.xlsx is given).
Prints every recomputed number used in writeup.md and writes, next to this file,
  fig1.svg/png, fig2.svg/png               (the figures used in writeup.md)
  candidates/c1..c5_*.svg/png              (five alternative figure designs)

Inputs are copied from the paper as archived in PMC:
  Results (discovery-phase study): 40 subjects, 25 cancerous, 15 benign; cutoff 1.25
      gives 100% sensitivity and 80% specificity
  Results (validation study) and Figure 4D legend: 29 OSCC, 23 non-cancerous lesions
      in three cohorts of 21, 19 and 12 subjects
  Results: "Among 92 subjects, we recorded eight false positives (FPs) and one FN"
  Table 1: TP 54, TN 38, FP 8, FN 1 and the four accuracy estimates with 95% CIs
  STAR Methods: proof-of-concept cohort of 8 subjects (4 OSCC + 4 benign, Results)
Per-cohort classification counts are aggregated from the TP/TN/FP/FN column of
Table S1 (supplement mmc2.xlsx). Pass the file path to re-derive them from the
subject-level rows; only aggregate counts are used, no subject-level data are printed.
"""
import os
import sys

import numpy as np
from scipy.stats import beta, norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (paper)
TABLE1 = dict(TP=54, TN=38, FP=8, FN=1)
TABLE1_PRINTED = {  # estimate, CI low, CI high (percent)
    "Sensitivity": (98.2, 90.28, 99.95),
    "Specificity": (82.6, 68.58, 92.18),
    "PPV": (87.10, 78.22, 92.69),
    "NPV": (97.44, 84.44, 99.63),
}
ABSTRACT_PRINTED = {"Sensitivity": (98.2, 90.3, 99.9), "Specificity": (82.6, 68.6, 92.2)}
DISC = dict(cancer=25, benign=15, sens=1.00, spec=0.80)       # Results, discovery
VAL = dict(cancer=29, benign=23)                               # Results, Fig 4D legend
SITES = dict(discovery=40, internal=21, external_19=19, external_12=12)  # Methods
STATED_N, STATED_FP, STATED_FN = 92, 8, 1                      # Results, validation
POC = dict(n=8, cancer=4, benign=4)                            # proof-of-concept
CUTOFF = 1.25

# Table S1 (mmc2.xlsx), TP/TN/FP/FN column, aggregated per cohort
S1 = {
    "Discovery (S1B, n=40)":            dict(TP=25, TN=12, FP=3, FN=0),
    "Internal validation (S1C, n=21)":  dict(TP=15, TN=4, FP=2, FN=0),
    "External, 12 subjects (S1D)":      dict(TP=0, TN=10, FP=2, FN=0),
    "External, 19 subjects (S1E)":      dict(TP=13, TN=4, FP=1, FN=1),
}


def s1_from_xlsx(path):
    """Re-derive the per-cohort counts from Table S1; check labels against BDI > 1.25."""
    import openpyxl
    ws = openpyxl.load_workbook(path, data_only=True).worksheets[0]
    out, sec, mism = {}, None, 0
    for r in list(ws.iter_rows(values_only=True))[1:]:
        if r[0] and str(r[0]).upper().startswith("TABLE"):
            sec = str(r[0]).split("]")[0] + "]"
            out[sec] = dict(TP=0, TN=0, FP=0, FN=0, cancer=0, benign=0)
            continue
        if sec is None or r[6] is None:
            continue
        canc = str(r[5]).strip().upper().startswith("Y")
        out[sec]["cancer" if canc else "benign"] += 1
        if r[11]:
            lab = str(r[11]).strip()
            out[sec][lab] += 1
            pos = float(r[6]) > CUTOFF
            exp = {(True, True): "TP", (False, False): "TN",
                   (True, False): "FP", (False, True): "FN"}[(pos, canc)]
            mism += exp != lab
    return out, mism


# ------------------------------------------------------------------ statistics
def cp(x, n, a=0.05):
    """Exact Clopper-Pearson interval."""
    lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
    hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
    return lo, hi


def logit_pv(TP, TN, FP, FN):
    """PPV/NPV with the Mercaldo et al. (2007) logit CI (as in MedCalc)."""
    z = norm.ppf(0.975)
    npos, nneg = TP + FN, TN + FP
    se, sp = TP / npos, TN / nneg
    ppv, npv = TP / (TP + FP), TN / (TN + FN)
    v_ppv = ((1 - se) / se) / npos + (sp / (1 - sp)) / nneg
    v_npv = (se / (1 - se)) / npos + ((1 - sp) / sp) / nneg if se < 1 else np.nan
    f = lambda p, v: (1 / (1 + np.exp(-(np.log(p / (1 - p)) - z * np.sqrt(v)))),
                      1 / (1 + np.exp(-(np.log(p / (1 - p)) + z * np.sqrt(v)))))
    return (ppv, *f(ppv, v_ppv)), (npv, *f(npv, v_npv))


def metrics(c):
    TP, TN, FP, FN = c["TP"], c["TN"], c["FP"], c["FN"]
    se, sp = TP / (TP + FN), TN / (TN + FP)
    (ppv, pl, ph), (npv, nl, nh) = logit_pv(TP, TN, FP, FN)
    return {
        "Sensitivity": (100 * se, *(100 * v for v in cp(TP, TP + FN)), f"{TP}/{TP + FN}"),
        "Specificity": (100 * sp, *(100 * v for v in cp(TN, TN + FP)), f"{TN}/{TN + FP}"),
        "PPV": (100 * ppv, 100 * pl, 100 * ph, f"{TP}/{TP + FP}"),
        "NPV": (100 * npv, 100 * nl, 100 * nh, f"{TN}/{TN + FN}"),
    }


def show(label, c):
    n = sum(c[k] for k in ("TP", "TN", "FP", "FN"))
    print(f"\n--- {label}: TP={c['TP']} TN={c['TN']} FP={c['FP']} FN={c['FN']}  "
          f"N={n}  cancers(TP+FN)={c['TP'] + c['FN']}  benign(TN+FP)={c['TN'] + c['FP']}")
    m = metrics(c)
    for k, (est, lo, hi, frac) in m.items():
        meth = "Clopper-Pearson" if k in ("Sensitivity", "Specificity") else "logit"
        print(f"  {k:<11} {frac:>6} = {est:6.2f}%   95% CI {lo:.2f}-{hi:.2f} ({meth})")
    return m


def add(*cs):
    return {k: sum(c[k] for c in cs) for k in ("TP", "TN", "FP", "FN")}


# ------------------------------------------------------------------ 1. head counts
print("=" * 78)
print("1. Head counts stated in the paper")
print(f"  Subjects by site 40+21+19+12 = {sum(SITES.values())}")
print(f"  Validation subjects 21+19+12 = {SITES['internal'] + SITES['external_19'] + SITES['external_12']}"
      f"  (= {VAL['cancer']} OSCC + {VAL['benign']} benign = {VAL['cancer'] + VAL['benign']})")
canc = DISC["cancer"] + VAL["cancer"]
ben = DISC["benign"] + VAL["benign"]
print(f"  Cancers 25+29 = {canc}; benign 15+23 = {ben}; total = {canc + ben}")
t1n = sum(TABLE1.values())
print(f"  Table 1 cells 54+38+8+1 = {t1n}  (stated n = {STATED_N}; excess = {t1n - STATED_N}"
      f" = FP+FN = {TABLE1['FP'] + TABLE1['FN']})")
print(f"  Table 1 implies cancers TP+FN = {TABLE1['TP'] + TABLE1['FN']} (stated {canc}),"
      f" benign TN+FP = {TABLE1['TN'] + TABLE1['FP']} (stated {ben})")
print(f"  Table 1 TP = {TABLE1['TP']} equals the stated cancer total ({canc});"
      f" TN = {TABLE1['TN']} equals the stated benign total ({ben})")

# ------------------------------------------------------------------ 2. accuracy
print("\n" + "=" * 78)
print("2. Accuracy estimates")
m_print = show("Table 1 cells as printed", TABLE1)
print("  Printed in Table 1:", {k: v for k, v in TABLE1_PRINTED.items()})
for k, (e, lo, hi) in TABLE1_PRINTED.items():
    r = m_print[k]
    print(f"   {k:<11} printed {e} ({lo}-{hi}) vs recomputed {r[0]:.2f} ({r[1]:.2f}-{r[2]:.2f})"
          f" -> {'match' if abs(round(r[1], 2) - lo) < 0.006 and abs(round(r[2], 2) - hi) < 0.006 else 'DIFFER'}")

implied = dict(TP=canc - STATED_FN, TN=ben - STATED_FP, FP=STATED_FP, FN=STATED_FN)
m_impl = show("Implied by stated cohort (54 cancers, 38 benign) with 8 FP and 1 FN", implied)

pooled_s1 = add(*S1.values())
m_s1 = show("Table S1 labels, discovery + validation pooled", pooled_s1)
print(f"  identical to the cohort-implied table: {pooled_s1 == implied}")

disc = S1["Discovery (S1B, n=40)"]
val = add(*[v for k, v in S1.items() if not k.startswith("Discovery")])
print(f"\n  Discovery check: sensitivity {disc['TP']}/{disc['TP'] + disc['FN']} ="
      f" {disc['TP'] / (disc['TP'] + disc['FN']):.0%}; specificity {disc['TN']}/{disc['TN'] + disc['FP']}"
      f" = {disc['TN'] / (disc['TN'] + disc['FP']):.0%}  (paper: 100%, 80%)")
print(f"  Discovery counts implied by the text alone: TP = {round(DISC['sens'] * DISC['cancer'])},"
      f" TN = {round(DISC['spec'] * DISC['benign'])}, FP = {DISC['benign'] - round(DISC['spec'] * DISC['benign'])}")
m_val = show("Validation cohorts only (S1C+S1D+S1E)", val)
for k, c in S1.items():
    nb = c["TN"] + c["FP"]
    print(f"  {k:<34} specificity {c['TN']}/{nb} = {100 * c['TN'] / nb:.1f}%"
          + (f"; sensitivity {c['TP']}/{c['TP'] + c['FN']}" if c['TP'] + c['FN'] else ""))

print("\n  Abstract (rounded to 1 dp):", ABSTRACT_PRINTED)
for k in ("Sensitivity", "Specificity"):
    print(f"   {k}: Table 1 cells -> {m_print[k][0]:.1f} ({m_print[k][1]:.2f}-{m_print[k][2]:.2f});"
          f" stated cohort -> {m_impl[k][0]:.1f} ({m_impl[k][1]:.2f}-{m_impl[k][2]:.2f})")
print(f"  Highlights '98% and 83%' -> stated cohort {m_impl['Sensitivity'][0]:.0f}% and"
      f" {m_impl['Specificity'][0]:.0f}%")

# ------------------------------------------------------------------ 3. derived claims
print("\n" + "=" * 78)
print("3. Derived claims")
print(f"  Discussion '17.4%' = 100 - 82.6 = {100 - 82.6:.1f}% (i.e. 1 - specificity, 8/46 = {100 * 8 / 46:.1f}%)")
print(f"  With the stated cohort: 1 - specificity = 8/38 = {100 * 8 / 38:.1f}%")
print(f"  Point-estimate change in specificity: {m_print['Specificity'][0]:.2f} -> {m_impl['Specificity'][0]:.2f}"
      f" ({m_impl['Specificity'][0] - m_print['Specificity'][0]:+.2f} points);"
      f" lower CI bound {m_print['Specificity'][1]:.2f} -> {m_impl['Specificity'][1]:.2f}")

# ------------------------------------------------------------------ 4. alternatives
print("\n" + "=" * 78)
print("4. Innocent explanations: can any stated grouping give 55 cancers / 46 benign / 101?")
print(f"  + proof-of-concept cohort: {STATED_N} + {POC['n']} = {STATED_N + POC['n']};"
      f" cancers {canc + POC['cancer']}, benign {ben + POC['benign']}")
print(f"  validation only: {VAL['cancer']} cancers, {VAL['benign']} benign, n = {VAL['cancer'] + VAL['benign']}")
print(f"  discovery only: {DISC['cancer']} cancers, {DISC['benign']} benign, n = {DISC['cancer'] + DISC['benign']}")
print(f"  rounding: 53/54 = {100 * 53 / 54:.3f}% -> {100 * 53 / 54:.1f}; CP lower bound {100 * cp(53, 54)[0]:.2f}"
      f" vs printed 90.28; 54/55 CP lower bound {100 * cp(54, 55)[0]:.2f}")
print(f"  30/38 = {100 * 30 / 38:.2f}%; 38/46 = {100 * 38 / 46:.2f}%; with FP = 8, TN/(TN+8) = 0.826 needs TN = 38, i.e. 46 benign")

if len(sys.argv) > 1:
    print("\n" + "=" * 78)
    print("5. Re-derived from", os.path.basename(sys.argv[1]))
    out, mism = s1_from_xlsx(sys.argv[1])
    for k, v in out.items():
        print(f"  {k}: {v}")
    print(f"  labels disagreeing with BDI > {CUTOFF}: {mism}")

# ------------------------------------------------------------------ figures
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
TINT_O, TINT_B = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5  # 2 px at 96 px/in


def save(fig, path):
    fig.savefig(path + ".svg")
    fig.savefig(path + ".png", dpi=200)
    plt.close(fig)


def box(ax, x, y, w, h, fc=BG, ec=INK2, lw=1.0, r=0.012):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
                                fc=fc, ec=ec, lw=lw, transform=ax.transAxes))


def f1(v):
    """One decimal, except keep two when one decimal would print 100.0."""
    t = f"{v:.1f}"
    return f"{v:.2f}" if t == "100.0" else t


# --- design 3: two 2x2 tables (default fig1)
def fig_two_tables(path):
    fig = plt.figure(figsize=(7.2, 3.5))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    panels = [
        (0.025, "Table 1 as printed", ORANGE, TINT_O, TABLE1, m_print,
         "the paper's reported cells"),
        (0.525, "Rebuilt from the stated cohort", BLUE, TINT_B, implied, m_impl,
         "54 cancers, 38 benign; 8 FP, 1 FN"),
    ]
    for x0, title, col, tint, c, m, sub in panels:
        ax.add_patch(Rectangle((x0, 0.865), 0.011, 0.075, fc=col, ec="none", transform=ax.transAxes))
        ax.text(x0 + 0.022, 0.915, title, fontsize=12, weight="bold", va="center")
        ax.text(x0 + 0.022, 0.865, sub, fontsize=10, color=INK2, va="center")
        cw, ch = 0.10, 0.14
        gx, gy = x0 + 0.10, 0.44
        for i, lab in enumerate(("Cancer", "Benign")):
            ax.text(gx + (i + 0.5) * cw, gy + 2 * ch + 0.03, lab, ha="center", fontsize=10.5, color=INK2)
        ax.text(gx + 2 * cw + 0.06, gy + 2 * ch + 0.03, "Total", ha="center", fontsize=10.5, color=INK2)
        ax.text(gx - 0.012, gy + 1.5 * ch, "BDI +", ha="right", va="center", fontsize=10.5, color=INK2)
        ax.text(gx - 0.012, gy + 0.5 * ch, "BDI −", ha="right", va="center", fontsize=10.5, color=INK2)
        cells = [(0, 1, "TP", c["TP"]), (1, 1, "FP", c["FP"]), (0, 0, "FN", c["FN"]), (1, 0, "TN", c["TN"])]
        for i, j, lab, v in cells:
            hl = lab in ("TP", "TN")
            ax.add_patch(Rectangle((gx + i * cw, gy + j * ch), cw, ch, fc=tint if hl else BG,
                                   ec=INK2, lw=0.8, transform=ax.transAxes))
            ax.text(gx + i * cw + 0.008, gy + j * ch + ch - 0.028, lab, fontsize=10, color=INK2, va="center")
            ax.text(gx + (i + 0.5) * cw, gy + (j + 0.40) * ch, str(v), fontsize=14,
                    weight="bold" if hl else "normal", ha="center", va="center")
        ncan, nben = c["TP"] + c["FN"], c["TN"] + c["FP"]
        n = ncan + nben
        for i, v in ((0, ncan), (1, nben)):
            ax.text(gx + (i + 0.5) * cw, gy - 0.065, str(v), ha="center", fontsize=12, weight="bold")
        ax.text(gx + 2 * cw + 0.06, gy - 0.065, f"N = {n}", ha="center", fontsize=12, weight="bold")
        ax.plot([gx, gx + 2 * cw + 0.115], [gy - 0.018] * 2, color=INK2, lw=0.8, transform=ax.transAxes)
        ok = (ncan, nben, n) == (canc, ben, STATED_N)
        ax.text(gx + 2 * cw + 0.115, gy - 0.145,
                "= stated 54 / 38 / 92" if ok else
                f"stated 54 / 38 / 92:  +{ncan - canc}, +{nben - ben}, +{n - STATED_N}",
                ha="right", fontsize=10, color=INK2)
        se, sp = m["Sensitivity"], m["Specificity"]
        ax.text(x0 + 0.022, 0.155, f"Sensitivity {se[3]} = {f1(se[0])}%  ({f1(se[1])}–{f1(se[2])})",
                fontsize=10.5)
        ax.text(x0 + 0.022, 0.085, f"Specificity {sp[3]} = {f1(sp[0])}%  ({f1(sp[1])}–{f1(sp[2])})",
                fontsize=10.5, weight="bold")
    ax.plot([0.5, 0.5], [0.06, 0.95], color=GRID, lw=1.0, transform=ax.transAxes)
    ax.text(0.5, 0.018, "Shaded cells differ between the two tables. 95% CIs: Clopper–Pearson.",
            ha="center", fontsize=10, color=INK2)
    save(fig, path)


# --- design 4: dot-and-interval (default fig2)
def fig_dots(path):
    rows = ["Sensitivity", "Specificity", "PPV", "NPV"]
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    fig.subplots_adjust(left=0.16, right=0.67, top=0.86, bottom=0.17)
    for i, k in enumerate(rows):
        y = len(rows) - 1 - i
        e, lo, hi = TABLE1_PRINTED[k]
        ax.plot([lo, hi], [y + 0.14] * 2, color=ORANGE, lw=LW, solid_capstyle="butt")
        ax.plot(e, y + 0.14, "o", color=ORANGE, ms=8, mec=BG, mew=1.2)
        r = m_impl[k]
        ax.plot([r[1], r[2]], [y - 0.14] * 2, color=BLUE, lw=LW, solid_capstyle="butt")
        ax.plot(r[0], y - 0.14, "s", color=BLUE, ms=8, mec=BG, mew=1.2)
        ax.text(101.5, y + 0.14, f"{f1(e)}  ({f1(lo)}–{f1(hi)})  {m_print[k][3]}", va="center", fontsize=10, color=INK)
        ax.text(101.5, y - 0.14, f"{f1(r[0])}  ({f1(r[1])}–{f1(r[2])})  {r[3]}", va="center", fontsize=10, color=INK)
    ax.set_yticks(range(len(rows))); ax.set_yticklabels(rows[::-1], fontsize=11, color=INK)
    ax.set_xlim(50, 100); ax.set_ylim(-0.6, len(rows) - 0.4)
    ax.set_xlabel("Estimate, % (95% CI); axis starts at 50%")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.plot([], [], "o-", color=ORANGE, ms=8, lw=LW, label="Reported (Table 1 cells, N = 101)")
    ax.plot([], [], "s-", color=BLUE, ms=8, lw=LW, label="Recomputed (stated cohort, N = 92)")
    ax.legend(loc="lower left", bbox_to_anchor=(-0.02, 1.0), ncol=2, frameon=False, fontsize=10,
              handlelength=1.8, columnspacing=1.2)
    save(fig, path)


# --- design 1: text vs table
def fig_text_vs_table(path):
    fig = plt.figure(figsize=(7.2, 3.7))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    box(ax, 0.02, 0.30, 0.59, 0.62)
    ax.text(0.04, 0.86, "Results text and Figure 4 legends", fontsize=11.5, weight="bold", va="center")
    lines = [
        ("Discovery:", "“25 were diagnosed ... as being cancerous"),
        ("", "while 15 were benign”"),
        ("Validation:", "“23 non-cancerous and 29 OSCC lesions”"),
        ("Pooled:", "“For all the subjects (n = 92)”"),
        ("", "“eight false positives (FPs) and one FN”"),
    ]
    for i, (a, b) in enumerate(lines):
        y = 0.77 - i * 0.075
        ax.text(0.04, y, a, fontsize=10, color=INK2, va="center")
        ax.text(0.155, y, b, fontsize=10, va="center")
    ax.add_patch(Rectangle((0.02, 0.30), 0.008, 0.62, fc=BLUE, ec="none", transform=ax.transAxes))
    ax.text(0.04, 0.36, f"→ {canc} cancers + {ben} benign = {STATED_N}", fontsize=11, weight="bold", va="center")
    box(ax, 0.69, 0.30, 0.29, 0.62)
    ax.add_patch(Rectangle((0.69, 0.30), 0.008, 0.62, fc=ORANGE, ec="none", transform=ax.transAxes))
    ax.text(0.71, 0.86, "Table 1", fontsize=11.5, weight="bold", va="center")
    for i, (k, v) in enumerate(TABLE1.items()):
        y = 0.77 - i * 0.075
        ax.text(0.72, y, {"TP": "True positive", "TN": "True negative", "FP": "False positive",
                          "FN": "False negative"}[k], fontsize=10, va="center")
        ax.text(0.955, y, str(v), fontsize=10.5, weight="bold", ha="right", va="center")
    ax.text(0.72, 0.43, "Sensitivity 98.2", fontsize=10, va="center")
    ax.text(0.72, 0.36, "Specificity 82.6", fontsize=10, va="center")
    ax.annotate("", xy=(0.685, 0.61), xytext=(0.615, 0.61), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
    ax.text(0.65, 0.655, "conflict", ha="center", fontsize=10, color=INK2)
    ax.text(0.5, 0.17, f"Table 1 cells sum to 54 + 38 + 8 + 1 = {t1n}, not {STATED_N}: they imply "
            f"{TABLE1['TP'] + TABLE1['FN']} cancers and {TABLE1['TN'] + TABLE1['FP']} benign lesions.",
            ha="center", fontsize=10.5)
    ax.text(0.5, 0.09, f"TP and TN equal the cohort totals ({canc}, {ben}), so the 8 FP and 1 FN are counted twice.",
            ha="center", fontsize=10.5, color=INK2)
    save(fig, path)


# --- design 2: calculation flow
def fig_flow(path):
    fig = plt.figure(figsize=(7.2, 3.4))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    box(ax, 0.02, 0.33, 0.19, 0.32)
    ax.text(0.115, 0.59, "Stated cohort", ha="center", fontsize=10.5, weight="bold")
    ax.text(0.115, 0.51, f"{canc} cancers", ha="center", fontsize=10.5)
    ax.text(0.115, 0.44, f"{ben} benign", ha="center", fontsize=10.5)
    ax.text(0.115, 0.37, "8 FP, 1 FN", ha="center", fontsize=10, color=INK2)
    paths = [
        (0.60, ORANGE, "As reported in Table 1",
         [f"TP = {canc}\nTN = {ben}", "FP 8, FN 1 added\non top: N = 101", f"Se 54/55 = {m_print['Sensitivity'][0]:.1f}%\nSp 38/46 = {m_print['Specificity'][0]:.1f}%"],
         "cohort totals used\nas TP and TN"),
        (0.08, BLUE, "Recomputed from the paper's counts",
         [f"TP = {canc} − 1 = {implied['TP']}\nTN = {ben} − 8 = {implied['TN']}", "FP 8, FN 1 inside\nthe totals: N = 92",
          f"Se 53/54 = {m_impl['Sensitivity'][0]:.1f}%\nSp 30/38 = {m_impl['Specificity'][0]:.1f}%"],
         "errors taken out\nof the totals"),
    ]
    xs = [0.28, 0.49, 0.70]
    ws = [0.18, 0.18, 0.28]
    h = 0.24
    for y0, col, lab, steps, note in paths:
        ax.add_patch(Rectangle((0.28, y0 + h + 0.035), 0.011, 0.055, fc=col, ec="none", transform=ax.transAxes))
        ax.text(0.30, y0 + h + 0.062, lab, fontsize=10.5, weight="bold", va="center")
        for j, (x, w, s) in enumerate(zip(xs, ws, steps)):
            last = j == len(steps) - 1
            box(ax, x, y0, w, h, ec=col if last else INK2, lw=LW if last else 0.8)
            ax.text(x + w / 2, y0 + h / 2, s, ha="center", va="center", fontsize=10,
                    weight="bold" if last else "normal", linespacing=1.4)
            if j:
                ax.annotate("", xy=(x, y0 + h / 2), xytext=(xs[j - 1] + ws[j - 1], y0 + h / 2),
                            xycoords="axes fraction", arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0))
        ax.annotate("", xy=(xs[0], y0 + h / 2), xytext=(0.21, 0.49), xycoords="axes fraction",
                    arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0, connectionstyle="arc3,rad=0"))
    save(fig, path)


# --- design 5: icon array by cell
def fig_icons(path):
    fig = plt.figure(figsize=(7.2, 3.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.set_xlim(0, 72); ax.set_ylim(0, 30)
    s, gap, per = 1.0, 0.25, 30
    rows = [("TP", implied["TP"], TABLE1["TP"]), ("TN", implied["TN"], TABLE1["TN"]),
            ("FP", implied["FP"], TABLE1["FP"]), ("FN", implied["FN"], TABLE1["FN"])]
    y = 24.0
    for lab, n_ok, n_tab in rows:
        ax.text(1.0, y - 0.6, lab, fontsize=11, weight="bold", va="center")
        for k in range(n_tab):
            cx, cy = 5 + (k % per) * (s + gap), y - (k // per) * (s + gap) - 1
            extra = k >= n_ok
            ax.add_patch(Rectangle((cx, cy), s, s, fc=BG if extra else BLUE,
                                   ec=ORANGE if extra else BLUE, lw=1.2,
                                   hatch="////" if extra else None))
        ax.text(5 + per * (s + gap) + 1.0, y - 0.6,
                f"Table 1: {n_tab}   stated cohort: {n_ok}" + (f"   (+{n_tab - n_ok})" if n_tab > n_ok else ""),
                fontsize=10.5, va="center")
        y -= 2.5 + 2.5 * ((n_tab - 1) // per)
    ax.add_patch(Rectangle((5, 3.6), s, s, fc=BLUE, ec=BLUE))
    ax.text(6.6, 4.1, "subject counted in both (Table S1 label; recomputed)", fontsize=10, va="center")
    ax.add_patch(Rectangle((5, 1.4), s, s, fc=BG, ec=ORANGE, lw=1.2, hatch="////"))
    ax.text(6.6, 1.9, "extra count present only in Table 1 as reported (9 in all: N = 101 vs 92)",
            fontsize=10, va="center")
    ax.text(1.0, 28.0, "Each square is one count in the 2×2 table", fontsize=11.5, weight="bold", va="center")
    save(fig, path)


fig_two_tables(os.path.join(HERE, "fig1"))
fig_dots(os.path.join(HERE, "fig2"))
fig_text_vs_table(os.path.join(CAND, "c1_text_vs_table"))
fig_flow(os.path.join(CAND, "c2_calculation_flow"))
fig_two_tables(os.path.join(CAND, "c3_two_tables"))
fig_dots(os.path.join(CAND, "c4_dot_interval"))
fig_icons(os.path.join(CAND, "c5_icon_array"))
print("\nFigures written to", HERE, "and", CAND)


# ------------------------------------------------------------------ contact sheet of the five designs
def contact_sheet(path):
    import matplotlib.image as mpimg
    names = [("c1_text_vs_table", "1. Text vs table: conflicting excerpts joined by a connector"),
             ("c2_calculation_flow", "2. Calculation flow: how 54 / 38 became TP / TN"),
             ("c3_two_tables", "3. Two 2×2 tables side by side (used as fig1)"),
             ("c4_dot_interval", "4. Dot-and-interval plot of the four estimates (used as fig2)"),
             ("c5_icon_array", "5. Icon array: one square per count, extra counts hatched")]
    imgs = [mpimg.imread(os.path.join(CAND, n + ".png")) for n, _ in names]
    hs = [im.shape[0] / im.shape[1] * 7.2 for im in imgs]
    fig = plt.figure(figsize=(7.2, sum(hs) + 0.45 * len(imgs)))
    y = 1.0
    H = sum(hs) + 0.45 * len(imgs)
    for im, h, (_, lab) in zip(imgs, hs, names):
        y -= 0.45 / H
        fig.text(0.01, y + 0.12 / H, lab, fontsize=11, weight="bold", color=INK)
        ax = fig.add_axes([0, y - h / H, 1, h / H]); ax.imshow(im); ax.axis("off")
        y -= h / H
    fig.savefig(path + ".png", dpi=110)
    plt.close(fig)


contact_sheet(os.path.join(CAND, "all_five_designs"))
print("Contact sheet written to", os.path.join(CAND, "all_five_designs.png"))

Abstract reports all-grade adverse event rates as grade ≥3 treatment-related rates in a phase I trial

Clin Cancer Res (2025) · doi:10.1158/1078-0432.CCR-24-1983 · PMC12163594

Summary

The Abstract of this first-in-human phase I trial of intratumoral SAR441000, given alone or with cemiplimab, states that the most common grade ≥3 treatment-related adverse events (TRAEs) were fatigue in escalation (28.6% with monotherapy, 66.7% with the combination) and injection-site pain in expansion (31.7%). All three percentages equal the all-grade, all-causality treatment-emergent AE (TEAE) rates in Table 2B (6/21, 10/15 and 13/41). In the same table, grade ≥3 fatigue is 0/21 and 0/15, and grade ≥3 injection-site pain is 1/41 (2.4%). The paper's own Table 2A and Results text add that no monotherapy patient had a grade ≥3 TRAE of any kind and that only 1 of 15 escalation-combination patients did. Safety was a primary objective, and this sentence is the Abstract's only quantitative safety result. As printed, it overstates severe treatment-related toxicity by about 10-fold or more.

What the paper reports

The most common grade ≥3 treatment–related adverse events was fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%) and injection-site pain (31.7%) in the expansion phase.

— Abstract, Results

Of the 21 patients, 19 (90.5%) experienced any-grade AEs, of which the most common were vomiting and fatigue in 6 (28.6%) patients each, followed by hypertension, chills, and pain in extremities in 4 (19.0%) patients each. Nine (42.9%) patients experienced grade ≥3 AEs (Table 2). Furthermore, 13 (61.9%) patients reported TRAEs, but none were grade ≥3 or considered serious; grade ≥5 AEs were not reported.

— Results, Safety, "Escalation monotherapy"

All 15 patients experienced any-grade AEs, with the most common being fatigue (10/15; 66.7%), followed by hypertension and nausea in 5 (33.3%) patients each. Grade ≥3 AEs—hypertension, pyrexia, and anemia—were experienced by one (6.7%) patient each (Table 2). Ten (66.7%) patients experienced any-grade TRAEs, with the most common being fatigue (n = 7; 46.7%). One (6.7%) patient experienced grade ≥3 syncope (Supplementary Table S5).

— Results, Safety, "Escalation combination therapy"

All 41 patients experienced any-grade AEs, of which injection-site pain (n = 13; 31.7%) was the most common all-grade AE, followed by diarrhea, decreased appetite, and asthenia (n = 11 each; 26.8%; Table 2). [...] Thirty-three (80.5%) patients had any-grade TRAEs, of which six (14.6%) patients had TRAEs of grade ≥3 that were diarrhea, injection-site pain, and fatigue (Supplementary Table S5).

— Results, Safety, "Expansion combination"

Patients with any TRAE of grade ≥3 | 0 | 1 (6.7) | 6 (14.6)

— Table 2A ("AEs: (A) summary and (B) TEAEs occurring in ≥15% of patients"). The columns are escalation monotherapy (N = 21), escalation combination (N = 15) and expansion combination (N = 41).

Fatigue | 6 (28.6) | 0 | 10 (66.7) | 0 | 9 (22.0) | 1 (2.4)

Injection-site pain | 0 | 0 | 0 | 0 | 13 (31.7) | 1 (2.4)

— Table 2B, same three cohorts. Each cohort has an "All grades" column followed by a "Grade ≥3" column. The table footer defines the rows as TEAEs.

Intratumoral administration of SAR441000 in combination with cemiplimab was generally well tolerated with antitumor activity in the locoregional disease setting.

— Abstract, Conclusions

What the paper's own numbers show

Method. Each incidence is 100 × (patients with the event) / (patients in the cohort), with the cohort sizes from the Abstract and Table 2 (21, 15, 41). For each of the Abstract's three figures, the script tries every incidence the paper reports for the same AE and cohort: all-grade and grade ≥3 rates, all-causality TEAEs (Table 2B) and treatment-related AEs (Supplementary Table S5). It also checks the figure against the cohort's ceiling for grade ≥3 TRAEs of any kind (Table 2A). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Fatigue, escalation monotherapy, labelled "grade ≥3 treatment-related" Abstract 28.6% equals all-grade TEAE 6/21 = 28.6% (Table 2B)
Grade ≥3 fatigue, escalation monotherapy Table 2B; Suppl. Table S5 0; 0 0/21 = 0.0% (all causes and treatment-related)
Any grade ≥3 TRAE, escalation monotherapy Table 2A; Results 0; "none were grade ≥3" 0/21 = 0.0%. Grade ≥3 TR fatigue of 28.6% would need 6.0 patients
Fatigue, escalation combination, labelled "grade ≥3 treatment-related" Abstract 66.7% equals all-grade TEAE 10/15 = 66.7% (Table 2B)
Grade ≥3 fatigue, escalation combination Table 2B; Suppl. Table S5 0; 0 0/15 = 0.0%
Any grade ≥3 TRAE, escalation combination Table 2A; Results 1 (6.7%) 1/15 = 6.7%. Grade ≥3 TR fatigue of 66.7% would need 10.0 patients
Treatment-related fatigue, any grade, escalation combination Results; Suppl. Table S5 n = 7; 46.7% 7/15 = 46.7%, so 66.7% is not a TRAE rate at any grade
Injection-site pain, expansion, labelled "grade ≥3 treatment-related" Abstract 31.7% equals all-grade TEAE 13/41 = 31.7% (Table 2B)
Grade ≥3 injection-site pain, expansion Table 2B; Suppl. Table S5 1 (2.4); 1 (2.4) 1/41 = 2.4%; the Abstract's figure is 13.0 times this
Treatment-related injection-site pain, any grade, expansion Suppl. Table S5 11 (26.8) 11/41 = 26.8%
Any grade ≥3 TRAE, expansion Table 2A; Results 6 (14.6%) 6/41 = 14.6%
Largest number of patients with any single grade ≥3 TRAE term, expansion Suppl. Table S5 7 terms with 1 (2.4) each 1/41 = 2.4%; no term is more common than any other
Ratio of Abstract figure to the cohort's any-grade ≥3 TRAE rate derived — monotherapy: 28.6% vs 0% (no finite ratio); combination: 66.7 / 6.7 = 10.0; expansion: 31.7 / 14.6 = 2.2

All three Abstract figures match the all-grade, all-causality TEAE rate exactly. None matches any grade ≥3 rate or any treatment-related rate the paper reports. In the two escalation cohorts the stated figures cannot be grade ≥3 TRAE rates at all. Even if every patient with a grade ≥3 TRAE of any kind had had grade ≥3 fatigue, the maximum would be 0/21 and 1/15.

Figure

The abstract's three "grade ≥3 treatment-related" figures next to the Table 2B cells they equal and the grade ≥3 rates from the same rows

Figure 1. The left panel shows the Abstract sentence and its three percentages (orange). Each arrow joins a percentage to the Table 2B row it reproduces exactly. In all three rows the match is the "All grades" cell (orange), the count of patients with the event at any grade and from any cause. The blue cells show the grade ≥3 rate recomputed from the adjacent "Grade ≥3" count in the same row: 0/21, 0/15 and 1/41. Underneath, the right panel gives the cohort-level ceiling from Table 2A (any grade ≥3 TRAE: 0/21, 1/15, 6/41) and the Results text on monotherapy and on treatment-related fatigue. Only patient counts and percentages printed in the paper, or computed from them by recompute.py.txt, are shown.

Dot plot comparing, on one percentage axis, the abstract's figures with the all-grade rate, the grade ≥3 rate and the any grade ≥3 TRAE ceiling for each cohort

Figure 2. The same comparison on one axis (percent of patients in each cohort, 0–100%). Orange dots are the Abstract's figures. Each sits inside an open ring marking the all-grade, all-causality rate from Table 2B. Blue diamonds are the grade ≥3 rates of the same AE (Table 2B), and open blue squares are the cohort's rate of grade ≥3 TRAEs of any kind (Table 2A). That second rate is an upper bound for any single grade ≥3 TRAE. In escalation monotherapy the diamond and the square coincide at 0%. In each cohort the Abstract's figure lies well above that upper bound.

Why it matters

Overall safety and tolerability are part of the trial's primary objective. The Abstract sentence quoted above is its only quantitative statement on safety. A reader of the Abstract alone would conclude that two-thirds of patients in escalation combination had severe treatment-related fatigue, and that nearly a third of expansion patients had severe treatment-related injection-site pain. The paper's tables say neither happened: no escalation patient had grade ≥3 fatigue, and 1 of 41 expansion patients had grade ≥3 injection-site pain. The sentence also misidentifies the "most common" grade ≥3 TRAE. Monotherapy had none. In escalation combination, the one patient with a grade ≥3 TRAE is described as having syncope. In expansion, Supplementary Table S5 lists seven grade ≥3 TRAE terms with one patient each, so no single term stands out.

The error runs against the paper's own conclusion. As printed, the Abstract's figures sit uneasily with "generally well tolerated"; the correct figures (grade ≥3 TRAEs of any kind in 0%, 6.7% and 14.6% of patients) support it. Abstract-level safety figures from first-in-human trials are often carried into reviews, meta-analyses of intratumoral immunotherapies and dose-selection discussions. A 10-fold or larger overstatement of severe toxicity could be passed on in those places.

The error does not affect the maximum administered dose (4,000 µg), the response counts, the pharmacokinetic and pharmacodynamic results, or the Discussion. The Discussion describes AE frequencies without the mislabelled percentages. It ends: "SAR441000 demonstrated a generally acceptable safety profile in both monotherapy and combination therapy cohorts, although a higher number of high-grade AEs were observed in the combination therapy cohort."

Explanations considered

A smaller inconsistency sits in the paper's supporting material. The Results text names the single grade ≥3 TRAE in escalation combination as syncope, citing Supplementary Table S5. That table lists syncope as 1 (6.7) at all grades and 0 at grade ≥3, and lists dizziness and peripheral neuropathy as 1 (6.7) each at grade ≥3. This does not change the main finding: Table 2A gives one patient with any grade ≥3 TRAE in that cohort either way.

Status

As of 2026-10-23:

Sources: doi.org/10.1158/1078-0432.CCR-24-1983 · PMC12163594 · Europe PMC record · Europe PMC supplementary files · Crossref works record · ClinicalTrials.gov NCT03871348

Suggested correction

The Abstract sentence could be replaced with figures that match Table 2 and Supplementary Table S5. One option:

Grade ≥3 treatment-related adverse events occurred in 0 of 21 (0%), 1 of 15 (6.7%) and 6 of 41 (14.6%) patients in escalation monotherapy, escalation combination and expansion combination, respectively. The most common any-grade adverse events were fatigue in escalation (monotherapy: 28.6%; combination: 66.7%) and injection-site pain (31.7%) in expansion.

The authors could also check which preferred term was recorded for the single grade ≥3 TRAE in escalation combination. The Results text names syncope, while Supplementary Table S5 lists dizziness and peripheral neuropathy.

Recomputation script (python recompute.py.txt)
"""Recompute the abstract's grade >=3 treatment-related AE figures of PMC12163594
(Clin Cancer Res 2025, doi:10.1158/1078-0432.CCR-24-1983) and regenerate the figures.

Run:   python recompute.py.txt
Needs: numpy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand1..cand5, .svg/.png) into ./candidates/.

All inputs are patient counts copied from the paper:
  Table 2A  (AE summary), Table 2B (TEAEs occurring in >=15% of patients, all causality),
  the Results subsections "Escalation monotherapy", "Escalation combination therapy",
  "Expansion combination", and Supplementary Table S5 (treatment-related AEs).
Incidence (%) = 100 * patients with the event / patients in the cohort.
"""
import os

import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ----------------------------------------------------------------------------- inputs
N = {"mono": 21, "comb": 15, "exp": 41}
LABEL = {"mono": "Escalation monotherapy", "comb": "Escalation combination",
         "exp": "Expansion combination"}

# Abstract, Results: "The most common grade >=3 treatment-related adverse events was
# fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%)
# and injection-site pain (31.7%) in the expansion phase."
ABSTRACT = {"mono": ("Fatigue", 28.6), "comb": ("Fatigue", 66.7),
            "exp": ("Injection-site pain", 31.7)}

# Table 2B (TEAEs, all causality): (all grades n, grade >=3 n)
TEAE = {
    ("mono", "Fatigue"): (6, 0),
    ("comb", "Fatigue"): (10, 0),
    ("exp", "Fatigue"): (9, 1),
    ("mono", "Injection-site pain"): (0, 0),
    ("comb", "Injection-site pain"): (0, 0),
    ("exp", "Injection-site pain"): (13, 1),
}
# Supplementary Table S5 (treatment-related AEs): (all grades n, grade >=3 n)
TRAE = {
    ("mono", "Fatigue"): (2, 0),
    ("comb", "Fatigue"): (7, 0),
    ("exp", "Fatigue"): (7, 1),
    ("mono", "Injection-site pain"): (0, 0),
    ("comb", "Injection-site pain"): (1, 0),
    ("exp", "Injection-site pain"): (11, 1),
}
# Table 2A: patients with any TRAE / any TRAE of grade >=3 (any preferred term)
ANY_TRAE = {"mono": 13, "comb": 10, "exp": 33}
ANY_TRAE_G3 = {"mono": 0, "comb": 1, "exp": 6}
# Table 2A: patients with grade >=3 TEAEs (any term, all causality)
ANY_TEAE_G3 = {"mono": 9, "comb": 5, "exp": 21}
# Supplementary Table S5: preferred terms with a grade >=3 TRAE, patients per term
S5_G3_TERMS = {
    "mono": {},
    "comb": {"Dizziness": 1, "Neuropathy peripheral": 1},
    "exp": {"Diarrhoea": 1, "Muscular weakness": 1, "Fatigue": 1,
            "Injection site pain": 1, "(term ending '...deterioration')": 1,
            "Troponin increased": 1, "Hyperlipasaemia": 1},
}


def pct(n, d):
    return round(100 * n / d, 1)


# ----------------------------------------------------------------------- recompute
print("=" * 78)
print("1. The abstract's three figures against the paper's own counts")
print("=" * 78)
rows = []
for k in ("mono", "comb", "exp"):
    term, rep = ABSTRACT[k]
    a, g3 = TEAE[(k, term)]
    ta, tg3 = TRAE[(k, term)]
    d = N[k]
    r = dict(k=k, term=term, rep=rep, a=a, g3=g3, ta=ta, tg3=tg3, d=d,
             p_all=pct(a, d), p_g3=pct(g3, d), p_tall=pct(ta, d), p_tg3=pct(tg3, d),
             p_ceiling=pct(ANY_TRAE_G3[k], d), need=rep * d / 100)
    rows.append(r)
    print(f"\n{LABEL[k]} (N = {d}), {term}")
    print(f"  Abstract, labelled 'grade >=3 treatment-related':   {rep}%")
    print(f"  All-grade TEAE, all causality (Table 2B):          {a}/{d} = {r['p_all']}%"
          f"   -> matches abstract: {r['p_all'] == rep}")
    print(f"  Grade >=3 TEAE, all causality (Table 2B):          {g3}/{d} = {r['p_g3']}%")
    print(f"  Any-grade treatment-related (Suppl. Table S5):     {ta}/{d} = {r['p_tall']}%")
    print(f"  Grade >=3 treatment-related (Suppl. Table S5):     {tg3}/{d} = {r['p_tg3']}%")
    print(f"  Any grade >=3 TRAE of any type (Table 2A):         {ANY_TRAE_G3[k]}/{d} = {r['p_ceiling']}%")
    print(f"  Patients needed for {rep}%: {rep}*{d}/100 = {r['need']:.2f}"
          f"  (ceiling from Table 2A: {ANY_TRAE_G3[k]})")

print("\n" + "=" * 78)
print("2. Overstatement factors")
print("=" * 78)
for r in rows:
    k = r["k"]
    c = ANY_TRAE_G3[k]
    if c == 0:
        print(f"{LABEL[k]}: abstract {r['rep']}% vs any grade >=3 TRAE 0/{r['d']} = 0% (no finite ratio)")
    else:
        print(f"{LABEL[k]}: abstract {r['rep']}% / any grade >=3 TRAE {r['p_ceiling']}% = "
              f"{(r['rep'] / 100) / (c / r['d']):.1f}x  (in patients: {r['a']} vs {c})")
for r in rows:
    if r["tg3"]:
        print(f"{LABEL[r['k']]}: abstract {r['rep']}% vs grade >=3 TR {r['term'].lower()} "
              f"{r['p_tg3']}% = {r['a'] / r['tg3']:.1f}x  ({r['a']} vs {r['tg3']} patients)")

print("\n" + "=" * 78)
print("3. Other figures quoted in the write-up")
print("=" * 78)
for k in ("mono", "comb", "exp"):
    print(f"{LABEL[k]}: any TRAE {ANY_TRAE[k]}/{N[k]} = {pct(ANY_TRAE[k], N[k])}%; "
          f"grade >=3 TEAE (any) {ANY_TEAE_G3[k]}/{N[k]} = {pct(ANY_TEAE_G3[k], N[k])}%; "
          f"grade >=3 TRAE (any) {ANY_TRAE_G3[k]}/{N[k]} = {pct(ANY_TRAE_G3[k], N[k])}%")
print(f"Expansion fatigue: all-grade TEAE 9/41 = {pct(9, 41)}%, grade >=3 1/41 = {pct(1, 41)}%")
print("Grade >=3 TRAE terms in Suppl. Table S5 (patients per term):")
for k in ("mono", "comb", "exp"):
    terms = S5_G3_TERMS[k]
    mx = max(terms.values()) if terms else 0
    print(f"  {LABEL[k]}: {len(terms)} terms, max patients per term = {mx}"
          f"{' (' + str(round(100 * mx / N[k], 1)) + '%)' if mx else ''}: "
          f"{', '.join(terms) if terms else 'none'}")
print("  Escalation combination: Results text names the one grade >=3 TRAE as 'syncope';"
      " S5 lists syncope as 1 (6.7) all grades, 0 grade >=3.")

# --------------------------------------------------------------------------- style
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
    "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG,
    "axes.edgecolor": GRID, "figure.facecolor": BG, "axes.facecolor": BG,
    "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", facecolor=BG)
        fig.savefig(p + ".png", dpi=200, facecolor=BG)
    plt.close(fig)


def box(ax, x, y, w, h, fc, ec, lw=1.0, r=0.012):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
                                fc=fc, ec=ec, lw=lw, transform=ax.transAxes))


SHORT = {"mono": "Escalation mono (N = 21)", "comb": "Escalation combo (N = 15)",
         "exp": "Expansion combo (N = 41)"}


# ------------------------------------------------- candidate 1 / fig1: text panels
def draw_text_panels():
    fig = plt.figure(figsize=(W, 4.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    # left panel: abstract
    L0, L1 = 0.025, 0.36
    box(ax, L0, 0.20, L1 - L0, 0.71, "#ffffff", GRID)
    ax.text(L0 + 0.015, 0.875, "Abstract, Results", fontsize=11, fontweight="bold", va="top")
    ax.text(L0 + 0.015, 0.815,
            "“The most common grade ≥3\ntreatment–related adverse\n"
            "events was fatigue in the\nescalation phase\n(monotherapy: 28.6% and\n"
            "combination therapy:\n66.7%) and injection-site\npain (31.7%) in the\n"
            "expansion phase.”",
            fontsize=10, va="top", color=FG, linespacing=1.3, style="italic")
    ys = [0.395, 0.315, 0.235]
    for y, r in zip(ys, rows):
        box(ax, L0 + 0.012, y - 0.03, L1 - L0 - 0.024, 0.06, ORANGE_T, ORANGE, 1.2)
        ax.text(L0 + 0.025, y, f"{r['rep']}%", color=FG, fontsize=10.5, fontweight="bold",
                va="center")
        ax.text(L0 + 0.105, y, f"{r['term'].replace('Injection-site pain', 'Inj.-site pain')},"
                f" {('esc. mono', 'esc. combo', 'exp.')[ys.index(y)]}",
                color=FG, fontsize=10, va="center")
    # right panel: Table 2B + Table 2A + text
    R0, R1 = 0.53, 0.985
    box(ax, R0, 0.20, R1 - R0, 0.71, "#ffffff", GRID)
    ax.text(R0 + 0.015, 0.875, "Table 2B (TEAEs, all causality)", fontsize=11,
            fontweight="bold", va="top")
    cA, cG = R0 + 0.27, R0 + 0.385
    ax.text(cA, 0.765, "All\ngrades", ha="center", va="bottom", fontsize=10, color=FG2)
    ax.text(cG, 0.765, "Grade\n≥3", ha="center", va="bottom", fontsize=10, color=FG2)
    ax.plot([R0 + 0.015, R1 - 0.015], [0.755, 0.755], color=GRID, lw=1, transform=ax.transAxes)
    names = ["Fatigue\nesc. mono, N=21", "Fatigue\nesc. combo, N=15",
             "Inj.-site pain\nexpansion, N=41"]
    yr = [0.68, 0.565, 0.45]
    for y, nm, r in zip(yr, names, rows):
        ax.text(R0 + 0.015, y, nm, fontsize=10, va="center", linespacing=1.15)
        box(ax, cA - 0.06, y - 0.035, 0.12, 0.07, ORANGE_T, ORANGE, 1.2)
        ax.text(cA, y, f"{r['a']} ({r['p_all']})", ha="center", va="center", fontsize=10.5,
                fontweight="bold")
        box(ax, cG - 0.045, y - 0.035, 0.09, 0.07, BLUE_T, BLUE, 1.2)
        ax.text(cG, y, f"{r['p_g3']}%", ha="center", va="center", fontsize=10.5,
                fontweight="bold")
    ax.plot([R0 + 0.015, R1 - 0.015], [0.39, 0.39], color=GRID, lw=1, transform=ax.transAxes)
    ax.text(R0 + 0.015, 0.37,
            "Table 2A, any grade ≥3 TRAE:\n"
            f"  0/21 · 1/15 ({rows[1]['p_ceiling']}%) · 6/41 ({rows[2]['p_ceiling']}%)\n"
            "Results: monotherapy TRAEs “none\n  were grade ≥3”; combo "
            f"treatment-\n  related fatigue 7/15 = {rows[1]['p_tall']}%",
            fontsize=10, va="top", color=FG2, linespacing=1.3)
    # connectors: abstract value -> all-grade cell
    for y0, y1 in zip(ys, yr):
        ax.annotate("", xy=(R0 - 0.004, y1), xytext=(L1 - 0.01, y0),
                    xycoords="axes fraction",
                    arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4,
                                    connectionstyle="arc3,rad=-0.15", shrinkA=0, shrinkB=2))
    ax.text(0.445, 0.86, "same value,\ndifferent label:\n\u2018grade \u22653 TR\u2019\nvs \u2018all grades\u2019",
            ha="center", va="top", fontsize=10, color=FG2, linespacing=1.3)
    ax.text(0.5, 0.952, "Each abstract figure equals the all-grade, all-cause cell of Table 2B;\n"
            "the grade \u22653 cells are 0, 0 and 1 patient.", ha="center", va="center",
            fontsize=10, color=FG, fontweight="bold", linespacing=1.25)
    # legend / key at bottom
    box(ax, 0.025, 0.115, 0.03, 0.045, ORANGE_T, ORANGE, 1.2, r=0.006)
    ax.text(0.065, 0.1375, "value printed in the paper (abstract figure and the Table 2B cell it equals)",
            va="center", fontsize=10, color=FG2)
    box(ax, 0.025, 0.045, 0.03, 0.045, BLUE_T, BLUE, 1.2, r=0.006)
    ax.text(0.065, 0.0675, "grade ≥3 rate recomputed from Table 2B counts (n / N)",
            va="center", fontsize=10, color=FG2)
    return fig


save(draw_text_panels(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))


# ------------------------------------------------------ candidate 2 / fig2: dot plot
def draw_dots():
    fig, ax = plt.subplots(figsize=(W, 3.5))
    fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
    ylab = [f"{SHORT[r['k']]}\n{r['term']}" for r in rows]
    yy = [2, 1, 0]
    ax.set_xlim(-2, 100); ax.set_ylim(-0.6, 2.6)
    ax.set_yticks(yy); ax.set_yticklabels(ylab, fontsize=10)
    ax.set_xticks(range(0, 101, 20))
    ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
    ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Patients in cohort (%)", color=FG2)
    for y, r in zip(yy, rows):
        ax.plot([r["p_g3"], r["rep"]], [y, y], color=GRID, lw=2, zorder=1)
        ax.plot(r["p_all"], y, "o", ms=15, mfc="none", mec=FG2, mew=1.5, zorder=2)
        ax.plot(r["rep"], y, "o", ms=9, color=ORANGE, zorder=3)
        ax.plot(r["p_ceiling"], y, "s", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
        ax.plot(r["p_g3"], y, "D", ms=8, color=BLUE, zorder=4)
        ax.text(r["rep"] + 2.2, y + 0.08, f"{r['rep']}%", color=FG, fontsize=10, va="bottom",
                fontweight="bold")
        g3txt = f"{r['p_g3']}%" if r["p_g3"] == r["p_ceiling"] else \
            f"{r['p_g3']}% / {r['p_ceiling']}%"
        ax.text(max(r["p_g3"], r["p_ceiling"]) + 2.2, y - 0.1, g3txt, color=FG, fontsize=10,
                va="top")
    h = [plt.Line2D([], [], ls="none", marker="o", ms=9, color=ORANGE),
         plt.Line2D([], [], ls="none", marker="o", ms=13, mfc="none", mec=FG2, mew=1.5),
         plt.Line2D([], [], ls="none", marker="D", ms=8, color=BLUE),
         plt.Line2D([], [], ls="none", marker="s", ms=9, mfc=BG, mec=BLUE, mew=2)]
    lab = ["Abstract (‘grade ≥3 TRAE’)",
           "All grades, all causes (Table 2B)",
           "Grade ≥3, this AE (Table 2B)",
           "Any grade ≥3 TRAE (Table 2A)"]
    fig.legend(h, lab, loc="upper center", ncol=2, frameon=False, fontsize=10,
               bbox_to_anchor=(0.56, 1.0), handletextpad=0.4, columnspacing=1.2)
    return fig


save(draw_dots(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_plot"))


# ------------------------------------------------------ candidate 3: grouped bars
def draw_bars():
    fig, ax = plt.subplots(figsize=(W, 4.6))
    fig.subplots_adjust(left=0.36, right=0.95, top=0.95, bottom=0.11)
    metrics = [("Abstract (‘grade ≥3 TR’)", "rep", ORANGE),
               ("All grades, all cause", "p_all", BLUE),
               ("All grades, treatment-related", "p_tall", BLUE),
               ("Grade ≥3, all cause", "p_g3", BLUE),
               ("Grade ≥3, treatment-related", "p_tg3", BLUE)]
    ticks, labs = [], []
    y = 0
    for gi, r in enumerate(rows):
        for name, key, col in metrics:
            v = r[key]
            ax.barh(y, v, height=0.72, color=col, zorder=2)
            ax.text(v + 1.2, y, f"{v}%", va="center", fontsize=10, color=FG)
            ticks.append(y); labs.append(name)
            y -= 1
        ax.text(-0.53, y + len(metrics) + 0.95, f"{SHORT[r['k']]}: {r['term'].lower()}",
                transform=ax.get_yaxis_transform(), fontsize=10, fontweight="bold", va="center")
        y -= 1.6
    ax.set_yticks(ticks); ax.set_yticklabels(labs, fontsize=10)
    ax.set_xlim(0, 100)
    ax.set_xticks(range(0, 101, 20)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
    ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Patients in cohort (%). Orange: abstract figure; blue: Table 2B / Suppl. Table S5 counts ÷ N",
                  color=FG2, fontsize=10, loc="right")
    return fig


save(draw_bars(), os.path.join(CAND, "cand3_grouped_bars"))


# ------------------------------------------------------- candidate 4: calc flow
def draw_flow():
    fig = plt.figure(figsize=(W, 3.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    hdr = [(0.03, "Table 2B count"), (0.30, "÷ cohort N"), (0.53, "Result"),
           (0.775, "Abstract label")]
    for x, t in hdr:
        ax.text(x, 0.94, t, fontsize=10, color=FG2, va="center", fontweight="bold")
    ys = [0.74, 0.47, 0.20]
    for y, r in zip(ys, rows):
        term = r["term"].lower()
        # all-grade path
        box(ax, 0.03, y + 0.015, 0.24, 0.085, "#ffffff", GRID)
        ax.text(0.04, y + 0.0575, f"{r['a']} pts, {term},\nany grade", fontsize=10, va="center")
        ax.text(0.30, y + 0.0575, f"÷ {r['d']}", fontsize=10.5, va="center")
        box(ax, 0.53, y + 0.015, 0.22, 0.085, ORANGE_T, ORANGE, 1.2)
        ax.text(0.64, y + 0.0575, f"{r['p_all']}% (any grade)", fontsize=10, ha="center",
                va="center")
        ax.text(0.775, y + 0.0575, f"“grade ≥3 TR” {r['rep']}%\n= the any-grade rate",
                fontsize=10, va="center", color=FG)
        # grade >=3 path
        box(ax, 0.03, y - 0.09, 0.24, 0.085, "#ffffff", GRID)
        ax.text(0.04, y - 0.0475, f"{r['g3']} pt{'s' if r['g3'] != 1 else ''}, {term},\ngrade ≥3",
                fontsize=10, va="center")
        ax.text(0.30, y - 0.0475, f"÷ {r['d']}", fontsize=10.5, va="center")
        box(ax, 0.53, y - 0.09, 0.22, 0.085, BLUE_T, BLUE, 1.2)
        ax.text(0.64, y - 0.0475, f"{r['p_g3']}% (grade ≥3)", fontsize=10, ha="center",
                va="center")
        ax.text(0.775, y - 0.0475, "not reported\nin the abstract", fontsize=10, va="center",
                color=FG2)
        for yy in (y + 0.0575, y - 0.0475):
            ax.annotate("", xy=(0.525, yy), xytext=(0.36, yy), xycoords="axes fraction",
                        arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4))
        ax.text(0.03, y + 0.125, f"{LABEL[r['k']]} (N = {r['d']})", fontsize=10,
                fontweight="bold", va="center")
    ax.text(0.03, 0.035, "Orange: rate printed in the paper.  Blue: grade ≥3 rate recomputed "
            "from the same table.", fontsize=10, color=FG2, va="center")
    return fig


save(draw_flow(), os.path.join(CAND, "cand4_calc_flow"))


# ------------------------------------------------------- candidate 5: patient grid
def draw_grid():
    fig = plt.figure(figsize=(W, 4.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.set_xlim(0, 72); ax.set_ylim(0, 37)
    s, gap = 1.25, 0.35
    x0 = 32
    y = 33.5
    for r in rows:
        ax.text(1, y + 0.6, f"{LABEL[r['k']]}, {r['term'].lower()} (N = {r['d']})",
                fontsize=10, fontweight="bold", va="center")
        y -= 3.2
        for lab, cnt, col, colt in (
                (f"Abstract (‘grade ≥3 TR’): {r['rep']}% → {round(r['need'])} pts", round(r["need"]), ORANGE,
                 ORANGE_T),
                (f"Table 2B grade ≥3: {r['g3']} pt{'s' if r['g3'] != 1 else ''}", r["g3"], BLUE,
                 BLUE_T)):
            ax.text(1, y + s / 2, lab, fontsize=10, va="center")
            for i in range(r["d"]):
                xi = x0 + i * (s + gap) if r["d"] <= 27 else x0 + (i % 21) * (s + gap)
                yi = y if r["d"] <= 27 or i < 21 else y - (s + gap)
                filled = i < cnt
                ax.add_patch(Rectangle((xi, yi), s, s, fc=col if filled else BG,
                                       ec=col if filled else "#bdbbb5", lw=0.8))
            y -= 3.2 if r["d"] <= 27 else 4.8
        y -= 0.6
    ax.text(1, 1.2, "Each square is one patient. Filled: patients with the event at the rate "
            "named on the left.", fontsize=10, color=FG2, va="center")
    return fig


save(draw_grid(), os.path.join(CAND, "cand5_patient_grid"))
print("\nFigures written to", HERE, "and", CAND)

Table 1 crystallographic statistics do not match the AcrIIA19 structure described in the text and deposited

Commun Biol (2025) · doi:10.1038/s42003-025-09417-6 · PMC12855784

Summary

The paper's central result is a crystal structure of the anti-CRISPR protein AcrIIA19. The Results text describes it as a C222₁ crystal with four molecules in the asymmetric unit (ASU), solved at 1.98 Å, and refers the reader to Table 1 for the crystallographic statistics. Table 1 instead reports a tetragonal P4₁2₁2 cell (a = b = 64.43, c = 62.82 Å), data to 1.45 Å and one molecule in the ASU. These cannot describe the same crystal: four molecules per ASU in Table 1's cell would give a Matthews coefficient of 0.56 ų/Da, far below the physical minimum. The deposited entry the paper cites (PDB 9WA8) agrees with the text, not with Table 1, so the published data-collection and refinement statistics (resolution, R factors, completeness, B factors, atom counts, geometry) do not appear to belong to the structure the paper analyses.

What the paper reports

Results, "High-resolution structural analysis of AcrIIA19 reveals a previously uncharacterized structural fold", paragraph 1:

Following successful crystallization, the structure of AcrIIA19 was solved at a 1.98 Å resolution using MR, with the AlphaFold2 model used as a search template. Details on crystallographic and refinement statistics are listed in Table 1.

Same section, paragraph 2:

The crystal belonged to space group C2221 and contained four molecules (MolA to MolD) in the asymmetric unit (ASU; Fig. 1 C). The final structural model covered the complete AcrIIA19 (residues M1 to G115).

B-factor analysis showed relatively higher values at the N-terminus; however, the overall B-factor was low (average 48.8 Å), indicating that the structure was highly rigid (Fig. 1G).

The observation that all four molecules in the same ASU exhibited nearly identical structures served as additional indirect evidence supporting the high structural stability of AcrIIA19 (Fig. 1H).

Table 1, "Data collection and refinement statistics" (selected rows, as archived):

Space group | P 41 21 2

a, b, c (Å) | a = 64.43, b = 64.43, c = 62.82

Resolution range (Å) | 28.81–1.45

Total reflections | 616,092

Unique reflections | 24,054

Wilson B-factor (Å2) | 20.90

Rwork (%)a | 20.91 (27.03)

Rfree (%)a | 21.89 (25.83)

No. of molecules in the asymmetric unit | 1

No. of non-hydrogen atoms | 821

Macromolecules | 733

Average B-factor values (Åb) | 23.55

(Both resolution-range rows, under "Data collection" and under "Refinement", read 28.81–1.45. On the publisher's page the space group is typeset P 4₁ 2₁ 2.)

Figure 1 legend, panels C and H:

C Four molecules of AcrIIA19 in the same crystallographic asymmetry unit (ASU).

H Four superimposed AcrIIA19 structures in ASU.

Results, "AcrIIA19 forms a stable dimer in solution", paragraph 2:

The PISA complex formation score, which ranges from 0 to 1 with higher values indicating greater biological relevance, was 1.0 for both the AB and CD dimers.

Since the AB dimer and CD dimer represent essentially the same dimeric form (Fig. 2C, D), we selected the AB dimer as a representative structure to analyze in detail.

Data availability (back matter):

The coordinate and structural factors have been deposited in the Research Collaboratory for Structural Bioinformatics (RCSB) Protein Data Bank (PDB) under the PDB code 9WA8.

What the paper's own numbers show

Method. I compared the three crystallographic descriptors the text gives (space group, resolution, ASU content) with Table 1. I then tested whether each description is physically self-consistent. For each cell I computed the volume, and from it the Matthews coefficient Vm = V / (n_sym × n_mol × M), with n_sym = 8 general positions (true of both P4₁2₁2 and C222₁) and M = 14,545 Da, the chain mass of the deposited construct (residues 1–115 plus the LEHHHHHH tag). I counted the non-hydrogen atoms of a complete residue 1–115 chain from its sequence, and I enumerated the symmetry-unique reflections that Table 1's cell, space group and resolution range allow. Finally I compared both descriptions with the deposited entry PDB 9WA8 (RCSB metadata, coordinates and wwPDB validation report). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed
Space group Results ¶2 vs Table 1 C222₁ (text); P4₁2₁2 (Table 1) Orthorhombic vs tetragonal: different crystal systems, not alternative settings. PDB 9WA8: C 2 2 2₁
High-resolution limit Results ¶1 vs Table 1 1.98 Å (text); 28.81–1.45 Å (Table 1, both rows) Differ by 0.53 Å (26.8% of 1.98). PDB 9WA8: 28.95–1.98 Å
Molecules in the ASU Results ¶2, Fig. 1C/H vs Table 1 four (MolA to MolD); 1 PDB 9WA8: 4 chains (A–D)
Cell Table 1 a = b = 64.43, c = 62.82 Å Tetragonal metric (a = b), consistent with P4₁2₁2. V = 260,780 ų. PDB 9WA8 cell 63.41 × 98.98 × 196.66 Å, V = 1,234,301 ų (4.73× larger)
Matthews coefficient, Table 1 cell, 1 molecule per ASU derived from Table 1 — 2.24 ų/Da (45% solvent): normal
Matthews coefficient, Table 1 cell, 4 molecules per ASU derived from Table 1 + text — 0.56 ų/Da: impossible (protein crystals typically lie at about 1.7–3.5)
Matthews coefficient, deposited cell, 4 molecules per ASU PDB 9WA8 — 2.65 ų/Da (54% solvent); the entry itself records 2.65 and 53.62%
Protein non-H atoms Table 1 vs sequence 733 One complete chain M1–G115 has 946 (deposited chains C and D: 945 each), so 733 is 0.77 of one complete chain; four chains need about 3784. PDB 9WA8: 3845
Unique reflections Table 1 24,054 at 99.98% complete Table 1's cell and space group allow 24,053 to 28.81–1.45 Å: consistent. The deposited cell would allow 109,549 over that range
Multiplicity Table 1 25.6 616,092 / 24,054 = 25.6: consistent. PDB 9WA8 redundancy 13.2
R-work / R-free Table 1 vs PDB 9WA8 20.91 / 21.89% PDB 9WA8: 23.88 / 27.5%; 42,609 reflections in refinement vs Table 1's 24,052
Average B-factor Results ¶2 vs Table 1 48.8 (text); 23.55, Wilson 20.90 (Table 1) PDB 9WA8: mean atomic B 64.4, Wilson B 50.9 (validation report). The text's value is not reproduced by either source; it is far from Table 1's

Table 1 is internally consistent: its cell is tetragonal as its space group requires, the reflection count matches the cell and resolution almost exactly, the multiplicity matches its totals, and the atom count fits a single chain. It therefore reads as a complete statistics table for a single-chain, 1.45 Å, P4₁2₁2 crystal. The Results text, the Figure 1 and 2 legends, the dimer analysis and the deposited entry all describe a different crystal: C222₁, 1.98 Å, four chains. Table 1's model is also not a complete 1–115 chain, which the text says the final model is.

Figure

Results text and Table 1 give incompatible crystallographic descriptions; the deposited entry matches the text

Figure 1. The four descriptors on which the Results text and Table 1 disagree, as printed (orange). Each connector names the conflict. The blue lines are recomputed from Table 1's own cell: with the eight symmetry copies of P4₁2₁2 and a 14,545 Da chain, one molecule per ASU leaves a normal 2.24 ų/Da, while the text's four molecules would leave 0.56 ų/Da, which is impossible. The gray strip gives the deposited entry PDB 9WA8 that the paper's Data availability statement names; it matches the text on space group, resolution and ASU content.

Matthews coefficients implied by each description

Figure 2. Matthews coefficient (unit-cell volume per dalton of protein) for each combination of cell and ASU content, on a scale starting at zero. The shaded band is the typical range for protein crystals (about 1.7–3.5 ų/Da). Table 1's cell is plausible only with the one molecule Table 1 lists. The four molecules described in the text and in Figure 1C/H fit only the deposited C222₁ cell (2.65 ų/Da, the value recorded in the PDB entry).

Why it matters

The crystal structure is the paper's headline deliverable ("Here, we present the crystal structure of AcrIIA19" in the Abstract). Table 1 is where a reader, referee or re-user checks its quality: resolution, completeness, R-work/R-free, geometry, Ramachandran statistics and B factors. As published, those numbers appear to describe a different crystal. The quality indicators a reader would rely on for the 1.98 Å, four-chain structure are therefore not in the paper. Its deposited R-free (27.5%) is noticeably higher than Table 1's 21.89%, and the deposited 1.98 Å resolution is lower than Table 1's 1.45 Å.

The four-molecule ASU is not incidental. The near-identity of the four copies is offered as evidence of rigidity (Fig. 1H), and the dimer analysis (PISA scores for the AB and CD dimers, the AB interface, Fig. 2B–H) depends on four chains in the ASU. Those analyses are consistent with the deposited entry, so they are not themselves refuted; what is missing is the correct statistical description of the data behind them.

The main biological conclusions (a new fold, a homodimer in solution by SEC-MALS, WED-domain binding and inhibition of several Cas9 orthologs) rest largely on the deposited structure and on independent biochemical data, and are not contradicted by this problem.

Two smaller inconsistencies sit in the same section and do not change any conclusion. The Figure 2C legend says "The model shows two dimers formed by MolA/MolD and MolC/MolD", which names MolD twice, while the text analyses AB and CD dimers. The B-factor is given in Å in the text and "Åb" in Table 1, where Ų is meant.

Explanations considered

The paper does not say what crystal Table 1 describes. The average B-factor in the text (48.8) matches neither Table 1 (23.55) nor the deposited coordinates (mean 64.4; Wilson estimate 50.9). That leaves its source unclear.

Status

Suggested correction

  1. Replace Table 1 with the data-collection and refinement statistics for the deposited 9WA8 dataset: space group C222₁, cell 63.41 × 98.98 × 196.66 Å, 28.95–1.98 Å, four molecules per ASU, and the matching reflection counts, completeness, multiplicity, I/σ(I), R-merge, R-work/R-free, atom counts, B factors and geometry. The values should match those in the PDB entry and its validation report.
  2. If the present Table 1 describes another AcrIIA19 crystal (or another protein) from the same work, identify it, state its relation to the structure analysed, and deposit it, or remove it.
  3. Reconcile the "average 48.8" B-factor in the Results with the corrected table, and give its unit as Ų.
  4. Minor: correct the Figure 2C legend ("MolA/MolD and MolC/MolD") to the AB and CD dimers analysed in the text.
Recomputation script (python recompute.py.txt)
"""Recompute the crystallographic consistency checks for PMC12855784
(Commun Biol 2025, doi:10.1038/s42003-025-09417-6).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs in ./candidates/ (cand1..cand5 .svg/.png).

Inputs are copied from the paper as archived in PMC (Results text and Table 1)
and from the deposited entry the paper's Data availability statement names,
PDB 9WA8 (RCSB entry metadata, mmCIF header, coordinates and wwPDB validation
report, retrieved 2026-10-23). No network access is needed: the deposited values
are written out below.
"""
import os

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
import matplotlib.ticker  # noqa: E402,F401
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs: the paper
TEXT = {  # Results, "High-resolution structural analysis of AcrIIA19 ...", paragraphs 1-2
    "space_group": "C2221", "d_high": 1.98, "mol_asu": 4, "mean_B": 48.8,
    "residues": (1, 115),     # "The final structural model covered the complete AcrIIA19 (residues M1 to G115)"
}
TABLE1 = {  # Table 1, "Data collection and refinement statistics"
    "space_group": "P 41 21 2", "cell": (64.43, 64.43, 62.82), "angles": (90, 90, 90),
    "d_low": 28.81, "d_high": 1.45, "total_refl": 616092, "unique_refl": 24054,
    "multiplicity": 25.6, "completeness": 99.98, "wilson_B": 20.90,
    "refl_refine": 24052, "Rwork": 20.91, "Rfree": 21.89, "mol_asu": 1,
    "atoms_nonH": 821, "atoms_macro": 733, "atoms_ligand": 1, "atoms_solvent": 87,
    "mean_B": 23.55, "mean_B_macro": 22.41,
}
# ---------------------------------------------------------------- inputs: deposited PDB 9WA8
PDB = {
    "space_group": "C 2 2 21", "cell": (63.41, 98.98, 196.66), "Z_PDB": 32,
    "d_low": 28.95, "d_high": 1.98, "unique_refl": 42893, "completeness": 97.92,
    "redundancy": 13.2, "refl_refine": 42609, "Rwork": 23.88, "Rfree": 27.5,
    "chains": 4, "atoms_total": 3904, "atoms_protein": 3845, "atoms_solvent": 59,
    "matthews_deposited": 2.65, "solvent_pct_deposited": 53.62,
    "chain_fw_Da": 14545.0,     # entity 1 formula weight (residues 1-115 + LEHHHHHH tag)
    "wilson_B_validation": 50.871,
    "mean_B_coords": 64.36,     # mean B_iso_or_equiv over all 3904 deposited atoms
    "chain_atoms": {"A": 1002, "B": 953, "C": 945, "D": 945},  # protein atoms per chain
    "chain_last_res": {"A": 121, "B": 116, "C": 115, "D": 115},
}
SEQ = ("MKLIINIEDKNYKYLTELAQQDNTNIGSIVNNLIQTHITDVNESYRSVDKKELDEFSRVMQHYFHEDLASMYDVIG"
       "SDEELSTDKQMLKVYKKLYQDVALRNGIALELFNAYKKGLEHHHHHH")  # PDB 9WA8 entity 1 sequence

# Number of general positions (symmetry operators incl. centring) per cell
NSYM = {"P41212": 8, "C2221": 8}
# Typical range of Matthews coefficients for protein crystals (about 1.7-3.5; Matthews 1968)
VM_RANGE = (1.7, 3.5)


def vol(c):
    return c[0] * c[1] * c[2]  # all angles are 90 degrees in both cells


def matthews(V, nsym, nmol, mw):
    return V / (nsym * nmol * mw)


def solvent(vm):
    return 100 * (1 - 1.230 / vm)


# ---------------------------------------------------------------- 1. direct comparison
print("=== 1. Results text vs Table 1 vs deposited PDB 9WA8 ===")
rows = [
    ("Space group", TEXT["space_group"], TABLE1["space_group"], PDB["space_group"]),
    ("High-resolution limit (A)", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
    ("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
    ("Average B (A^2)", TEXT["mean_B"], TABLE1["mean_B"], f"{PDB['mean_B_coords']} (coords), Wilson {PDB['wilson_B_validation']}"),
]
for r in rows:
    print(f"  {r[0]:28s} text={r[1]!s:8s} table1={r[2]!s:10s} PDB={r[3]}")
print(f"  resolution difference text - table = {TEXT['d_high'] - TABLE1['d_high']:.2f} A "
      f"({100 * (TEXT['d_high'] - TABLE1['d_high']) / TEXT['d_high']:.1f}% of 1.98)")
a, b, c = TABLE1["cell"]
print(f"  Table 1 cell a=b ({a}={b}), c={c}: tetragonal metric, as P41212 requires; "
      f"C2221 (orthorhombic, C-centred) needs no a=b, and PDB cell a/b = {PDB['cell'][0] / PDB['cell'][1]:.3f}")

# ---------------------------------------------------------------- 2. Matthews coefficients
print("\n=== 2. Cell volume and Matthews coefficient (chain MW = 14,545 Da, PDB entity 1) ===")
mw = PDB["chain_fw_Da"]
V1 = vol(TABLE1["cell"])
V2 = vol(PDB["cell"])
VM = {
    "t1_1": matthews(V1, NSYM["P41212"], 1, mw),
    "t1_4": matthews(V1, NSYM["P41212"], 4, mw),
    "pdb_4": matthews(V2, NSYM["C2221"], 4, mw),
    "pdb_1": matthews(V2, NSYM["C2221"], 1, mw),
}
print(f"  Table 1 cell volume  = {V1:,.0f} A^3")
print(f"  PDB 9WA8 cell volume = {V2:,.0f} A^3  (ratio PDB/Table1 = {V2 / V1:.2f})")
print(f"  Table 1 cell, P41212, 1 molecule/ASU (as Table 1)  : Vm = {VM['t1_1']:.2f} A^3/Da, solvent {solvent(VM['t1_1']):.0f}%")
print(f"  Table 1 cell, P41212, 4 molecules/ASU (as text)    : Vm = {VM['t1_4']:.2f} A^3/Da, solvent {solvent(VM['t1_4']):.0f}%"
      f"  -> below the typical minimum {VM_RANGE[0]}")
print(f"  PDB cell,     C2221, 4 molecules/ASU (text, deposit): Vm = {VM['pdb_4']:.2f} A^3/Da, solvent {solvent(VM['pdb_4']):.0f}%"
      f"  (deposited value {PDB['matthews_deposited']}, {PDB['solvent_pct_deposited']}%)")
print(f"  PDB cell,     C2221, 1 molecule/ASU                 : Vm = {VM['pdb_1']:.2f} A^3/Da -> above the typical maximum {VM_RANGE[1]}")
print(f"  molecules per cell: Table 1 = {NSYM['P41212'] * 1}, text = {NSYM['C2221'] * 4} (PDB Z = {PDB['Z_PDB']})")

# ---------------------------------------------------------------- 3. atom counts
print("\n=== 3. Non-hydrogen atom counts ===")
HEAVY = dict(G=4, A=5, S=6, C=6, V=7, T=7, P=7, I=8, L=8, D=8, N=8, M=8, E=9, Q=9, K=9,
             H=10, F=11, R=11, Y=12, W=14)
seq115 = SEQ[:115]
assert seq115[0] == "M" and seq115[-1] == "G"   # "residues M1 to G115"
full115 = sum(HEAVY[x] for x in seq115) + 1      # +1 for the C-terminal OXT
print(f"  complete chain M1-G115 from sequence: {full115} non-H atoms (deposited chains C, D: "
      f"{PDB['chain_atoms']['C']}, {PDB['chain_atoms']['D']})")
print(f"  Table 1 macromolecule atoms = {TABLE1['atoms_macro']} = {TABLE1['atoms_macro'] / full115:.2f} x one complete chain; "
      f"four complete chains = {4 * full115}")
print(f"  PDB 9WA8 protein atoms = {PDB['atoms_protein']} = {PDB['atoms_protein'] / TABLE1['atoms_macro']:.2f} x Table 1")
print(f"  Table 1 sum check: {TABLE1['atoms_macro']} + {TABLE1['atoms_ligand']} + {TABLE1['atoms_solvent']} = "
      f"{TABLE1['atoms_macro'] + TABLE1['atoms_ligand'] + TABLE1['atoms_solvent']} (reported {TABLE1['atoms_nonH']})")

# ---------------------------------------------------------------- 4. reflection counts
print("\n=== 4. Unique reflections expected from each cell and resolution range ===")


def n_unique(cell, dlo, dhi, sg):
    a, b, c = cell
    H = int(a / dhi) + 1
    K = int(b / dhi) + 1
    L = int(c / dhi) + 1
    h, k, l = np.meshgrid(np.arange(0, H + 1), np.arange(0, K + 1), np.arange(0, L + 1), indexing="ij")
    s2 = (h / a) ** 2 + (k / b) ** 2 + (l / c) ** 2          # 1/d^2, orthogonal cell
    ok = (s2 <= 1 / dhi ** 2) & (s2 >= 1 / dlo ** 2) & ~((h == 0) & (k == 0) & (l == 0))
    if sg == "P41212":     # Laue 4/mmm asymmetric unit h>=k>=0, l>=0; absences 00l l=4n, h00 h=2n
        ok &= h >= k
        ok &= ~((h == 0) & (k == 0) & (l % 4 != 0))
        ok &= ~((k == 0) & (l == 0) & (h % 2 != 0))
    elif sg == "C2221":    # Laue mmm asymmetric unit h,k,l>=0; C-centring h+k=2n; 00l l=2n
        ok &= (h + k) % 2 == 0
        ok &= ~((h == 0) & (k == 0) & (l % 2 != 0))
    return int(ok.sum())


N1 = n_unique(TABLE1["cell"], TABLE1["d_low"], TABLE1["d_high"], "P41212")
N2 = n_unique(PDB["cell"], PDB["d_low"], PDB["d_high"], "C2221")
N1x = n_unique(PDB["cell"], TABLE1["d_low"], TABLE1["d_high"], "C2221")
print(f"  Table 1 cell, P41212, 28.81-1.45 A: {N1:,} possible; reported {TABLE1['unique_refl']:,} "
      f"at {TABLE1['completeness']}% -> implies {TABLE1['unique_refl'] / (TABLE1['completeness'] / 100):,.0f}")
print(f"  PDB cell, C2221, 28.95-1.98 A:     {N2:,} possible; deposited {PDB['unique_refl']:,} "
      f"at {PDB['completeness']}% -> implies {PDB['unique_refl'] / (PDB['completeness'] / 100):,.0f}")
print(f"  PDB cell, C2221, if it had been 28.81-1.45 A: {N1x:,} possible (Table 1 reports {TABLE1['unique_refl']:,})")
print(f"  multiplicity check Table 1: {TABLE1['total_refl']:,} / {TABLE1['unique_refl']:,} = "
      f"{TABLE1['total_refl'] / TABLE1['unique_refl']:.1f} (reported {TABLE1['multiplicity']})")

# ---------------------------------------------------------------- 5. ratios to the deposit
print("\n=== 5. Table 1 and text values relative to the deposited entry ===")
RATIOS = [  # label, text value, Table 1 value, PDB value
    ("High-resolution limit", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
    ("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
    ("Unit-cell volume", None, V1, V2),
    ("Unique reflections", None, TABLE1["unique_refl"], PDB["unique_refl"]),
    ("Protein non-H atoms", None, TABLE1["atoms_macro"], PDB["atoms_protein"]),
    ("Multiplicity", None, TABLE1["multiplicity"], PDB["redundancy"]),
    ("R-free (%)", None, TABLE1["Rfree"], PDB["Rfree"]),
]
for lab, t, tb, p in RATIOS:
    ts = f"text/PDB={t / p:.2f}" if t is not None else "text: not stated"
    print(f"  {lab:22s} Table1/PDB = {tb / p:.2f}   {ts}")
print(f"  Rwork Table 1 {TABLE1['Rwork']}% vs PDB {PDB['Rwork']}%; reflections in refinement {TABLE1['refl_refine']:,} vs {PDB['refl_refine']:,}")
print(f"  Average B: text {TEXT['mean_B']}; Table 1 {TABLE1['mean_B']} (Wilson {TABLE1['wilson_B']}); "
      f"PDB coordinates mean {PDB['mean_B_coords']}, validation Wilson {PDB['wilson_B_validation']}")

# ================================================================= figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})


def save(fig, path_stem):
    fig.savefig(path_stem + ".svg")
    fig.savefig(path_stem + ".png", dpi=200)
    plt.close(fig)


SG_T = "C222₁"          # C2221
SG_TB = "P4₁2₁2"   # P41212


# ---------- design 1: two text panels joined by annotated connectors (+ deposit strip)
def design_contradiction(stem):
    fig = plt.figure(figsize=(7.2, 5.6))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(0, 78)
    ax.axis("off")
    PT, PB = 67, 25  # panel top / bottom
    for x0, title, sub in [(2, "Results text", "High-resolution structural\nanalysis, paragraphs 1\u20132"),
                           (62, "Table 1", "Data collection and\nrefinement statistics")]:
        ax.add_patch(FancyBboxPatch((x0, PB), 36, PT - PB, boxstyle="round,pad=0.4,rounding_size=1.2",
                                    fc="white", ec=SEC, lw=1.0))
        ax.text(x0 + 1.5, PT - 3, title, fontsize=12, fontweight="bold", va="center")
        ax.text(x0 + 1.5, PT - 8.3, sub, fontsize=10, color=SEC, va="center", linespacing=1.15)
    left = [("space group", SG_T), ("solved at", "1.98 \u00c5 resolution"),
            ("asymmetric unit", "four molecules (MolA to MolD)"), ("overall B-factor", "average 48.8")]
    right = [("Space group", SG_TB), ("Resolution range", "28.81\u20131.45 \u00c5"),
             ("Molecules in the ASU", "1"), ("Average B-factor", "23.55")]
    conflict = ["orthorhombic vs\ntetragonal", "1.98 vs 1.45 \u00c5", "4 vs 1", "48.8 vs 23.55"]
    ys = [51.5, 43.5, 35.5, 28.5]
    for y, (l1, v1), (l2, v2), cf in zip(ys, left, right, conflict):
        ax.text(3.5, y + 1.6, l1, fontsize=10, color=SEC, va="center")
        ax.text(3.5, y - 1.3, v1, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
        ax.text(63.5, y + 1.6, l2, fontsize=10, color=SEC, va="center")
        ax.text(63.5, y - 1.3, v2, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
        ax.annotate("", xy=(61.0, y), xytext=(39.0, y),
                    arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5, shrinkA=0, shrinkB=0))
        ax.text(50, y + 0.9, cf, ha="center", va="bottom", fontsize=10, color=TXT, linespacing=1.05)
    ax.text(50, PT - 3, "conflict", ha="center", va="center", fontsize=11, fontweight="bold", color=TXT)
    # recomputed strip
    ax.text(2.5, 20.5, "Recomputed from Table 1's own cell (8 symmetry copies per cell, 14,545 Da chain):",
            fontsize=10, color=BLUE, va="center")
    ax.text(2.5, 17.2, f"1 molecule per ASU (as Table 1 states):  Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da, a normal value",
            fontsize=10, color=BLUE, va="center", fontweight="bold")
    ax.text(2.5, 13.9, f"4 molecules per ASU (as the text states):  Vm = {VM['t1_4']:.2f} \u00c5\u00b3/Da, physically impossible",
            fontsize=10, color=BLUE, va="center", fontweight="bold")
    # deposit strip
    ax.add_patch(FancyBboxPatch((2, 1.6), 96, 7, boxstyle="round,pad=0.3,rounding_size=1.0",
                                fc="#f1f0ec", ec="none"))
    ax.text(3.5, 6.6, "Deposited entry PDB 9WA8 (named in Data availability) matches the text:",
            fontsize=10, color=SEC, va="center")
    ax.text(3.5, 3.5, f"{SG_T}  \u00b7  1.98 \u00c5  \u00b7  4 chains  \u00b7  cell 63.41 \u00d7 98.98 \u00d7 196.66 \u00c5"
            f"  \u00b7  Vm {VM['pdb_4']:.2f} \u00c5\u00b3/Da", fontsize=10, color=SEC, va="center")
    # legend
    for x, col, lab in [(2.5, ORANGE, "printed in the paper"), (36, BLUE, "recomputed from Table 1"),
                        (71, SEC, "deposited entry")]:
        ax.add_patch(Rectangle((x, 73.2), 2.2, 2.6, fc=col, ec="none"))
        ax.text(x + 3.2, 74.5, lab, fontsize=10, color=TXT, va="center")
    save(fig, stem)


# ---------- design 2: scorecard matrix
def design_matrix(stem):
    rows = [
        ("Space group", SG_T, SG_TB, SG_T),
        ("Crystal system", "orthorhombic", "tetragonal", "orthorhombic"),
        ("Cell a, b, c (Å)", "not stated", "64.43, 64.43, 62.82", "63.41, 98.98, 196.66"),
        ("Resolution limit (\u00c5)", "1.98", "1.45", "1.98"),
        ("Molecules in ASU", "4 (MolA–MolD)", "1", "4 chains"),
        ("Average B-factor", "48.8", "23.55", "64.4 (Wilson 50.9)"),
        ("Protein non-H atoms", "not stated", "733", "3845"),
        ("Unique reflections", "not stated", "24,054", "42,893"),
        ("R-work / R-free (%)", "not stated", "20.91 / 21.89", "23.9 / 27.5"),
    ]
    fig = plt.figure(figsize=(7.2, 4.6))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(-0.4, len(rows) + 2.0)
    ax.axis("off")
    xs = [2, 27, 44, 66.5]
    heads = ["", "Results text", "Table 1", "PDB 9WA8"]
    top = len(rows) + 1.2
    for x, h in zip(xs, heads):
        ax.text(x, top, h, fontsize=11, fontweight="bold", va="center")
    ax.plot([2, 98], [top - 0.5, top - 0.5], color=SEC, lw=1)
    for i, (lab, t, tb, p) in enumerate(rows):
        y = len(rows) - i
        if i % 2 == 0:
            ax.add_patch(Rectangle((1, y - 0.48), 98, 0.96, fc="#f1f0ec", ec="none"))
        ax.text(xs[0], y, lab, fontsize=10, va="center", color=TXT)
        for x, v, col in [(xs[1], t, ORANGE), (xs[2], tb, ORANGE), (xs[3], p, SEC)]:
            ns = v == "not stated"
            ax.text(x, y, v, fontsize=10, va="center", color=SEC if ns else col,
                    fontweight="normal" if (ns or col == SEC) else "bold", style="italic" if ns else "normal")
        diff_tb = tb.split(" ")[0] != p.split(" ")[0]
        mark = "✕ differs" if diff_tb else "= same"
        ax.text(98.5, y, mark, fontsize=10, va="center", ha="right", color=TXT if diff_tb else SEC)
    ax.text(98.5, top, "Table 1 vs\ndeposit", fontsize=10, ha="right", va="center", color=SEC, linespacing=1.05)
    ax.text(2, 0.1, "Orange: printed in the paper.  Gray: deposited entry PDB 9WA8 (named in Data availability).",
            fontsize=10, color=SEC, va="center")
    save(fig, stem)


# ---------- design 3: unit cells to scale
def design_cells(stem):
    fig, axs = plt.subplots(1, 2, figsize=(7.2, 5.0), gridspec_kw=dict(width_ratios=[1, 1], wspace=0.3))
    a1, b1, c1 = TABLE1["cell"]
    a2, b2, c2 = PDB["cell"]
    for ax, (w1, h1, w2, h2, xl, yl, ttl) in zip(axs, [
            (a1, b1, a2, b2, "a (Å)", "b (Å)", "a × b face"),
            (a1, c1, a2, c2, "a (Å)", "c (Å)", "a × c face")]):
        ax.add_patch(Rectangle((0, 0), w2, h2, fc="none", ec=SEC, lw=2))
        ax.add_patch(Rectangle((0, 0), w1, h1, fc="none", ec=ORANGE, lw=2, ls=(0, (4, 2))))
        ax.set_xlim(0, 105)
        ax.set_ylim(0, 205 if "c face" in ttl else 105)
        ax.set_aspect("equal")
        ax.set_xlabel(xl)
        ax.set_ylabel(yl)
        ax.set_title(ttl, fontsize=11, loc="left")
        ax.grid(color=GRID, lw=0.6)
        ax.set_axisbelow(True)
        ax.tick_params(labelsize=10)
        for s in ("top", "right"):
            ax.spines[s].set_visible(False)
    axs[0].text(4, 32, "Table 1\n64.43 \u00d7 64.43", color=TXT, fontsize=10, va="center")
    axs[0].text(4, 82, "PDB 9WA8\n63.41 \u00d7 98.98", color=SEC, fontsize=10, va="center")
    axs[1].text(4, 31, "Table 1\n64.43 \u00d7\n62.82", color=TXT, fontsize=10, va="center")
    axs[1].text(4, 130, "PDB 9WA8\n63.41 \u00d7\n196.66", color=SEC, fontsize=10, va="center")
    fig.legend(handles=[Line2D([], [], color=ORANGE, lw=2, ls=(0, (4, 2)), label=f"Table 1 cell ({SG_TB}), V = {V1/1e3:,.0f} × 10³ ų"),
                        Line2D([], [], color=SEC, lw=2, label=f"Deposited 9WA8 cell ({SG_T}), V = {V2/1e3:,.0f} × 10³ ų")],
               loc="upper left", bbox_to_anchor=(0.06, 0.995), frameon=False, fontsize=10, ncol=1)
    fig.text(0.06, 0.065, f"Both space groups have 8 symmetry copies per cell; the deposited cell is {V2 / V1:.2f}\u00d7 larger.\n"
             f"Recomputed from Table 1's cell: Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da for 1 molecule per ASU (normal),\n"
             f"{VM['t1_4']:.2f} \u00c5\u00b3/Da for 4 molecules per ASU (physically impossible).",
             fontsize=10, color=BLUE, va="center", linespacing=1.2)
    fig.subplots_adjust(left=0.1, right=0.98, top=0.82, bottom=0.24)
    save(fig, stem)


# ---------- design 4: Matthews coefficient dot plot
def design_matthews(stem):
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    items = [
        ("Table 1 cell, P4\u20812\u20812, 1 molecule per ASU (as Table 1 states)", VM["t1_1"], BLUE, "o"),
        ("Table 1 cell, P4\u20812\u20812, 4 molecules per ASU (as the text states)", VM["t1_4"], BLUE, "o"),
        ("Deposited 9WA8 cell, C222\u2081, 4 molecules per ASU (text and deposit)", VM["pdb_4"], SEC, "D"),
    ]
    ax.axvspan(*VM_RANGE, color="#eceae4", zorder=0)
    ax.text(VM_RANGE[1] - 0.05, 3.05, f"typical range for protein\ncrystals, about {VM_RANGE[0]}\u2013{VM_RANGE[1]}",
            ha="right", va="top", fontsize=10, color=SEC, linespacing=1.1)
    for i, (lab, v, col, m) in enumerate(items):
        y = 2 - i
        ax.text(0.02, y + 0.3, lab, fontsize=10, color=TXT, va="center", ha="left")
        ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
        ax.plot(v, y, marker=m, ms=10, color=col, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.text(v + 0.09, y, f"{v:.2f}", va="center", fontsize=10.5, color=TXT, fontweight="bold")
    ax.set_yticks([])
    ax.set_xlim(0, 4.6)
    ax.set_ylim(-0.4, 3.1)
    ax.set_xlabel("Matthews coefficient Vm (\u00c5\u00b3/Da): cell volume \u00f7 (8 \u00d7 molecules per ASU \u00d7 14,545 Da)")
    ax.grid(axis="x", color=GRID, lw=0.6)
    ax.set_axisbelow(True)
    for sp in ("top", "right", "left"):
        ax.spines[sp].set_visible(False)
    fig.legend(handles=[Line2D([], [], marker="o", color=BLUE, ls="none", ms=9, label="recomputed from Table 1's cell"),
                        Line2D([], [], marker="D", color=SEC, ls="none", ms=9, label="computed from the deposited cell")],
               loc="lower center", bbox_to_anchor=(0.5, 0.0), frameon=False, fontsize=10, ncol=2)
    fig.tight_layout(rect=(0, 0.07, 1, 1))
    save(fig, stem)


# ---------- design 5: ratio-to-deposit dot plot (log scale)
def design_ratios(stem):
    fig, ax = plt.subplots(figsize=(7.2, 4.2))
    n = len(RATIOS)
    for i, (lab, t, tb, p) in enumerate(RATIOS):
        y = n - 1 - i
        ax.plot(tb / p, y, "o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.text(tb / p / 1.07, y, f"{tb / p:.2f}", fontsize=10, ha="right", va="center", color=TXT)
        if t is not None:
            ax.plot(t / p, y, "o", ms=10, mfc=BG, mec=ORANGE, mew=2, zorder=4)
    ax.axvline(1, color=SEC, lw=1)
    ax.text(1.03, -0.45, "same as deposit", fontsize=10, color=SEC, va="center")
    ax.set_xscale("log")
    ax.set_xlim(0.1, 2.5)
    ax.set_xticks([0.1, 0.2, 0.5, 1, 2])
    ax.set_xticklabels(["0.1", "0.2", "0.5", "1", "2"])
    ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_yticks(range(n))
    ax.set_yticklabels([r[0] for r in RATIOS][::-1], fontsize=10)
    ax.set_ylim(-0.6, n - 0.1)
    ax.set_xlabel("Value in the paper ÷ value in deposited PDB 9WA8 (log scale)")
    ax.grid(axis="x", color=GRID, lw=0.6)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, ls="none", ms=9, label="Table 1 (reported)"),
                       Line2D([], [], marker="o", mfc=BG, mec=ORANGE, mew=2, ls="none", ms=9,
                              label="Results text (reported)")],
              loc="lower left", bbox_to_anchor=(0.0, 1.0), frameon=False, fontsize=10, ncol=2)
    fig.tight_layout()
    save(fig, stem)


design_contradiction(os.path.join(HERE, "fig1"))
design_matthews(os.path.join(HERE, "fig2"))
design_contradiction(os.path.join(CAND, "cand1_contradiction_panels"))
design_matrix(os.path.join(CAND, "cand2_scorecard"))
design_cells(os.path.join(CAND, "cand3_unit_cells_to_scale"))
design_matthews(os.path.join(CAND, "cand4_matthews"))
design_ratios(os.path.join(CAND, "cand5_ratio_to_deposit"))
print("\nwrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")

Sahel-wide foliage increases of 6% and 20% do not match the relative increases implied by the paper's own totals

Commun Biol (2019) · doi:10.1038/s42003-019-0383-9 · PMC6478729

Summary

The Abstract and Results report that satellite (VOD) estimates show Sahel-wide increases of 6% in total herbaceous mass (AGH) and 20% in total woody foliage mass (WPF) between 1992 and 2012. The same Results sentence gives the totals behind these percentages: AGH 0.28 Pg in 1992 and 0.31 Pg in 2012, WPF 0.16 Pg and 0.24 Pg. Those totals give relative increases of 10.7% and 50.0%. Allowing for rounding of the two-decimal totals, the possible ranges are 7.0–14.5% and 42.4–58.1%, so neither reported figure can be reached as an increase over each component's own 1992 value. Two denominators the paper does not state (1992 total vegetation mass, or the sum of the two years) can reach both figures within rounding of the printed totals, but the unrounded Supplementary Data does not reproduce the 20% WPF figure on either (see "Explanations considered"). The totals agree with the rest of the paragraph and with the paper's Supplementary Data, which points to the two percentages as the inconsistent figures. They are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases. On the printed totals, the woody increase is understated about 2.5-fold (2.1–2.9-fold over rounding; 2.3-fold from the Supplementary Data endpoints) and the herbaceous increase about 1.8-fold (1.2–2.4-fold over rounding; 1.46-fold from the Supplementary Data endpoints). The direction of the main finding, that woody foliage rose faster than herbaceous mass, is unchanged.

What the paper reports

Abstract:

Satellite observations confirm these findings for the majority of the Sahel, with total herbaceous/woody foliage mass increases by 6%/20%.

Results, "Scaling field-site observations to the Sahel", paragraph 1 (the three sentences are consecutive):

For the entire western and central Sahel area, the summed AGH increased by 0.0025 Pg yr-1 over the 21-year period, and WPF by 0.004 Pg yr-1. Total AGH (WPF) was 0.28 Pg (0.16 Pg) in 1992 and 0.31 Pg (0.24 Pg) in 2012, which is a relative increase of 6% for AGH and 20% for WPF. The share of VOD estimated WPF (as related to the total vegetation mass) increased from 35% in 1992 to 43% in 2012, which is comparable to the percentage numbers observed from the Ferlo field samples.

The same paragraph, earlier:

According to fitted trends derived from VOD metrics from 1992 to 2012, AGH increased by + 5 ± 7 kg ha-1 yr-1 and WPF increased by +9 ± 8 kg ha-1 yr-1.

and later, giving the size of the significant-trend areas:

Areas with a significant (P < 0.05) increase in AGH cover a smaller region (289,375 km²; ~7%) than areas with a significant increase in WPF (2,711,875 km²; ~64%)

Discussion, paragraph 5, which builds on the composition of the increase:

However, a considerable portion of the observed increase in vegetation production consists of woody foliage mass, the fodder value of which is generally lower than that of herbaceous plants and would benefit primarily browsers (goats) over grazers (sheep and cattle).

What the paper's own numbers show

Method. The Results sentence defines the percentages as a "relative increase" computed from the stated 1992 and 2012 totals, so I took (2012 − 1992) / 1992 for each component. The totals are printed to two decimals, so I also computed the smallest and largest increase possible when each total varies by ±0.005 Pg. I then checked the totals against the WPF share given in the next sentence. Next I tried every other basis that could plausibly give 6% and 20%: the stated fitted trends, the per-hectare trends, other denominators, and the yearly Sahel series in the paper's Supplementary Data (file 42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha, 1992–2012). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
AGH relative increase, 1992→2012 Results p1; Abstract 6% (0.31 − 0.28) / 0.28 = 10.7%; 7.0–14.5% over rounding
WPF relative increase, 1992→2012 Results p1; Abstract 20% (0.24 − 0.16) / 0.16 = 50.0%; 42.4–58.1% over rounding
Recomputed / reported derived — Printed totals: AGH 1.79×, WPF 2.50×; over rounding AGH 1.17–2.42×, WPF 2.12–2.90×; Supplementary Data endpoints AGH 1.46×, WPF 2.33×
What the reported % would mean in Pg derived 6% of 0.28; 20% of 0.16 0.017 Pg and 0.032 Pg, against stated differences of 0.03 and 0.08 Pg
WPF share of total mass, 1992 Results p1 35% 0.16 / 0.44 = 36.4% (consistent)
WPF share of total mass, 2012 Results p1 43% 0.24 / 0.55 = 43.6% (consistent)
Summed trend × 20 years Results p1 AGH 0.0025, WPF 0.004 Pg/yr AGH 0.050 Pg = 17.9% of 0.28; WPF 0.080 Pg = 50.0% of 0.16 (18.7% / 52.5% over 21 years)
Per-hectare trend × 20 years / 1992 per-hectare mass Results p1 5 and 9 kg/ha/yr AGH 14.9%; WPF 47.1% (study area of about 4.19 million km² implied by 289,375 km² ≈ 7% and 2,711,875 km² ≈ 64%)
Pg/yr trend converted to kg/ha/yr with that area Results p1 5 and 9 kg/ha/yr 5.97 and 9.56 kg/ha/yr (roughly consistent)
Supplementary Data, endpoints MOESM4 csv — AGH 595.34 → 647.31 kg/ha (+8.7%); WPF 337.62 → 494.97 kg/ha (+46.6%)
Supplementary Data, fitted OLS line 1992→2012 MOESM4 csv slopes 5 and 9 kg/ha/yr slopes 5.41 and 8.68 kg/ha/yr; AGH +17.5%, WPF +51.8%
Supplementary Data scaled to Pg (area implied by 0.44 Pg in 1992, 4.72 × 10⁸ ha) MOESM4 csv 0.28→0.31 and 0.16→0.24 Pg; 0.0025 and 0.004 Pg/yr 0.281→0.305 and 0.159→0.233 Pg; 0.0026 and 0.0041 Pg/yr
Supplementary Data, WPF share MOESM4 csv 35% → 43% 36.2% → 43.3%

The Pg totals, the trends, the shares and the supplementary series agree with one another. Only the two percentages do not. On the paper's totals, total vegetation (AGH + WPF) rose from 0.44 to 0.55 Pg, or 25.0%. The woody-to-herbaceous contrast in relative increase is 50.0 / 10.7 = 4.7× on the printed totals (5.3× from the Supplementary Data endpoints), compared with 20 / 6 = 3.3× as reported.

Figure

Reported and recomputed Sahel-wide relative increases for herbaceous and woody foliage mass

Figure 1. Relative increase from 1992 to 2012 for each component, on one axis starting at zero. Orange dots are the percentages reported in the Abstract and Results (6% for AGH, 20% for WPF). Blue dots are the increases recomputed from the totals stated in the same sentence (10.7% and 50.0%). The blue bars show every value the increase could take if each two-decimal total is anywhere within ±0.005 Pg of its printed value. Both reported values lie outside these ranges, by 1.0 percentage point for AGH and 22.4 points for WPF.

Reported values compared with every basis tried

Figure 2. Every basis tried for the two percentages. Each panel has its own scale, starting at zero. The orange line marks the reported value. Filled blue dots show each component's change measured against its own 1992 value: from the stated totals, from the Supplementary Data endpoints and fitted line, and from the stated Pg/yr and kg/ha/yr trends over 20 years. For the kg/ha/yr row the denominator is the 1992 per-hectare mass, 0.28 or 0.16 Pg spread over the study area of about 4.19 × 10⁸ ha implied by the significant-trend areas (669 and 382 kg/ha). The closest are 8.7% for AGH and 46.6% for WPF. Hollow dots use other denominators, none of which the paper states. Only two land near both reported values. Dividing by the sum of the two years gives 5.1% and 20.0%. Dividing by the 1992 total vegetation mass gives 6.8% and 18.2%. These are discussed under "Explanations considered".

Why it matters

The 6%/20% figures are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases for the satellite analysis; the Results also give the Sahel-wide trends in kg/ha/yr and Pg/yr, which are not affected. The Abstract uses them as the quantitative evidence that "Satellite observations confirm these findings for the majority of the Sahel". A reader who cites the Abstract would take the woody foliage increase to be 20% when the paper's own totals give 50%, and the herbaceous increase to be 6% when they give about 11%.

The errors do not reverse anything. Both components still increase, and woody foliage still increases faster than herbaceous mass. With the recomputed values the contrast is larger (4.7× on the printed totals and 5.3× from the Supplementary Data endpoints, rather than 3.3×), so the paper's qualitative conclusion is, if anything, stronger. The per-hectare trends (field data: +17 vs +3 kg/ha/yr; VOD: +9 vs +5 kg/ha/yr), the 35%→43% WPF share, the rainfall decomposition and the significance tests do not depend on the two percentages and are not affected. The Discussion's statement that "a considerable portion of the observed increase in vegetation production consists of woody foliage mass" is not wrong. On the stated totals the woody share of the total increase is 0.08 of 0.11 Pg, about 73%, which the 6%/20% figures do not convey.

So this is a misstatement of magnitude in a headline number, not a change in direction. The correct values cannot be read off anywhere in the paper and have to be recomputed.

Explanations considered

No basis that the paper states, or that measures each component against its own baseline, reproduces 6% and 20%. Two unstated denominators (the 1992 total vegetation mass and the sum of the two years) can reach both values within rounding of the printed totals. Neither is described in the paper, and on the unrounded Supplementary Data the first reproduces the AGH figure but not the WPF figure, while the second reproduces neither. The published material does not show how the percentages were obtained.

Status

Suggested correction

  1. In the Results, "Scaling field-site observations to the Sahel", paragraph 1, replace "which is a relative increase of 6% for AGH and 20% for WPF" with the increases implied by the unrounded totals. The printed totals give about 11% for AGH and 50% for WPF, and the Supplementary Data series gives 8.7% and 46.6% from endpoints.
  2. Make the same change in the Abstract ("total herbaceous/woody foliage mass increases by 6%/20%").
  3. If a different basis was intended, for example the change relative to total vegetation mass, state that basis in both places and recompute both figures on it from the unrounded data.
  4. Optionally, report the increase in total vegetation (0.44 → 0.55 Pg, about 25%) and the woody share of that increase (about 73%), which make the fodder implications in the Discussion concrete.
Recomputation script (python recompute.py.txt)
"""Recompute the Sahel-wide relative increases reported in PMC6478729
(Commun Biol 2019, doi:10.1038/s42003-019-0383-9).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC (Results, "Scaling field-site
observations to the Sahel", paragraph 1) and from the Supplementary Data file
(42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha).
AGH = above-ground herbaceous mass; WPF = woody plant foliage mass.
"""
import os

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
TOT = {"AGH": (0.28, 0.31), "WPF": (0.16, 0.24)}      # Pg, 1992 and 2012 (Results p1)
REPORTED = {"AGH": 6.0, "WPF": 20.0}                   # % "relative increase" (Results p1, Abstract)
TREND_PG = {"AGH": 0.0025, "WPF": 0.004}               # Pg/yr, summed trend (Results p1)
TREND_HA = {"AGH": 5.0, "WPF": 9.0}                    # kg/ha/yr, fitted VOD trend (Results p1)
SHARE_REPORTED = (35.0, 43.0)                          # % WPF share of total, 1992 and 2012
SIG_AREA = {"AGH": (289375, 0.07), "WPF": (2711875, 0.64)}  # km2 and "~%" of area (Results p1)
HALF = 0.005                                           # totals are printed to 2 decimals

# Supplementary Data (MOESM4), Sahel VOD estimates in kg/ha, 1992..2012
YEARS = np.arange(1992, 2013)
CSV = {
    "AGH": [595.3383, 594.5558, 564.6489, 618.3494, 566.197, 449.3458, 814.1302, 983.0184,
            661.9179, 710.5017, 506.1419, 829.6328, 539.4238, 802.3855, 687.8374, 726.8941,
            722.1816, 575.7401, 921.7831, 587.7719, 647.3133],
    "WPF": [337.616, 338.2679, 338.3046, 434.1801, 373.5995, 313.1169, 301.23, 391.4529,
            495.0082, 439.4241, 429.4322, 403.0291, 458.2273, 429.9664, 464.0112, 453.4704,
            493.9232, 486.6456, 452.9526, 531.8839, 494.9722],
}
CSV = {k: np.array(v) for k, v in CSV.items()}


def pct(x):
    return f"{100 * x:.1f}%"


def grid(x):
    return np.linspace(x - HALF, x + HALF, 101)


def rng(f, *vals):
    """Min and max of f over every combination of the rounded inputs."""
    mesh = np.meshgrid(*[grid(v) for v in vals], indexing="ij")
    out = f(*mesh)
    return out.min(), out.max()


R = {}  # results reused by the figures
print("=== 1. Relative increase from the stated totals: (2012 - 1992) / 1992 ===")
for k, (a, b) in TOT.items():
    rel = (b - a) / a
    lo, hi = rng(lambda x, y: (y - x) / x, a, b)
    R[k] = dict(rel=100 * rel, lo=100 * lo, hi=100 * hi)
    print(f"  {k}: ({b} - {a}) / {a} = {pct(rel)}   reported {REPORTED[k]:.0f}%")
    print(f"       range over rounding of both totals (+/-{HALF} Pg): {pct(lo)} to {pct(hi)}")
    print(f"       ratio recomputed / reported = {100 * rel / REPORTED[k]:.2f}")
    print(f"       reported % of the 1992 total = {REPORTED[k] / 100 * a:.3f} Pg "
          f"(stated difference {b - a:.2f} Pg)")
print(f"  WPF/AGH contrast: reported {REPORTED['WPF'] / REPORTED['AGH']:.1f}x, "
      f"recomputed {R['WPF']['rel'] / R['AGH']['rel']:.1f}x")

print("\n=== 2. Consistency of the totals with the stated WPF share (35% -> 43%) ===")
s92 = TOT["WPF"][0] / (TOT["WPF"][0] + TOT["AGH"][0])
s12 = TOT["WPF"][1] / (TOT["WPF"][1] + TOT["AGH"][1])
print(f"  1992: 0.16 / 0.44 = {pct(s92)}  (reported {SHARE_REPORTED[0]:.0f}%)")
print(f"  2012: 0.24 / 0.55 = {pct(s12)}  (reported {SHARE_REPORTED[1]:.0f}%)")
print(f"  total vegetation 0.44 -> 0.55 Pg: {pct(0.55 / 0.44 - 1)}")
dw, da = TOT["WPF"][1] - TOT["WPF"][0], TOT["AGH"][1] - TOT["AGH"][0]
print(f"  woody share of the total increase: {dw:.2f} / {dw + da:.2f} Pg = {pct(dw / (dw + da))}")

print("\n=== 3. Alternative bases (none is stated in the paper) ===")
ALT = {}
for k, (a, b) in TOT.items():
    o = "WPF" if k == "AGH" else "AGH"
    oa, ob = TOT[o]
    ALT.setdefault(k, {})
    ALT[k]["Fitted trend x 20 yr / 1992 total"] = 100 * TREND_PG[k] * 20 / a
    ALT[k]["Fitted trend x 21 yr / 1992 total"] = 100 * TREND_PG[k] * 21 / a
    ALT[k]["Difference / 2012 total"] = 100 * (b - a) / b
    ALT[k]["Difference / mean of both years"] = 100 * (b - a) / ((a + b) / 2)
    ALT[k]["Difference / sum of both years"] = 100 * (b - a) / (a + b)
    ALT[k]["Difference / 1992 AGH+WPF"] = 100 * (b - a) / (a + oa)
    ALT[k]["Difference / 2012 AGH+WPF"] = 100 * (b - a) / (b + ob)
    print(f"  {k}: trend {TREND_PG[k]} Pg/yr x 20 = {TREND_PG[k] * 20:.3f} Pg "
          f"(endpoint difference {b - a:.2f} Pg)")
    for name, v in ALT[k].items():
        print(f"     {name:<36s} {v:5.1f}%")
lo, hi = rng(lambda x, y: (y - x) / (x + y), *TOT["AGH"])
lo2, hi2 = rng(lambda x, y: (y - x) / (x + y), *TOT["WPF"])
print(f"  rounding range, difference / sum: AGH {pct(lo)} to {pct(hi)}; WPF {pct(lo2)} to {pct(hi2)}")
lo, hi = rng(lambda x, y, w: (y - x) / (x + w), TOT["AGH"][0], TOT["AGH"][1], TOT["WPF"][0])
lo2, hi2 = rng(lambda w, y, x: (y - w) / (x + w), TOT["WPF"][0], TOT["WPF"][1], TOT["AGH"][0])
print(f"  rounding range, difference / 1992 AGH+WPF: AGH {pct(lo)} to {pct(hi)}; "
      f"WPF {pct(lo2)} to {pct(hi2)}")
s_agh92, s_agh12 = 1 - s92, 1 - s12
print(f"  relative change of each component's share of total: WPF {pct(s12 / s92 - 1)}, "
      f"AGH {pct(s_agh12 / s_agh92 - 1)}")

print("\n=== 4. Per-hectare trends and the implied study area ===")
areas = {k: a / p for k, (a, p) in SIG_AREA.items()}
for k, v in areas.items():
    print(f"  area implied by {k} significant area: {SIG_AREA[k][0]} / {SIG_AREA[k][1]} = {v:,.0f} km2")
area_ha = np.mean(list(areas.values())) * 100
print(f"  mean implied area = {area_ha:.3e} ha")
for k, (a, b) in TOT.items():
    per_ha = TREND_PG[k] * 1e12 / area_ha
    base = a * 1e12 / area_ha
    v = 100 * TREND_HA[k] * 20 / base
    ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"] = v
    print(f"  {k}: {TREND_PG[k]} Pg/yr -> {per_ha:.2f} kg/ha/yr (reported {TREND_HA[k]:.0f}); "
          f"1992 mass {base:.0f} kg/ha; {TREND_HA[k]:.0f} x 20 / {base:.0f} = {v:.1f}%")

print("\n=== 5. Supplementary Data (MOESM4), Sahel kg/ha series 1992-2012 ===")
tot_csv = CSV["AGH"] + CSV["WPF"]
area_csv = 0.44e12 / tot_csv[0]        # ha implied by the paper's 0.44 Pg total for 1992
print(f"  1992 AGH+WPF = {tot_csv[0]:.2f} kg/ha; area implied by 0.44 Pg = {area_csv:.3e} ha")
CSVR = {}
for k, y in CSV.items():
    o = "WPF" if k == "AGH" else "AGH"
    slope, icpt = np.polyfit(YEARS, y, 1)
    end = (y[-1] - y[0]) / y[0]
    fit0, fit1 = icpt + slope * YEARS[0], icpt + slope * YEARS[-1]
    CSVR[k] = dict(end=100 * end, trend=100 * (fit1 - fit0) / fit0)
    print(f"  {k}: {y[0]:.2f} -> {y[-1]:.2f} kg/ha; scaled {y[0] * area_csv / 1e12:.3f} -> "
          f"{y[-1] * area_csv / 1e12:.3f} Pg")
    print(f"       OLS slope {slope:.2f} kg/ha/yr = {slope * area_csv / 1e12:.4f} Pg/yr")
    print(f"       endpoint increase {pct(end)}; fitted-line increase {CSVR[k]['trend']:.1f}%")
    print(f"       difference / 1992 AGH+WPF {pct((y[-1] - y[0]) / tot_csv[0])}; "
          f"difference / sum of both years {pct((y[-1] - y[0]) / (y[-1] + y[0]))}")
print(f"  WPF share: {pct(CSV['WPF'][0] / tot_csv[0])} (1992), {pct(CSV['WPF'][-1] / tot_csv[-1])} (2012)")
for k in CSVR:
    print(f"  {k}: closest filled-dot value to reported {REPORTED[k]:.0f}% is "
          f"{min([R[k]['rel'], CSVR[k]['end'], CSVR[k]['trend'], ALT[k]['Fitted trend x 20 yr / 1992 total'], ALT[k]['Per-ha trend x 20 yr / 1992 per-ha mass']], key=lambda v: abs(v - REPORTED[k])):.1f}%")
for k in R:
    print(f"  {k}: gap between reported value and nearest end of rounding range: {R[k]['lo'] - REPORTED[k]:.1f} points")

print("\n=== 6. Understatement factor (recomputed / reported) ===")
for k in R:
    print(f"  {k}: printed totals {R[k]['rel'] / REPORTED[k]:.2f}x; over rounding "
          f"{R[k]['lo'] / REPORTED[k]:.2f}x to {R[k]['hi'] / REPORTED[k]:.2f}x; "
          f"Supplementary Data endpoints {CSVR[k]['end'] / REPORTED[k]:.2f}x")
print(f"  WPF/AGH contrast from Supplementary Data endpoints: {CSVR['WPF']['end'] / CSVR['AGH']['end']:.1f}x")

# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
    "axes.spines.top": False, "axes.spines.right": False,
})
MS = 9  # marker size in points (>= 8 px)


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: reported vs recomputed relative increase, with rounding range
fig, ax = plt.subplots(figsize=(7.2, 3.2))
rows = [("WPF", "Woody foliage (WPF)\n0.16 → 0.24 Pg"), ("AGH", "Herbaceous (AGH)\n0.28 → 0.31 Pg")]
for i, (k, lab) in enumerate(rows):
    y = i
    r = R[k]
    ax.plot([r["lo"], r["hi"]], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
    for xe in (r["lo"], r["hi"]):
        ax.plot([xe, xe], [y - 0.09, y + 0.09], color=BLUE, lw=2, zorder=2)
    ax.plot(r["rel"], y, "o", ms=MS + 1, color=BLUE, zorder=3)
    ax.plot(REPORTED[k], y, "o", ms=MS + 1, color=ORANGE, zorder=3)
    if r["lo"] - REPORTED[k] > 3:  # connector only where the gap is wide enough to draw
        ax.annotate("", xy=(r["lo"] - 0.6, y), xytext=(REPORTED[k] + 0.9, y),
                    arrowprops=dict(arrowstyle="-", color=SEC, lw=1, ls=(0, (2, 2))))
    ax.text(REPORTED[k], y - 0.2, f"reported {REPORTED[k]:.0f}%", ha="center", va="top",
            color=ORANGE, fontsize=10, fontweight="bold")
    ax.text(r["rel"], y + 0.2, f"recomputed {r['rel']:.1f}%", ha="center", va="bottom",
            color=BLUE, fontsize=10, fontweight="bold")
    ax.text(r["hi"] + 1.0, y, f"{r['lo']:.1f}–{r['hi']:.1f}%\nwith rounding", ha="left",
            va="center", color=SEC, fontsize=10, linespacing=1.1)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([lab for _, lab in rows])
ax.set_ylim(-0.6, 1.55)
ax.set_xlim(0, 70)
ax.set_xlabel("Relative increase 1992 → 2012 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[
    Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="Reported in Abstract and Results"),
    Line2D([], [], marker="o", ls="-", lw=2, ms=MS, color=BLUE,
           label="Recomputed from the stated Pg totals (bar: range over rounding)"),
], loc="upper center", bbox_to_anchor=(0.5, -0.27), frameon=False, ncol=1, fontsize=10)
fig.tight_layout()
save(fig, "fig1")

# Figure 2: every basis tried, for both components, against the reported values
order = [
    ("Stated totals: (2012 − 1992) / 1992", lambda k: R[k]["rel"], "main"),
    ("Supplementary Data: endpoints", lambda k: CSVR[k]["end"], "main"),
    ("Supplementary Data: fitted line", lambda k: CSVR[k]["trend"], "main"),
    ("Stated Pg/yr trend × 20 yr / 1992 total", lambda k: ALT[k]["Fitted trend x 20 yr / 1992 total"], "main"),
    ("Stated kg/ha/yr trend × 20 yr / 1992 kg/ha", lambda k: ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"], "main"),
    ("Difference / 2012 total", lambda k: ALT[k]["Difference / 2012 total"], "alt"),
    ("Difference / mean of both years", lambda k: ALT[k]["Difference / mean of both years"], "alt"),
    ("Difference / 1992 AGH+WPF", lambda k: ALT[k]["Difference / 1992 AGH+WPF"], "alt"),
    ("Difference / sum of both years", lambda k: ALT[k]["Difference / sum of both years"], "alt"),
]
fig, axes = plt.subplots(1, 2, figsize=(7.2, 4.4), sharey=True)
for ax, k in zip(axes, ("AGH", "WPF")):
    n = len(order)
    for i, (lab, f, kind) in enumerate(order):
        y = n - 1 - i
        v = f(k)
        face = BLUE if kind == "main" else BG
        ax.plot(v, y, "o", ms=MS, color=BLUE, mfc=face, mew=2, zorder=3)
        ax.text(v + (1.4 if k == "AGH" else 3.4), y, f"{v:.1f}", va="center", fontsize=10, color=TXT)
    ax.axvline(REPORTED[k], color=ORANGE, lw=2, zorder=2)
    ax.text(REPORTED[k] + (0.5 if k == "AGH" else 1.2), n - 0.35, f"reported {REPORTED[k]:.0f}%", color=ORANGE, ha="left",
            va="bottom", fontsize=10, fontweight="bold")
    ax.axhline(n - 5.5, color=SEC, lw=0.8, ls=(0, (3, 3)))
    ax.set_xlim(0, 24 if k == "AGH" else 60)
    ax.set_ylim(-0.6, n + 0.3)
    ax.set_title("Herbaceous (AGH)" if k == "AGH" else "Woody foliage (WPF)", fontsize=11,
                 loc="left", pad=16)
    ax.set_xlabel("Increase 1992 → 2012 (%)")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)
axes[0].set_yticks(range(len(order)))
axes[0].set_yticklabels([lab for lab, _, _ in order][::-1], fontsize=10)
fig.legend(handles=[
    Line2D([], [], color=ORANGE, lw=2, label="Reported (6% / 20%)"),
    Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mew=2,
           label="Component's change over its own 1992 value (endpoints or fitted trend)"),
    Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mfc=BG, mew=2,
           label="Other denominators (none is stated in the paper)"),
], loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)

Table 3 extrapolates VA costs with the 75% population share instead of the stated 5%

Diabetes Care (2009) · doi:10.2337/dc09-0670 · PMC2732170

Summary

The paper projects per-patient lifetime costs of Hurricane Katrina for three health care systems with the CORE Diabetes Model. It then scales them up to the diabetic adults of greater New Orleans (964,677 adults × 9.2%), with population shares stated as MCLNO 20%, VA 5% and TUHC 75%. In Table 3, the MCLNO and TUHC extrapolations reproduce exactly at their stated shares. All three VA extrapolations (direct, indirect, total), however, equal the per-patient effect times 0.75 × 964,677 × 9.2%, which is the TUHC share, not the stated 5%. The VA total is therefore 15 times too large ($260.1 million instead of $17.3 million). The population lifetime cost, the only dollar figure in the Abstract ("USD $504 million"), would be about $261.5 million (±10% range about $235–288 million) at the paper's own stated shares, not $504 million ($454–555 million).

What the paper reports

Methods, "Statistical analysis" (the extrapolation and its assumptions):

We further extrapolated the prevalence-based mean impact of an individual patient to the entire population in the greater New Orleans area using 2000 U.S. Census data.

MCLNO represented 20% of the population, VA represented 5%, and TUHC/privately insured represented 75%; the adult population affected in the greater New Orleans area was 964,677 adults; and an estimated diabetes prevalence of 9.2%

Table 3, footnotes to the "Extrapolated cost (US$)" column (they apply to every row):

Adult population (age ≥18 years): 964,677

Prevalence of diabetes in 2006 (Louisiana): 9.2%

Table 3, block headings: "MCLNO (20%)", "TUHC (75%)", "VA (5%)". Rows under the VA (5%) heading and the final row (columns: Effect | SD | P | Extrapolated cost | −10% variation | +10% variation):

Direct cost | 2,683 | 5,423 | <0.0001 | 178,587,759 | 160,728,983 | 196,446,535

Indirect cost | 1,224 | 2,850 | <0.0001 | 81,472,761 | 73,325,485 | 89,620,037

Total cost | 3,907 | 7,201 | <0.0001 | 260,060,520 | 234,054,468 | 286,066,572

Population cost | | | | 504,221,426 | 453,799,283 | 554,643,569

For comparison, the total-cost rows of the other two systems in Table 3:

Total cost | 5,243 | 8,664 | <0.0001 | 93,063,548 (MCLNO (20%) block)

Total cost | 2,270 | 6,927 | <0.0001 | 151,097,359 (TUHC (75%) block)

Abstract, "Results":

We estimate a lifetime cost of USD $504 million for the adult population affected, with the largest economic impact seen among MCLNO patients.

with the economic impact being quite substantial because of the large population size affected

Results, paragraph 3:

The largest economic impact was seen in MCLNO patients with $5,243 over their lifetime, followed by VA patients ($3,907), and privately insured TUHC patients ($2,270).

we estimated a lifetime cost of USD $504 million for the population affected by Hurricane Katrina. A 10% variation around the mean cost of each sample produced an estimate of lifetime costs of USD $454–$555 million due to the impact of Hurricane Katrina.

What the paper's own numbers show

Method. The diabetic adult population is N = 964,677 × 0.092 = 88,750.28. For each system and each cost row of Table 3, I divided the reported extrapolated cost by the per-patient effect. That gives the number of patients the effect was multiplied by, and dividing by N gives the population share actually used. I then recomputed each extrapolation at the share stated in the Methods and in the Table 3 heading, and summed the systems. I also checked that direct + indirect = total for every row, that the ±10% columns are 0.9× and 1.1× the extrapolated cost, and that the Population cost row is the sum of the three system totals. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Diabetic adults in the area Methods; Table 3 footnotes 964,677 adults; prevalence 9.2% 88,750.28
MCLNO patients implied (direct, indirect, total rows) Table 3 62,160,699 / 3,502; 30,902,849 / 1,741; 93,063,548 / 5,243 17,750.06 each = 0.2000 × N (stated 20%)
TUHC patients implied (direct, indirect, total rows) Table 3 122,475,392 / 1,840; 28,621,967 / 430; 151,097,359 / 2,270 66,562.71 each = 0.7500 × N (stated 75%)
VA patients implied (direct, indirect, total rows) Table 3 178,587,759 / 2,683; 81,472,761 / 1,224; 260,060,520 / 3,907 66,562.71 each = 0.7500 × N (stated 5%: 4,437.51)
VA direct extrapolated cost Table 3 178,587,759 11,905,851 at 5% (reported is 15.00×)
VA indirect extrapolated cost Table 3 81,472,761 5,431,517 at 5% (15.00×)
VA total extrapolated cost Table 3 260,060,520 17,337,368 at 5% (15.00×)
MCLNO and TUHC total extrapolated costs Table 3 93,063,548; 151,097,359 93,063,548; 151,097,359 (reproduce to the dollar)
Population cost Table 3; Abstract; Results 504,221,426 ("USD $504 million") Sum of the reported rows 504,221,427; unrounded with VA at 75% exactly 504,221,426.00. With VA at the stated 5%: 261,498,275
±10% range of the population cost Table 3; Results 453,799,283 – 554,643,569 ("USD $454–$555 million") 0.9× and 1.1× of the reported total, as printed. With VA at 5%: 235,348,447 – 287,648,102
Sum of population shares used Methods vs Table 3 (implied) 20% + 75% + 5% = 100% 20% + 75% + 75% = 170%: 150,875 patients extrapolated vs 88,750 diabetic adults
Ranking of population-level costs by system Table 3 VA 260.1M > TUHC 151.1M > MCLNO 93.1M With VA at 5%: TUHC 151.1M > MCLNO 93.1M > VA 17.3M
Overstatement of the population cost derived — 504,221,426 − 261,498,275 = 242,723,151; reported / recomputed = 1.928

The internal arithmetic of Table 3 is otherwise consistent: per-patient direct + indirect equals total in every system (3,502 + 1,741 = 5,243; 1,840 + 430 = 2,270; 2,683 + 1,224 = 3,907), the extrapolated direct + indirect equals the extrapolated total, and every ±10% cell is 0.9× or 1.1× its extrapolated cost (to within $1). The only departure from the stated method is the multiplier in the three VA rows, and it carries through unchanged into the Population cost row, its ±10% range, the Abstract and the Results.

Figure

Extrapolated lifetime costs by system and for the population: reported in Table 3 vs recomputed at the stated population shares

Figure 1. Extrapolated lifetime cost in US$ millions, drawn from zero on one axis. Orange bars are the values printed in Table 3. Blue bars are the per-patient effect from Table 3 multiplied by 964,677 × 9.2% × the share stated in the Methods and in the Table 3 heading (20%, 75%, 5%); the blue population bar is the sum of the three blue system bars. MCLNO and TUHC reproduce exactly. The VA row is 15 times the value at its stated 5% share, and the population cost falls from $504.2 million to $261.5 million.

Number of diabetic adults the per-patient costs are multiplied by, implied by the reported costs vs the stated shares

Figure 2. The number of diabetic adults each system's per-patient cost is multiplied by. The orange bar is implied by the reported Table 3 costs (reported extrapolated cost ÷ per-patient effect, identical for the direct, indirect and total rows). The blue bar uses the stated shares. The vertical line is the whole diabetic adult population (964,677 × 9.2% = 88,750). The reported extrapolation counts 150,875 patients, 170% of the population, because the VA block uses the same 66,563 patients as the TUHC block rather than 4,438.

Why it matters

The population lifetime cost is the only dollar figure in the Abstract ("We estimate a lifetime cost of USD $504 million for the adult population affected") and is repeated with its sensitivity range in the Results. At the paper's own stated assumptions it is overstated about 1.93-fold: about $261.5 million (±10%: about $235–288 million) instead of $504 million ($454–555 million). The whole overstatement comes from the VA block, whose printed contribution ($260.1 million) would be $17.3 million at 5%. A reader citing the paper's economic burden estimate would be using a figure roughly twice what the described method produces.

The error also changes the relative picture at population level. As printed, Table 3 makes the VA the largest contributor to population cost (VA $260.1M > TUHC $151.1M > MCLNO $93.1M). At the stated shares the VA is the smallest (TUHC $151.1M > MCLNO $93.1M > VA $17.3M). The Abstract's phrase "with the largest economic impact seen among MCLNO patients" holds per patient (the Results give $5,243 vs $3,907 vs $2,270) and is unaffected.

What is not affected: the clinical results (changes in A1C, blood pressure and lipids, Table 2, and their P values), the per-patient CORE Diabetes Model estimates of life expectancy, QALE and costs in Table 3, the disparity conclusions, and the direction of the economic finding. A modelled population cost of about $261 million is still large, so the qualitative statement that the economic impact was substantial stands; its magnitude is roughly halved.

Explanations considered

Status

Suggested correction

  1. Recompute the VA extrapolated costs in Table 3 at the stated 5% share: direct about $11,905,851 (−10%/+10%: $10,715,266 / $13,096,436), indirect about $5,431,517 ($4,888,366 / $5,974,669), total about $17,337,368 ($15,603,631 / $19,071,105).
  2. Correct the Population cost row to about $261,498,275 (−10%/+10%: $235,348,447 / $287,648,102), and the Abstract and Results figures from "USD $504 million" and "USD $454–$555 million" to about $261 million and $235–$288 million.
  3. If the 75% multiplier for the VA was in fact intended, state the VA base population and its source, and explain how the shares relate to a single diabetic population of 88,750 adults; otherwise, the shares as stated (20/75/5) should govern all three blocks.
  4. While revising Table 3, the VA QALE entry (−1.208, larger in magnitude than the VA life-expectancy change of −0.264 and unlike the other two systems, where the QALE change is smaller than the life-expectancy change) may be worth confirming against the model output. This write-up does not establish it as an error.
Recomputation script (python recompute.py.txt)
"""Recompute the population-level cost extrapolation in Table 3 of PMC2732170
(Diabetes Care 2009;32(9):1632, doi:10.2337/dc09-0670).

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC:
  Methods (Statistical analysis): population shares MCLNO 20%, VA 5%, TUHC 75%;
      964,677 adults; diabetes prevalence 9.2%
  Table 3: per-patient effects (US$) and extrapolated costs with -10%/+10% columns
  Table 3 footnotes: 964,677 adults (age >=18 y); prevalence 9.2%
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (paper)
ADULTS = 964_677
PREV = 0.092
STATED_SHARE = {"MCLNO": 0.20, "TUHC": 0.75, "VA": 0.05}   # Methods; Table 3 headings
# per-patient effect (US$): direct, indirect, total                  (Table 3, "Effect")
EFFECT = {"MCLNO": (3502, 1741, 5243), "TUHC": (1840, 430, 2270), "VA": (2683, 1224, 3907)}
# extrapolated cost (US$): direct, indirect, total     (Table 3, "Extrapolated cost")
REPORTED = {
    "MCLNO": (62_160_699, 30_902_849, 93_063_548),
    "TUHC": (122_475_392, 28_621_967, 151_097_359),
    "VA": (178_587_759, 81_472_761, 260_060_520),
}
# -10% and +10% columns                                                   (Table 3)
REPORTED_PM10 = {
    "MCLNO": ((55_944_629, 68_376_769), (27_812_564, 33_993_134), (83_757_193, 102_369_903)),
    "TUHC": ((110_227_853, 134_722_931), (25_759_770, 31_484_163), (135_987_623, 166_207_094)),
    "VA": ((160_728_983, 196_446_535), (73_325_485, 89_620_037), (234_054_468, 286_066_572)),
}
REPORTED_POP = (504_221_426, 453_799_283, 554_643_569)   # Table 3 "Population cost"
SYSTEMS = ("MCLNO", "TUHC", "VA")
PARTS = ("direct", "indirect", "total")

# ------------------------------------------------------------------ recomputation
N = ADULTS * PREV
print("=== 1. Base population ===")
print(f"diabetic adults = {ADULTS:,} x {PREV} = {N:,.2f}")
print(f"stated shares sum to {sum(STATED_SHARE.values()):.2f}")

print("\n=== 2. Patients implied by each reported extrapolated cost ===")
implied_share = {}
recomputed = {}
for s in SYSTEMS:
    sh = STATED_SHARE[s]
    print(f"\n[{s}] stated share {sh:.0%} -> {N * sh:,.1f} diabetic adults")
    recomputed[s] = []
    shares = []
    for part, eff, rep in zip(PARTS, EFFECT[s], REPORTED[s]):
        n_impl = rep / eff
        shares.append(n_impl / N)
        rec = eff * N * sh
        recomputed[s].append(rec)
        print(f"  {part:8s}: reported {rep:>13,}; {rep:,} / {eff:,} = {n_impl:,.2f} patients"
              f" = {n_impl / N:.4f} x N; at stated share: {eff:,} x {N * sh:,.2f} = {rec:,.0f}"
              f"  (reported / recomputed = {rep / rec:.2f})")
    implied_share[s] = sum(shares) / 3
    d, i, t = EFFECT[s]
    rd, ri, rt = REPORTED[s]
    print(f"  checks: {d:,} + {i:,} = {d + i:,} (table {t:,}); "
          f"{rd:,} + {ri:,} = {rd + ri:,} (table {rt:,})")

print("\n=== 3. The +/-10% columns are 0.9x and 1.1x the extrapolated cost ===")
for s in SYSTEMS:
    for part, rep, (lo, hi) in zip(PARTS, REPORTED[s], REPORTED_PM10[s]):
        print(f"  {s:5s} {part:8s}: 0.9x = {0.9 * rep:,.0f} (table {lo:,}); "
              f"1.1x = {1.1 * rep:,.0f} (table {hi:,})")

print("\n=== 4. Population totals ===")
sum_rep = sum(REPORTED[s][2] for s in SYSTEMS)
exact_rep = sum(EFFECT[s][2] * N * (0.75 if s == "VA" else STATED_SHARE[s]) for s in SYSTEMS)
print(f"sum of reported system totals = {sum_rep:,} (table {REPORTED_POP[0]:,}); "
      f"unrounded with VA at 75% = {exact_rep:,.2f}")
print(f"reported +/-10%: 0.9x = {0.9 * REPORTED_POP[0]:,.0f}, 1.1x = {1.1 * REPORTED_POP[0]:,.0f} "
      f"(table {REPORTED_POP[1]:,} - {REPORTED_POP[2]:,})")
va5 = recomputed["VA"][2]
corr = REPORTED["MCLNO"][2] + REPORTED["TUHC"][2] + va5
print(f"VA total at stated 5% = {va5:,.0f}  (VA direct {recomputed['VA'][0]:,.0f}, "
      f"indirect {recomputed['VA'][1]:,.0f})")
for part, rec in zip(PARTS, recomputed["VA"]):
    print(f"  VA {part:8s} at 5%: {rec:,.0f}; -10% {0.9 * rec:,.0f}; +10% {1.1 * rec:,.0f}")
print(f"population total with VA at 5% = 93,063,548 + 151,097,359 + {va5:,.0f} = {corr:,.0f}")
print(f"+/-10% with VA at 5%: {0.9 * corr:,.0f} - {1.1 * corr:,.0f}")
print(f"reported / corrected population total = {REPORTED_POP[0] / corr:.3f}")
print(f"overstatement = {REPORTED_POP[0] - corr:,.0f}  "
      f"(= VA reported total - VA at 5% = {REPORTED['VA'][2] - va5:,.0f})")

print("\n=== 5. Shares and patients actually used ===")
for s in SYSTEMS:
    print(f"  {s:5s}: implied share {implied_share[s]:.4f} (stated {STATED_SHARE[s]:.2f})")
used = sum(implied_share.values())
print(f"implied shares sum to {used:.4f}; patients extrapolated = {N * used:,.0f} "
      f"vs {N:,.0f} diabetic adults ({used:.2f}x)")

print("\n=== 6. Alternative readings ===")
print(f"if the VA and TUHC labels were swapped, TUHC at 5% = 2,270 x {N * 0.05:,.2f} = "
      f"{2270 * N * 0.05:,.0f} (table 151,097,359)")
print(f"VA multiplier / stated VA multiplier = 0.75 / 0.05 = {0.75 / 0.05:.0f}")

print("\n=== 7. Ranking of system totals ===")
rep_rank = sorted(SYSTEMS, key=lambda s: -REPORTED[s][2])
cor_tot = {"MCLNO": REPORTED["MCLNO"][2], "TUHC": REPORTED["TUHC"][2], "VA": va5}
cor_rank = sorted(SYSTEMS, key=lambda s: -cor_tot[s])
print("reported: " + " > ".join(f"{s} {REPORTED[s][2] / 1e6:.1f}M" for s in rep_rank))
print("VA at 5%: " + " > ".join(f"{s} {cor_tot[s] / 1e6:.1f}M" for s in cor_rank))

# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
                     "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
                     "ytick.color": INK, "text.color": INK, "svg.fonttype": "none"})


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)


# fig1: reported vs recomputed totals, US$ millions, one axis from zero
rows = [("MCLNO (20%)", REPORTED["MCLNO"][2], recomputed["MCLNO"][2]),
        ("TUHC (75%)", REPORTED["TUHC"][2], recomputed["TUHC"][2]),
        ("VA (5%)", REPORTED["VA"][2], recomputed["VA"][2]),
        ("Population cost", REPORTED_POP[0], corr)]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG)
h = 0.34
for k, (lab, rep, rec) in enumerate(rows):
    y = len(rows) - 1 - k
    ax.barh(y + h / 2 + 0.01, rep / 1e6, height=h, color=ORANGE, edgecolor=BG, linewidth=2)
    ax.barh(y - h / 2 - 0.01, rec / 1e6, height=h, color=BLUE, edgecolor=BG, linewidth=2)
    ax.text(rep / 1e6 + 6, y + h / 2, f"{rep / 1e6:,.1f}", va="center", color=INK, fontsize=10)
    ax.text(rec / 1e6 + 6, y - h / 2, f"{rec / 1e6:,.1f}", va="center", color=INK, fontsize=10)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.axhline(0.5, color=INK2, linewidth=0.8)
ax.set_xlim(0, 590)
ax.set_xlabel("Extrapolated lifetime cost, US$ millions (Table 3)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
    ax.spines[sp].set_visible(False)
from matplotlib.patches import Patch  # noqa: E402
ax.legend(handles=[Patch(color=ORANGE, label="Reported in Table 3"),
                   Patch(color=BLUE, label="Recomputed at the stated share\n(effect x 964,677 x 9.2% x share)")],
          loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
ax.text(REPORTED["VA"][2] / 1e6 + 55, 1 + h / 2 + 0.01,
        "<- computed with the 75% share,\n    not the stated 5% (15x)", va="center", color=INK2, fontsize=10)
fig.tight_layout()
save(fig, "fig1")

# fig2: diabetic adults counted in the extrapolation vs the diabetic adults available
fig, ax = plt.subplots(figsize=(7.2, 3.2), facecolor=BG)
ax.set_facecolor(BG)
bars = [("Implied by the\nreported costs", ORANGE,
         [("MCLNO", N * implied_share["MCLNO"]), ("TUHC", N * implied_share["TUHC"]),
          ("VA", N * implied_share["VA"])]),
        ("Stated shares\n20% / 75% / 5%", BLUE,
         [("MCLNO", N * 0.20), ("TUHC", N * 0.75), ("VA", N * 0.05)])]
for k, (lab, col, segs) in enumerate(bars):
    y = 1 - k
    left = 0
    for name, v in segs:
        ax.barh(y, v / 1e3, left=left / 1e3, height=0.55, color=col, edgecolor=BG, linewidth=2)
        txt = f"{name}\n{v:,.0f}"
        if v > 10_000:
            ax.text((left + v / 2) / 1e3, y, txt, ha="center", va="center", color="white", fontsize=10)
        else:
            ax.text((left + v) / 1e3 + 1.5, y + 0.43, f"{name} {v:,.0f}", ha="left", va="center",
                    color=INK, fontsize=10)
        left += v
    ax.text(left / 1e3 + 1.5, y, f"{left:,.0f}\n({left / N:.0%})", va="center", color=INK, fontsize=10)
ax.axvline(N / 1e3, color=INK2, linewidth=1)
ax.text(N / 1e3 + 1.5, 1.68, f"all diabetic adults in the area\n964,677 x 9.2% = {N:,.0f}",
        color=INK2, fontsize=10, va="center")
ax.set_yticks([1, 0])
ax.set_yticklabels([b[0] for b in bars])
ax.set_ylim(-0.5, 1.95)
ax.set_xlim(0, 175)
ax.set_xlabel("Diabetic adults the costs are multiplied by (thousands)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
    ax.spines[sp].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")

Reported p values for HbA1c and other secondary outcomes contradict their own confidence intervals

Diabetologia (2025) · doi:10.1007/s00125-025-06585-2 · PMC12779728

Summary

This 76-participant randomised trial compared video consultations with in-person care in adults with type 1 diabetes who use insulin pumps. Table 2 gives the between-group difference in HbA1c change as −2.01 mmol/mol with a 95% CI of −5.32 to 1.29 and p < 0.001, and the Abstract and the Discussion present a favourable, statistically significant HbA1c effect. That interval includes zero and implies a two-sided p of about 0.23. A p below 0.001 would need a standard error of at most 0.61, about a third (36%) of the 1.69 implied by the printed interval. The same mismatch between interval and p value appears in every other Table 2 row that has a p value, including the treatment-satisfaction and quality-of-life outcomes the Abstract calls superior and inferior. As printed, none of the secondary-outcome claims of statistical significance is supported by the paper's own intervals. The primary result for time in range, no clinically meaningful difference, is not affected.

What the paper reports

Table 2, "Primary and secondary endpoints" (columns: endpoint | video group | control group | difference (95% CI) | p value). The footer reads "Estimates are reported as least squares means ± SEs".

Change in HbA1c, mmol/mol | -4.13 ± 1.43 | -2.12 ± 1.38 | -2.01 (-5.32, 1.29) | <0.001

Change in HbA1c, % | -0.38 ± 0.13 | -0.19 ± 0.13 | -0.18 (-0.49, 0.12) |

The other Table 2 rows that have a p value:

TiR (3.9–10.0 mmol/l), % | 64.28 ± 2.91 | 63.49 ± 2.50 | 0.8 (-5.30, 6.89) | 0.251

DTQSc (range -18 to 18) | 10.23 ± 1.61 | 9.01 ± 1.42 | 1.22 (-2.51, 4.96) | 0.004

DTQSs (range 0–36) | 30.06 ± 1.25 | 28.09 ± 1.13 | 1.98 (-0.98, 4.93) | <0.001

ADDQoL19 (range -9 to +3) | -1.6 ± 0.33 | -1.33 ± 0.29 | -0.27 (-1.07, 0.53) | 0.004

Methods, "Statistical analysis":

All 95% CIs and p values are two-sided.

the key secondary outcomes were analysed in a prioritised order (i.e. ‘gatekeeping procedure’) [17].

The statistical tests were performed and interpreted in sequence until one of the analyses failed to show a statistically significant difference at a statistical significance level of 0.05

Missing data were imputed using retrieved participants from the same randomised treatment group, with results combined according to Rubin’s rule [18]. The main analyses are reported as least squares means and SEs, with differences between groups and corresponding 95% CIs adjusted for stratification and baseline outcome measurements.

Results, second paragraph. The text repeats the intervals but gives no p values for the secondary outcomes:

the observed difference of 0.8 percentage points (95% CI -5.3, 6.9; p=0.25)

the intervention group showed better treatment satisfaction compared with the controls, with a potential difference of 1.22 points for DTSQc (95% CI -2.51, 4.96) and 1.98 points for DTSQs (95% CI -0.98, 4.93)

Regarding glycaemic control, only HbA1c demonstrated a favourable difference, being lower in the intervention group than in the control group, with a mean difference of -2.01 mmol/mol (95% CI -5.32, 1.29) (-0.18%; -0.49%, 0.12%).

Results, "Sensitivity analyses and adherence to protocol":

No discrepancies were observed between the main findings and those from sensitivity analyses for primary or secondary endpoints

Abstract, Results:

For secondary outcomes, the video group was superior in terms of treatment satisfaction and reduction in HbA1c. However, the video group experienced an inferior impact on quality of life.

Abstract, Conclusions/interpretation:

However, compared with control, the intervention was associated with superior treatment satisfaction and a favourable effect on HbA1c, albeit with an inferior impact on quality of life.

Discussion, first paragraph:

However, although a statistically significant difference was found for several of the secondary outcomes

Discussion, second paragraph:

We did find a statistically significant improvement in HbA1c, albeit not necessarily clinically relevant, in the video consultation group.

Discussion, on quality of life:

Hence, one might speculate that although statistically different, the finding might not be clinically meaningful

Discussion, concluding paragraph:

However, better treatment satisfaction and a greater reduction in HbA1c were seen, although with a greater negative impact on quality of life.

What the paper's own numbers show

Method. The Methods describe a two-sided Wald-type comparison: an ANCOVA difference in least squares means with a 95% CI. For such a comparison the CI and the p value come from the same estimate and standard error, so each one fixes the other. For each Table 2 row I took the SE as the CI width divided by 2 × 1.96, computed z = difference / SE and the two-sided normal p value, and compared it with the printed p. As a second, independent check I computed the SE of the difference from the two arm SEs, assuming the arm estimates are uncorrelated, as they are in a randomised parallel-group trial. I also asked what SE and CI the printed p would need, and repeated the CI-to-p calculation for the sensitivity analyses in ESM Tables 1 and 2. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
HbA1c difference, video − control (mmol/mol) Table 2; Results −2.01 (95% CI −5.32, 1.29) LS means −4.13 − (−2.12) = −2.01; CI midpoint −2.015, so the interval is centred on the estimate
SE of the HbA1c difference, from the CI Table 2 — half-width 3.305; SE = 3.305 / 1.96 = 1.686
HbA1c two-sided p, from the CI Table 2 <0.001 z = −1.19; p = 0.233 (t with 60 or 70 df: 0.24)
HbA1c p, from the arm SEs Table 2 <0.001 SE = √(1.43² + 1.38²) = 1.987; z = −1.01; p = 0.31; 95% CI (−5.91, 1.89)
HbA1c one-sided p derived — (Methods: all p values two-sided) 0.117
SE and CI that p = 0.001 would require derived — SE ≤ 0.611, CI half-width ≤ 1.20, i.e. about (−3.21, −0.81), against a printed half-width of 3.305
HbA1c difference, % scale Table 2 −0.18 (−0.49, 0.12), p cell blank SE 0.156, p = 0.25; −2.01 / 10.929 = −0.184 (−0.487, 0.118), so the two scales agree with each other
TiR (primary) Table 2; Abstract; Results 0.8 (−5.30, 6.89), p = 0.251 SE 3.110, z = 0.26, p = 0.80; p = 0.251 would need a CI half-width of 1.37, not 6.09
DTSQc Table 2 1.22 (−2.51, 4.96), p = 0.004 SE 1.906, z = 0.64, p = 0.52
DTSQs Table 2 1.98 (−0.98, 4.93), p < 0.001 SE 1.508, z = 1.31, p = 0.19
ADDQoL19 Table 2 −0.27 (−1.07, 0.53), p = 0.004 SE 0.408, z = −0.66, p = 0.51
TbR < 3.0 mmol/l Table 2 0.05 (−0.34, 0.44), p = 0.200 p = 0.80
Sensitivity analyses with a printed p ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) ESM T1 TiR 3.69 (−3.36, 10.75), p 0.294; ESM T2 TiR −0.56 (−5.49, 4.36), p 0.819; ESM T1 DTSQc 1.30 (−4.79, 7.39), p 0.658; ESM T2 DTSQc 0.94 (−3.57, 5.45), p 0.674 in the same order: p = 0.305, 0.824, 0.676 and 0.683; these agree with their CIs to within 0.02
HbA1c in the sensitivity analyses ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) −3.42 (−7.54, 0.69); −1.31 (−4.02, 1.4); p shown as "-" p = 0.10 and 0.34; both intervals include zero

All six Table 2 rows with both a printed p and a 95% CI are inconsistent with themselves, and in each case the printed p is smaller than the interval allows. Four of the six fall below 0.05 although every one of the six intervals includes zero. By contrast, the four p values printed in the ESM sensitivity tables match their intervals.

Side observation. In four Table 2 rows without a p value, the printed difference does not equal video minus control. TbR 3.0–3.8 mmol/l: 1.70 − 1.46 = 0.24, printed −0.20. TaR > 13.9 mmol/l: −1.54, printed 0.35. TaR 10.1–13.9 mmol/l: −0.88, printed −2.83. Glycaemic variability: 36.86 − 36.86 = 0.00, printed 2.24. In each case the printed CI is centred on the printed difference, not on the difference of the LS means. These rows do not feed the claims discussed here, but they suggest the Table 2 entries as a whole need checking.

Figure

Printed p values in Table 2 against the p values implied by each row's own 95% CI

Figure 1. For the primary outcome and the four secondary outcomes that the paper's conclusions rest on, the orange circle is the printed p (a left-pointing triangle marks "<0.001", which is an upper bound, drawn at 0.001), and the blue diamond is the two-sided p recomputed from the printed CI as described above. The axis is logarithmic, and the vertical line marks p = 0.05. Every recomputed value lies above 0.05, between 0.19 and 0.80, while four of the printed values lie below it. The gap is largest for HbA1c and DTSQs, where the printed p is at least 190 to 230 times smaller than the interval implies. The sixth row with a p value, TbR < 3.0 mmol/l (printed 0.200, recomputed 0.80), is not drawn. All values are from Table 2 or from recompute.py.txt.

HbA1c difference: printed CI, CI from the arm SEs, and the CI p = 0.001 would require

Figure 2. The HbA1c difference on its own scale (mmol/mol, video − control). The orange interval is the printed 95% CI (−5.32, 1.29). The solid blue interval is the CI implied by the two arm SEs, 1.43 and 1.38 (−5.91, 1.89; p = 0.31). The dashed blue interval is hypothetical: the 95% CI that a p of 0.001 would require around the same estimate (−3.21, −0.81). Both intervals built from the paper's numbers cross zero (vertical line). Only an interval less than half as wide as the printed one, and entirely below zero, would be consistent with p < 0.001.

Why it matters

The trial's primary outcome, time in range, showed no meaningful difference, and that conclusion is unaffected: the CI (−5.30, 6.89) and the stated decision rule both point the same way, whichever p value is right. The paper's positive findings all come from the secondary outcomes. The Abstract reports that the video group "was superior in terms of treatment satisfaction and reduction in HbA1c" and had "an inferior impact on quality of life", and the Abstract conclusion and several Discussion paragraphs, including the concluding one, repeat these findings. The Discussion explicitly calls the HbA1c improvement "statistically significant" and the quality-of-life difference "statistically different". Those characterisations match the printed p values (<0.001, <0.001, 0.004, 0.004) but not the printed intervals, all of which include zero and imply p between 0.19 and 0.52. On the intervals, none of the secondary differences is statistically significant at 0.05. That includes the % HbA1c row, which has no p value but whose CI (−0.49, 0.12) gives p ≈ 0.25.

The paper's own gatekeeping rule points the same way. The Methods say the key secondary outcomes were analysed in a prioritised order, "listed in order under 'Outcomes and data collection'", and that the tests were interpreted in sequence until one failed at 0.05. The first outcome listed there is DTSQc, whose interval (−2.51, 4.96) includes zero and implies p = 0.52. On the intervals, the sequence therefore stops at its first step, and none of the later secondary outcomes (DTSQs, ADDQoL19, HbA1c) would be interpreted as confirmatory. If, as is usual, the sequence starts with the primary outcome, it would stop even earlier, since time in range was not significant either (printed p = 0.251; p from the CI 0.80).

Readers of the Abstract are therefore told of a favourable HbA1c effect and of superior treatment satisfaction that the reported data do not show to be different from no effect. A meta-analysis that took the HbA1c result from the text or CI would treat it as null. A reader or guideline that relied on the Abstract would treat it as positive.

Explanations considered

None of these explanations reconciles the printed p values with the printed intervals.

Status

As of 2026-10-23:

Sources: PMC12779728 (PubMed Central archive, read with src/show.py); https://doi.org/10.1007/s00125-025-06585-2 and the publisher's Table 2 page; ESM file https://static-content.springer.com/esm/art%3A10.1007%2Fs00125-025-06585-2/MediaObjects/125_2025_6585_MOESM1_ESM.pdf; Europe PMC full-text XML; https://api.crossref.org/works?filter=updates:10.1007/s00125-025-06585-2; ClinicalTrials.gov NCT04612933 (registry entry, not used for any number here).

Suggested correction

  1. Re-derive the between-group p values for every Table 2 row from the same ANCOVA and multiple-imputation model that produced the differences and CIs, and correct the p column. Add the p value for the % HbA1c row, or state that it is the same test as the mmol/mol row.
  2. Revise the statements that depend on statistical significance of the secondary outcomes so that they match the corrected p values and the intervals. These are the Abstract Results and Conclusions/interpretation, and the first, second, quality-of-life and concluding paragraphs of the Discussion. If the intervals are correct, these differences should be described as not statistically significant, with the CI given.
  3. State how the gatekeeping procedure was applied: whether the sequence starts with the primary outcome, and how the secondary outcomes were interpreted given that the first listed key secondary outcome, DTSQc, has an interval that includes zero (−2.51, 4.96).
  4. Check the difference column for TbR 3.0–3.8 mmol/l, both TaR rows and glycaemic variability, where the printed difference does not equal video minus control.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 p values of PMC12779728
(Diabetologia 2025, doi:10.1007/s00125-025-06585-2).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs (cand_A ... cand_E) and a contact sheet in ./candidates/.

Inputs are copied from the paper as archived in PubMed Central (identical in the
publisher's version of record) and from the single ESM file (ESM Tables 1 and 2).
Table 2 reports least squares means +- SE for each arm, the between-group difference
with its 95% CI, and a p value. Methods: "All 95% CIs and p values are two-sided."
"""
import os
import re

import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (Table 2)
# name, LS mean video, SE, LS mean control, SE, difference, CI low, CI high, printed p
TABLE2 = [
    ("TiR (primary), %",        64.28, 2.91, 63.49, 2.50,  0.80, -5.30, 6.89, "0.251"),
    ("DTSQc",                   10.23, 1.61,  9.01, 1.42,  1.22, -2.51, 4.96, "0.004"),
    ("DTSQs",                   30.06, 1.25, 28.09, 1.13,  1.98, -0.98, 4.93, "<0.001"),
    ("ADDQoL19",                -1.60, 0.33, -1.33, 0.29, -0.27, -1.07, 0.53, "0.004"),
    ("Change in HbA1c, mmol/mol", -4.13, 1.43, -2.12, 1.38, -2.01, -5.32, 1.29, "<0.001"),
    ("Change in HbA1c, %",      -0.38, 0.13, -0.19, 0.13, -0.18, -0.49, 0.12, ""),
    ("TbR <3.0, %",              0.44, 0.18,  0.39, 0.16,  0.05, -0.34, 0.44, "0.200"),
    ("TbR 3.0-3.8, %",           1.70, 0.24,  1.46, 0.21, -0.20, -0.71, 0.31, "-"),
    ("TaR >13.9, %",             8.92, 2.29, 10.46, 1.87,  0.35, -3.04, 3.75, "-"),
    ("TaR 10.1-13.9, %",        24.30, 2.31, 25.18, 1.86, -2.83, -6.40, 0.74, "-"),
    ("Glycaemic variability, %", 36.86, 1.70, 36.86, 1.44,  2.24, -0.43, 4.90, "-"),
]
# ESM Tables 1 (per protocol) and 2 (ITT, baseline observation carried forward):
# rows that print a p value, plus the HbA1c rows (p printed as "-").
ESM = [
    ("ESM T1 per protocol, TiR",          3.69, -3.36, 10.75, "0.294"),
    ("ESM T1 per protocol, DTSQc",        1.30, -4.79,  7.39, "0.658"),
    ("ESM T2 BOCF, TiR",                 -0.56, -5.49,  4.36, "0.819"),
    ("ESM T2 BOCF, DTSQc",                0.94, -3.57,  5.45, "0.674"),
    ("ESM T1 per protocol, HbA1c mmol/mol", -3.42, -7.54, 0.69, "-"),
    ("ESM T2 BOCF, HbA1c mmol/mol",      -1.31, -4.02,  1.40, "-"),
]
N_RAND, N_COMPLETE = 76, 32 + 31          # Abstract: 76 randomised, 32 + 31 completed
IFCC_PER_NGSP = 10.929                      # mmol/mol per 1 percentage point of HbA1c


def se_from_ci(lo, hi):
    return (hi - lo) / (2 * Z975)


def p_two(z):
    return 2 * stats.norm.sf(abs(z))


def z_from_p(p):
    return stats.norm.isf(p / 2)


def p_value_of(printed):
    return float(printed.lstrip("<"))


# ---------------------------------------------------------------- 1. every Table 2 row
print("1. Table 2: p value implied by each row's own 95% CI and by its arm SEs (normal)")
print(f"{'endpoint':27s} {'V-C':>6s} {'diff':>6s} {'CImid':>6s} {'SE_CI':>6s} {'z_CI':>6s}"
      f" {'p_CI':>6s} {'SE_arm':>6s} {'p_arm':>6s} {'p_printed':>9s}")
REC = {}
for n, mv, sv, mc, sc, d, lo, hi, p in TABLE2:
    se_ci = se_from_ci(lo, hi)
    z_ci = d / se_ci
    se_arm = float(np.hypot(sv, sc))
    p_arm = p_two((mv - mc) / se_arm)
    REC[n] = dict(d=d, lo=lo, hi=hi, se=se_ci, z=z_ci, p=p_two(z_ci), se_arm=se_arm,
                  p_arm=p_arm, lsdiff=mv - mc, printed=p)
    print(f"{n:27s} {mv - mc:6.2f} {d:6.2f} {(lo + hi) / 2:6.2f} {se_ci:6.3f} {z_ci:6.2f}"
          f" {p_two(z_ci):6.3f} {se_arm:6.3f} {p_arm:6.3f} {p:>9s}")

TESTED = ["TiR (primary), %", "DTSQc", "DTSQs", "ADDQoL19", "Change in HbA1c, mmol/mol"]
print("\n   |z| implied by the printed p (largest p allowed by '<'):")
for n in TESTED:
    r = REC[n]
    zp = z_from_p(p_value_of(r["printed"]))
    print(f"   {n:27s} printed p {r['printed']:>6s} -> |z| {'>=' if r['printed'].startswith('<') else '='}"
          f" {zp:.2f};  CI gives |z| = {abs(r['z']):.2f};  SE the printed p needs = {abs(r['d']) / zp:.3f}"
          f" (CI half-width {abs(r['d']) / zp * Z975:.2f}, printed {(r['hi'] - r['lo']) / 2:.2f})")

# ---------------------------------------------------------------- 2. HbA1c in detail
h = REC["Change in HbA1c, mmol/mol"]
d, lo, hi = h["d"], h["lo"], h["hi"]
print("\n2. HbA1c, mmol/mol")
print(f"   LS means: -4.13 - (-2.12) = {-4.13 - -2.12:.2f}; CI midpoint {(lo + hi) / 2:.3f}")
print(f"   CI half-width {(hi - lo) / 2:.3f}; SE {h['se']:.3f}; z {h['z']:.3f}; "
      f"two-sided p {h['p']:.3f}; one-sided p {h['p'] / 2:.3f}")
for df in (60, 70):
    print(f"   t with df {df}: two-sided p {2 * stats.t.sf(abs(h['z']), df):.3f}")
print(f"   SE from arm SEs sqrt(1.43^2+1.38^2) = {h['se_arm']:.3f}; z {d / h['se_arm']:.3f}; p {h['p_arm']:.3f};"
      f" 95% CI ({d - Z975 * h['se_arm']:.2f}, {d + Z975 * h['se_arm']:.2f})")
z001 = z_from_p(0.001)
se_need = abs(d) / z001
HB_NEED = (d - Z975 * se_need, d + Z975 * se_need)
print(f"   p = 0.001 needs |z| {z001:.3f}, SE {se_need:.3f}, CI half-width {se_need * Z975:.2f},"
      f" i.e. 95% CI ({HB_NEED[0]:.2f}, {HB_NEED[1]:.2f})")
print(f"   SE ratio printed CI / needed: {h['se'] / se_need:.2f}; needed / printed {se_need / h['se']:.3f}"
      f" ({100 * se_need / h['se']:.0f}%); variance ratio {(h['se'] / se_need) ** 2:.1f}")
rho = (1.43 ** 2 + 1.38 ** 2 - se_need ** 2) / (2 * 1.43 * 1.38)
print(f"   correlation between the two LS means needed to shrink the arm-SE difference SE to {se_need:.3f}: {rho:.2f}")
miss = 1 - N_COMPLETE / N_RAND
print(f"   missing outcome share {N_RAND - N_COMPLETE}/{N_RAND} = {miss:.3f}; share of the CI's variance that "
      f"would have to come from imputation alone for p<0.001: {1 - (se_need / h['se']) ** 2:.3f}")
print(f"   within-group z (LS mean / SE): video {-4.13 / 1.43:.2f}, p {p_two(-4.13 / 1.43):.4f};"
      f" control {-2.12 / 1.38:.2f}, p {p_two(-2.12 / 1.38):.3f}")
hp = REC["Change in HbA1c, %"]
print(f"   % row: SE {hp['se']:.3f}, z {hp['z']:.2f}, p {hp['p']:.3f}")
print(f"   mmol/mol -> % (/ {IFCC_PER_NGSP}): diff {d / IFCC_PER_NGSP:.3f}, CI ({lo / IFCC_PER_NGSP:.3f},"
      f" {hi / IFCC_PER_NGSP:.3f}); arms {-4.13 / IFCC_PER_NGSP:.3f}, {-2.12 / IFCC_PER_NGSP:.3f}")

# ---------------------------------------------------------------- 3. ESM rows
print("\n3. ESM sensitivity tables: p implied by the printed CI")
for n, dd, l, u, p in ESM:
    se = se_from_ci(l, u)
    print(f"   {n:38s} {dd:6.2f} ({l}, {u})  SE {se:.3f}  p_CI {p_two(dd / se):.3f}  printed {p}")

# ---------------------------------------------------------------- 4. side observation
print("\n4. Printed differences that do not equal video minus control LS means")
for n in ["TbR 3.0-3.8, %", "TaR >13.9, %", "TaR 10.1-13.9, %", "Glycaemic variability, %"]:
    r = REC[n]
    print(f"   {n:27s} video - control = {r['lsdiff']:6.2f}; printed difference {r['d']:6.2f}"
          f" (CI midpoint {(r['lo'] + r['hi']) / 2:.2f})")

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
})
LABELS = {"TiR (primary), %": "TiR, % (primary)", "DTSQc": "DTSQc", "DTSQs": "DTSQs",
          "ADDQoL19": "ADDQoL19", "Change in HbA1c, mmol/mol": "HbA1c change, mmol/mol"}
MS = 9  # marker size in points (>= 8 px)


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight")
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


def fmt_p(p):
    return f"{p:.2f}" if p >= 0.01 else f"{p:.3f}"


def mn(text):
    """Typographic minus signs for numbers drawn in figures."""
    return re.sub(r"(?<![A-Za-z])-(?=\d)", "\u2212", text)


MASK = dict(boxstyle="square,pad=0.1", fc=BG, ec="none")


def design_A(paths):
    """Dot-and-interval (dumbbell) on one log p axis, five tested rows."""
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ys = np.arange(len(TESTED))[::-1]
    for y, n in zip(ys, TESTED):
        r = REC[n]
        pr = p_value_of(r["printed"])
        ax.plot([pr, r["p"]], [y, y], color=INK2, lw=1, alpha=0.5, zorder=1)
        mk = "<" if r["printed"].startswith("<") else "o"
        ax.plot(pr, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.plot(r["p"], y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.annotate(f"printed {r['printed']}", (pr, y), xytext=(-8, 9), textcoords="offset points",
                    ha="right", fontsize=10, color=INK, bbox=MASK, zorder=4)
        ax.annotate(f"from CI {fmt_p(r['p'])}", (r["p"], y), xytext=(8, 9), textcoords="offset points",
                    ha="left", fontsize=10, color=INK, bbox=MASK, zorder=4)
    ax.axvline(0.05, color=INK2, lw=1, zorder=0)
    ax.text(0.05, len(TESTED) - 0.35, "p = 0.05", ha="center", va="bottom", fontsize=10, color=INK2, bbox=MASK)
    ax.set_xscale("log")
    ax.set_xlim(3e-4, 3)
    ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
    ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
    ax.set_yticks(ys)
    ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
    ax.set_ylim(-0.6, len(TESTED) - 0.1)
    ax.set_xlabel("Two-sided p value (log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="p printed in Table 2 ('<' = upper bound)")
    h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="p recomputed from the row's own 95% CI")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
    ax.set_title("Table 2: printed p values against the p values their 95% CIs imply",
                 loc="left", fontsize=11.5, color=INK)
    save(fig, *paths)


def design_B(paths):
    """HbA1c only: printed CI vs CI from arm SEs vs the CI that p<0.001 would need."""
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    rows = [
        ("Printed in Table 2", d, lo, hi, ORANGE, "-", "o", f"printed p <0.001; this CI implies p = {fmt_p(h['p'])}"),
        ("From the arm SEs\n(1.43 and 1.38)", d, d - Z975 * h["se_arm"], d + Z975 * h["se_arm"], BLUE, "-", "D",
         f"p = {fmt_p(h['p_arm'])}"),
        ("Needed for p = 0.001", d, HB_NEED[0], HB_NEED[1], BLUE, (0, (4, 2)), "D", "hypothetical: half-width 1.20"),
    ]
    for i, (lab, c, l, u, col, ls, mk, note) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.plot([l, u], [y, y], color=col, lw=2, ls=ls, solid_capstyle="round")
        mfc = col if i < 2 else BG
        ax.plot(c, y, marker=mk, ms=MS, color=col, mfc=mfc, mec=col, mew=2, ls="none")
        ax.text(u + 0.15, y, mn(f"({l:.2f}, {u:.2f})"), va="center", fontsize=10, color=INK, bbox=MASK, zorder=4)
        ax.text(l, y + 0.2, note, ha="left", va="bottom", fontsize=10, color=INK2, bbox=MASK, zorder=4)
    ax.axvline(0, color=INK2, lw=0.8)
    ax.text(0.05, -0.55, "no difference", fontsize=10, color=INK2, va="bottom")
    ax.set_yticks(range(len(rows))[::-1])
    ax.set_yticklabels([r[0] for r in rows], color=INK)
    ax.set_ylim(-0.6, len(rows) - 0.2)
    ax.set_xlim(-7, 4.2)
    ax.set_xlabel("Difference in HbA1c change, video − control (mmol/mol), 95% CI")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in the paper")
    h2 = plt.Line2D([], [], marker="D", color=BLUE, lw=2, ms=MS - 1, label="recomputed from the paper's numbers")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
    ax.set_title("HbA1c difference −2.01 mmol/mol: the printed CI crosses zero", loc="left", fontsize=11.5)
    save(fig, *paths)


def design_C(paths):
    """Same five rows on a linear |z| axis: z the printed p requires vs z the CI gives."""
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ys = np.arange(len(TESTED))[::-1]
    for y, n in zip(ys, TESTED):
        r = REC[n]
        zp = z_from_p(p_value_of(r["printed"]))
        zc = abs(r["z"])
        ax.plot([zc, zp], [y, y], color=INK2, lw=1, alpha=0.5)
        mk = ">" if r["printed"].startswith("<") else "o"
        ax.plot(zp, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.plot(zc, y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.annotate(f"{'≥' if mk == '>' else ''}{zp:.2f}", (zp, y), xytext=(0, 9), textcoords="offset points",
                    ha="center", fontsize=10)
        ax.annotate(f"{zc:.2f}", (zc, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10)
    ax.axvline(Z975, color=INK2, lw=1)
    ax.text(Z975, len(TESTED) - 0.35, "|z| = 1.96 (p = 0.05)", ha="center", va="bottom", fontsize=10, color=INK2,
            bbox=MASK)
    ax.set_xlim(0, 4)
    ax.set_yticks(ys)
    ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
    ax.set_ylim(-0.6, len(TESTED) - 0.1)
    ax.set_xlabel("|z| = |difference| / SE")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="|z| the printed p requires ('>' = at least)")
    h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="|z| the printed 95% CI gives")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
    ax.set_title("Test statistic implied by each printed p value and by its CI", loc="left", fontsize=11.5)
    save(fig, *paths)


def design_D(paths):
    """Graphic table: rows with the printed CI drawn in SE units, printed p and CI-implied p."""
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    ys = np.arange(len(TESTED))[::-1]
    for y, n in zip(ys, TESTED):
        r = REC[n]
        zc = r["z"]
        ax.plot([zc - Z975, zc + Z975], [y, y], color=ORANGE, lw=2, solid_capstyle="round")
        ax.plot(zc, y, "o", ms=MS, color=ORANGE, mec=BG, mew=1.5)
        ax.plot(5.2, y, "o", ms=MS, color=ORANGE, mec=BG)
        ax.text(5.6, y, r["printed"], ha="left", va="center", fontsize=10.5, color=INK)
        ax.plot(8.0, y, "D", ms=MS - 1, color=BLUE, mec=BG)
        ax.text(8.4, y, fmt_p(r["p"]), ha="left", va="center", fontsize=10.5, color=INK, weight="bold")
    ax.axvline(0, color=INK2, lw=1)
    top = len(TESTED) - 0.45
    for x, t in [(0, "95% CI in SE units"), (5.9, "Printed p"), (8.6, "p from CI")]:
        ax.text(x, top, t, ha="center", va="bottom", fontsize=10, color=INK2, weight="bold", bbox=MASK)
    ax.set_xlim(-3.6, 10.0)
    ax.set_ylim(-0.6, len(TESTED) - 0.05)
    ax.set_yticks(ys)
    ax.set_yticklabels([mn(f"{LABELS[n]}\n{r['d']:.2f} ({r['lo']:.2f}, {r['hi']:.2f})")
                        for n, r in ((n, REC[n]) for n in TESTED)], color=INK, fontsize=10)
    ax.set_xticks([-2, 0, 2])
    ax.set_xticklabels(["−2", "0", "+2"])
    ax.tick_params(axis="y", length=0)
    for s in ("left", "bottom"):
        ax.spines[s].set_visible(False)
    ax.set_xlabel("CI drawn as (difference ÷ SE) ± 1.96; every interval crosses 0", fontsize=10, color=INK2)
    h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in Table 2")
    h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="recomputed from the printed CI")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
    save(fig, *paths)


def design_E(paths):
    """Calculation flow from the printed HbA1c CI to its p value, set against the printed p and the claims."""
    fig, ax = plt.subplots(figsize=(7.2, 4.4))
    ax.set_xlim(0, 10)
    ax.set_ylim(-0.1, 6.2)
    ax.axis("off")

    def box(x, y, w, hgt, text, edge, bold=False, fs=10.5):
        ax.add_patch(FancyBboxPatch((x, y), w, hgt, boxstyle="round,pad=0.02,rounding_size=0.12",
                                    fc=BG, ec=edge, lw=2))
        ax.text(x + w / 2, y + hgt / 2, text, ha="center", va="center", fontsize=fs, color=INK,
                weight="bold" if bold else "normal", wrap=True)

    def arrow(x0, y0, x1, y1, col=INK2):
        ax.annotate("", (x1, y1), (x0, y0), arrowprops=dict(arrowstyle="-|>", color=col, lw=1.5))

    box(0.1, 4.7, 3.1, 1.1, "Table 2, printed\n−2.01 (−5.32, 1.29)", ORANGE, bold=True)
    box(3.6, 4.7, 2.9, 1.1, f"CI half-width\n(1.29 + 5.32) / 2\n= {(hi - lo) / 2:.3f}", BLUE, fs=10)
    box(6.9, 4.7, 3.0, 1.1, f"SE = {(hi - lo) / 2:.3f} / 1.96\n= {h['se']:.3f}", BLUE)
    box(6.9, 3.0, 3.0, 1.1, mn(f"z = −2.01 / {h['se']:.3f}\n= {h['z']:.2f}"), BLUE)
    box(3.6, 3.0, 2.9, 1.1, f"two-sided p\n= {h['p']:.2f}", BLUE, bold=True)
    box(0.1, 3.0, 3.1, 1.1, "Table 2, printed\np < 0.001", ORANGE, bold=True)
    arrow(3.2, 5.25, 3.6, 5.25)
    arrow(6.5, 5.25, 6.9, 5.25)
    arrow(8.4, 4.7, 8.4, 4.1)
    arrow(6.9, 3.55, 6.5, 3.55)
    ax.text(3.4, 3.55, "≠", ha="center", va="center", fontsize=20, color=INK, weight="bold")
    ax.text(1.6, 2.8, f"conflict: p < 0.001 needs SE ≤ {se_need:.2f},\n"
            f"i.e. CI half-width ≤ {se_need * Z975:.2f} (printed {(hi - lo) / 2:.3f})",
            ha="left", va="top", fontsize=10, color=INK)
    box(0.1, 0.0, 9.8, 1.75,
        "Claims that rest on the printed p\n"
        "Abstract: “the video group was superior in terms of treatment satisfaction\nand reduction in HbA1c”\n"
        "Discussion: “We did find a statistically significant improvement in HbA1c”", INK2, fs=10)
    arrow(1.0, 3.0, 1.0, 1.7, ORANGE)
    h1 = plt.Line2D([], [], color=ORANGE, lw=2, label="printed in the paper")
    h2 = plt.Line2D([], [], color=BLUE, lw=2, label="recomputed from the paper's numbers")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, 1.06), ncol=2, frameon=False, fontsize=10)
    save(fig, *paths)


design_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_pvalue_dumbbell")])
design_B([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_B_hba1c_intervals")])
design_C([os.path.join(CAND, "cand_C_z_dumbbell")])
design_D([os.path.join(CAND, "cand_D_graphic_table")])
design_E([os.path.join(CAND, "cand_E_calculation_flow")])


def contact_sheet(path):
    """Five candidate designs side by side, labelled A-E, for choosing one."""
    names = ["cand_A_pvalue_dumbbell", "cand_B_hba1c_intervals", "cand_C_z_dumbbell",
             "cand_D_graphic_table", "cand_E_calculation_flow"]
    titles = ["A  p values, log axis (fig1)", "B  HbA1c intervals (fig2)", "C  |z| statistic, linear axis",
              "D  graphic table, CI in SE units", "E  calculation flow, HbA1c"]
    fig, axs = plt.subplots(3, 2, figsize=(14, 15))
    for ax in axs.flat:
        ax.axis("off")
    for ax, n, t in zip(axs.flat, names, titles):
        ax.imshow(plt.imread(os.path.join(CAND, n + ".png")))
        ax.set_title(t, loc="left", fontsize=13, weight="bold", color=INK)
    fig.tight_layout()
    fig.savefig(path, dpi=110, facecolor=BG)
    plt.close(fig)


contact_sheet(os.path.join(CAND, "contact_sheet.png"))
print("\nFigures written: fig1, fig2, candidates/cand_A ... cand_E (.svg and .png) and candidates/contact_sheet.png")

Abstract reports p<0.001 for the matched cystic fibrosis versus rheumatoid arthritis comparison; Table 4 gives p=0.055

Eur Respir J (2023) · doi:10.1183/13993003.00174-2023 · PMC10600351

Summary

The abstract reports the propensity-matched comparison of major adverse cardiac events (MACE) between people with cystic fibrosis (CF) and people with rheumatoid arthritis (RA) as HR 1.21 (95% CI 1.00–1.48) with p<0.001. Table 4 gives the same HR and CI for this comparison, with p=0.055. Back-calculating from the printed HR and CI gives p = 0.057, or 0.047–0.068 if the rounded values vary within their rounding ranges. That agrees with Table 4 and rules out p<0.001, which would need a CI of about 1.08–1.36. The only p<0.001 in the RA row of Table 4 belongs to the unadjusted analysis (HR 0.75), so the abstract presents a borderline, non-significant comparison as highly significant. The Results and Discussion also describe CF risk as "greater than" RA risk. The paper's main finding, a higher MACE risk in CF than in matched general-population comparators (HR 1.65, p<0.001), is not affected.

What the paper reports

Abstract, Results:

In propensity score-matched analyses, PwCF had higher risk of MACE than matched general population comparators (hazard ratio (HR) 1.65, 95% CI 1.40–1.95; p<0.001) and an equivalent or higher relative risk compared with other inflammatory conditions considered “high risk” for cardiovascular disease, including systemic lupus erythematosus (HR 0.95, 95% CI 0.82–1.09; p=0.44), rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001) and HIV (HR 0.93, 95% CI 0.82–1.06; p=0.29).

Table 4, "Rheumatoid arthritis" row. The columns are: unadjusted analysis (CF | events | non-CF | events | HR (95% CI) | p-value), then propensity score-matched analysis (the same six columns):

Rheumatoid arthritis | 6265 | 227 | 97 198 | 4547 | 0.75 (0.66–0.86) | <0.001 | 5901 | 223 | 5901 | 177 | 1.21 (1.00–1.48) | 0.055

Table 4 footer:

Propensity score matching based on known cardiac risk factors (age, sex, diabetes, hypertension, hypercholesterolaemia, family history of heart disease, smoking and chronic kidney disease).

Methods, "Statistical analysis":

After propensity score matching, a Cox's proportional hazards model was used to compare the matched cohorts.

Results, "Outcomes compared with other inflammatory diseases":

In CF, risk of MACE was equivalent to SLE and HIV (HR 0.95, 95% CI 0.82–1.09; p=0.44; and HR 0.93, 95% CI 0.82–1.06; p=0.29, respectively) and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48).

Discussion, first paragraph:

Furthermore, cardiac risk in PwCF was equivalent to or higher than other chronic inflammatory diseases considered “high risk” for cardiac disease.

Discussion, fifth paragraph (the paragraph beginning "Chronic inflammation is a hallmark of CF"):

Here, we found that despite adjusting for the presence of classic cardiac risk factors, risk of cardiac disease in CF was equivalent to or greater than that of other inflammatory conditions.

What the paper's own numbers show

Method. Hazard ratios from a Cox model are normally reported with a Wald 95% CI on the log scale. For such a CI, SE(log HR) = (ln upper − ln lower) / (2 × 1.96), z = ln HR / SE, and the two-sided p = 2(1 − Φ(|z|)). This cannot reproduce the exact Cox p-value without the individual event times, but it is the standard check that a printed HR, CI and p-value are consistent. I applied it to the four matched comparisons the abstract reports. To allow for rounding, I also varied the RA HR and both bounds over every value that rounds to the printed two decimals. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
CF vs RA, matched: p-value Abstract, Results HR 1.21 (1.00–1.48); p<0.001 SE(log HR) 0.100, z = 1.91, p = 0.057
CF vs RA, matched: p-value Table 4 HR 1.21 (1.00–1.48); p = 0.055 p = 0.057 (agrees)
CF vs RA, matched: p over rounding of HR and CI Table 4 / Abstract — 0.047–0.068
CF vs RA, matched: HR against CI midpoint Table 4 / Abstract 1.21 geometric midpoint √(1.00 × 1.48) = 1.217 (consistent)
CF vs RA, matched: what p<0.001 would need — — |z| > 3.29, SE ≤ 0.058, so a 95% CI no wider than 1.08–1.36
CF vs RA, matched: one-sided p — — 0.028
CF vs RA, unadjusted Table 4 HR 0.75 (0.66–0.86); p<0.001 z = −4.26, p = 2.0 × 10⁻⁵ (agrees); the only p<0.001 in the RA row
CF vs RA, matched: crude event counts Table 4 CF 223/5901 vs RA 177/5901 3.78% vs 3.00%; crude risk ratio 1.26 (1.04–1.53), p = 0.020 (ignores time to event and censoring, so it does not replace the Cox p)
CF vs SLE, matched (cross-check) Abstract / Table 4 HR 0.95 (0.82–1.09); p = 0.44 / 0.439 p = 0.48 (rough agreement, within rounding of the CI)
CF vs HIV, matched (cross-check) Abstract / Table 4 HR 0.93 (0.82–1.06); p = 0.29 / 0.293 p = 0.27 (rough agreement)
CF vs general population, matched (cross-check) Abstract / Table 3 HR 1.65 (1.40–1.95); p<0.001 z = 5.92, p = 3 × 10⁻⁹ (agrees)

The same back-calculation agrees with the printed p-values for the other three matched comparisons. For RA it agrees with Table 4 but not with the abstract. A CI whose lower bound is 1.00 is by construction close to p = 0.05, and a two-sided p<0.001 would require the lower bound to be at least about 1.08.

Figure

Reported and recomputed p-values for the four propensity-matched MACE comparisons, on a log p-value axis

Figure 1. P-values for the four propensity-matched MACE comparisons named in the abstract, on one log-scale axis. Orange diamonds are the values printed in the abstract and orange squares the values printed in Table 3 (general population) or Table 4 (SLE, RA, HIV). A left-pointing arrow marks a value printed as "p<". Blue circles are the p-values recomputed from the printed HR and 95% CI. For RA, the blue bar spans the p-values obtained when the HR and both CI bounds vary within their rounding ranges (0.047–0.068). The dashed line marks p = 0.05 and the dotted line p = 0.001. For the general population, SLE and HIV the three values agree. For RA, Table 4 and the recomputed value sit on the 0.05 line, while the abstract places the result beyond 0.001. The recomputed general-population p (3 × 10⁻⁹) lies beyond the left edge of the axis.

Diagram showing that the abstract pairs the matched RA HR and CI with the p-value from the unadjusted column of Table 4

Figure 2. Where the abstract's RA values occur in Table 4. The abstract's HR and CI (1.21, 1.00–1.48) are those of the propensity-matched columns, whose p-value is 0.055. The only p<0.001 in the RA row is in the unadjusted columns, for an HR of 0.75 (0.66–0.86) that points the other way. The figure shows where the printed values coincide. It does not show how the abstract's value arose.

Why it matters

Explanations considered

None of these accounts for p<0.001. The Table 4 value of 0.055 agrees with the recomputation, so the abstract's value is the one in error.

A side observation, not part of this finding: the Table 4 SLE matched row lists the CF group as n = 223 with 385 events and the comparator as n = 5223. The Results text states that 5223 PwCF were matched with SLE comparators, so the "223" looks like a dropped digit. This was not examined further.

Status

As of 2026-10-23:

Suggested correction

In the Abstract, Results, replace "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)" with "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)", in line with Table 4 and Supplementary Table S4.

Because this comparison does not reach p<0.05, the authors may also wish to revise the wording that reads it as an established difference. In the Results, "and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48)" could read "and was numerically higher than, but not significantly different from, rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)". The phrases "equivalent or higher" (Abstract), "equivalent to or higher than" (Discussion, first paragraph) and "equivalent to or greater than" (Discussion, fifth paragraph) could be qualified in the same way.

Recomputation script (python recompute.py.txt)
"""Recompute the p-values of the propensity-matched MACE comparisons in PMC10600351
(Eur Respir J 2023, doi:10.1183/13993003.00174-2023).

Run:  python recompute.py.txt              # prints all numbers, writes fig1/fig2 (.svg, .png)
      python recompute.py.txt --candidates # also writes five candidate figure designs
                                           # to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.

Inputs are copied from the paper as archived in PMC:
  Abstract (Results), Results "Outcomes compared with other inflammatory diseases",
  Table 3 (CF vs general population) and Table 4 (CF vs SLE, RA, HIV).
Method: for a ratio reported with a 95% Wald CI on the log scale,
  SE(log HR) = (ln upper - ln lower) / (2 * 1.959964),  z = ln HR / SE,
  two-sided p = 2 * (1 - Phi(|z|)).
The exact Cox p-value cannot be rebuilt without individual event times; this
back-calculation is the standard check of internal consistency.
"""
import itertools
import os
import sys

import numpy as np
from scipy.stats import norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)

# ----------------------------------------------------------------- inputs (from the paper)
# name: (HR, lower, upper, p in Table 3/4, p in abstract)
MATCHED = {
    "General population": (1.65, 1.40, 1.95, "<0.001", "<0.001"),   # Table 3, abstract
    "SLE":                (0.95, 0.82, 1.09, "0.439", "0.44"),       # Table 4, abstract
    "Rheumatoid arthritis": (1.21, 1.00, 1.48, "0.055", "<0.001"),   # Table 4, abstract
    "HIV":                (0.93, 0.82, 1.06, "0.293", "0.29"),       # Table 4, abstract
}
RA_UNADJ = (0.75, 0.66, 0.86, "<0.001")          # Table 4, RA row, unadjusted columns
RA_MATCHED_EVENTS = (223, 5901, 177, 5901)        # Table 4: CF events/n, RA events/n
SLE_ROW_N = (223, 385, 5223, 389)                 # Table 4 SLE matched: CF n, events, non-CF n, events


def p_from_ci(hr, lo, hi):
    se = (np.log(hi) - np.log(lo)) / (2 * Z975)
    z = np.log(hr) / se
    return se, z, 2 * norm.sf(abs(z))


def fmt_p(p):
    return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"


# ----------------------------------------------------------------- 1. back-calculated p
print("1. p-values back-calculated from the reported HR and 95% CI (matched analyses)")
RES = {}
for k, (h, lo, hi, p_tab, p_abs) in MATCHED.items():
    se, z, p = p_from_ci(h, lo, hi)
    RES[k] = (se, z, p)
    print(f"   {k:21s} HR {h:.2f} ({lo:.2f}-{hi:.2f}): SE(logHR)={se:.4f} z={z:+.3f} "
          f"p={fmt_p(p)}   table {p_tab}, abstract {p_abs}")
print(f"   General population p = {RES['General population'][2]:.1e}")

# ----------------------------------------------------------------- 2. rounding range for RA
h0, l0, u0 = MATCHED["Rheumatoid arthritis"][:3]
grid = []
for h, l, u in itertools.product(np.linspace(h0 - .005, h0 + .005, 21),
                                 np.linspace(l0 - .005, l0 + .005, 21),
                                 np.linspace(u0 - .005, u0 + .005, 21)):
    grid.append(p_from_ci(h, l, u)[2])
P_LO, P_HI = min(grid), max(grid)
print("\n2. RA matched: p over every HR, lower and upper bound consistent with rounding to 2 dp")
print(f"   p range {P_LO:.4f} - {P_HI:.4f}")
print(f"   geometric midpoint of CI sqrt(1.00*1.48) = {np.sqrt(l0 * u0):.3f} (reported HR 1.21)")
print(f"   one-sided p = {RES['Rheumatoid arthritis'][2] / 2:.4f}")

# ----------------------------------------------------------------- 3. what p<0.001 would need
z001 = norm.isf(0.0005)
se_need = np.log(h0) / z001
CI_NEED = (h0 * np.exp(-Z975 * se_need), h0 * np.exp(Z975 * se_need))
print("\n3. RA matched: what a two-sided p<0.001 would require at HR 1.21")
print(f"   |z| > {z001:.3f}; SE(logHR) <= {se_need:.4f} (reported CI implies "
      f"{RES['Rheumatoid arthritis'][0]:.4f}); 95% CI would be at most "
      f"{CI_NEED[0]:.2f}-{CI_NEED[1]:.2f}")

# ----------------------------------------------------------------- 4. the unadjusted RA row
se_u, z_u, p_u = p_from_ci(*RA_UNADJ[:3])
print("\n4. RA unadjusted (Table 4): HR 0.75 (0.66-0.86), reported p<0.001")
print(f"   SE={se_u:.4f} z={z_u:+.3f} p={p_u:.1e}  -> the only p<0.001 in the RA row")

# ----------------------------------------------------------------- 5. crude counts
a, n1, b, n2 = RA_MATCHED_EVENTS
rr = (a / n1) / (b / n2)
se_rr = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n2)
z_rr = np.log(rr) / se_rr
CRUDE = (rr, rr * np.exp(-Z975 * se_rr), rr * np.exp(Z975 * se_rr), 2 * norm.sf(abs(z_rr)))
print("\n5. RA matched event counts 223/5901 vs 177/5901 (ignores time to event)")
print(f"   risk {a / n1 * 100:.2f}% vs {b / n2 * 100:.2f}%; crude risk ratio {CRUDE[0]:.3f} "
      f"(95% CI {CRUDE[1]:.3f}-{CRUDE[2]:.3f}), p={CRUDE[3]:.4f}")

# ----------------------------------------------------------------- 6. side note (not part of the claim)
print("\n6. Side note: Table 4 SLE matched row lists CF n=%d with %d events, against %d "
      "non-CF; the Results text says 5223 PwCF were matched" % SLE_ROW_N[:3])

# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, path):
    fig.savefig(path + ".svg", bbox_inches="tight")
    fig.savefig(path + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


def style_ax(ax, grid_axis="x"):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis=grid_axis, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)


def fig_pvalue_axis(path):
    """Design A: reported vs recomputed p-values on one log axis, with alpha 0.05."""
    order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
    labels = {"General population": "CF vs general population\nHR 1.65 (1.40–1.95)",
              "SLE": "CF vs SLE\nHR 0.95 (0.82–1.09)",
              "Rheumatoid arthritis": "CF vs rheumatoid arthritis\nHR 1.21 (1.00–1.48)",
              "HIV": "CF vs HIV\nHR 0.93 (0.82–1.06)"}
    XMIN, XMAX = 1e-5, 1.0
    fig, ax = plt.subplots(figsize=(7.2, 4.8))
    ys = {k: len(order) - 1 - i for i, k in enumerate(order)}
    ax.axvline(0.05, color=INK2, lw=1, ls=(0, (4, 3)))
    ax.text(0.05 * 1.12, 3.5, "p = 0.05", color=INK2, fontsize=10, va="center", ha="left")
    ax.axvline(0.001, color=INK2, lw=0.8, ls=":")
    ax.text(0.001 * 1.12, 3.5, "p = 0.001", color=INK2, fontsize=10, va="center", ha="left")
    D = 0.24

    def lt_arrow(x, y, mk):
        ax.annotate("", xy=(x / 6, y), xytext=(x, y),
                    arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2))
        ax.plot(x, y, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)

    for k in order:
        y = ys[k]
        p_tab, p_abs = MATCHED[k][3], MATCHED[k][4]
        for val, mk, yy in ((p_abs, "D", y + D), (p_tab, "s", y)):
            if val.startswith("<"):
                lt_arrow(float(val[1:]), yy, mk)
            else:
                ax.plot(float(val), yy, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)
        yb = y - D
        p_rec = RES[k][2]
        if p_rec < XMIN:
            ax.annotate("", xy=(XMIN, yb), xytext=(XMIN * 8, yb),
                        arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
            ax.text(XMIN * 11, yb, f"recomputed: {p_rec:.0e} (beyond axis)", color=INK,
                    fontsize=10, va="center")
            continue
        if k == "Rheumatoid arthritis":
            ax.plot([P_LO, P_HI], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
            for xv in (P_LO, P_HI):
                ax.plot([xv, xv], [yb - .08, yb + .08], color=BLUE, lw=2)
        ax.plot(p_rec, yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
    y = ys["Rheumatoid arthritis"]
    ax.text(0.001 / 9, y + D, "abstract: p<0.001  ", color=INK, fontsize=10,
            ha="right", va="center", fontweight="bold")
    ax.text(0.055 / 1.4, y, "Table 4: 0.055", color=INK, fontsize=10, va="center", ha="right")
    ax.text(P_LO / 1.25, y - D, f"recomputed: {RES['Rheumatoid arthritis'][2]:.3f} "
            f"(rounding range {P_LO:.3f}–{P_HI:.3f})", color=INK, fontsize=10, va="center",
            ha="right")
    ax.set_yticks([ys[k] for k in order], [labels[k] for k in order])
    ax.set_xscale("log")
    ax.set_xlim(XMIN, XMAX)
    ax.set_xticks([1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1e-1, 1],
                  ["0.00001", "0.0001", "0.001", "0.01", "0.05", "0.1", "1"])
    ax.minorticks_off()
    ax.set_ylim(-0.5, 3.75)
    ax.set_xlabel("Two-sided p-value (log scale)")
    style_ax(ax)
    h = [plt.Line2D([], [], marker="D", ls="", ms=8, color=ORANGE, label="Reported in abstract\n(arrow: “p<” value)"),
         plt.Line2D([], [], marker="s", ls="", ms=8, color=ORANGE, label="Reported in Table 3 / 4"),
         plt.Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
                    label="Recomputed from HR and 95% CI\n(bar: range over rounding)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=3,
              frameon=False, fontsize=9.5, handletextpad=0.4, columnspacing=1.2)
    ax.set_title("Propensity-matched MACE comparisons: reported and recomputed p-values",
                 loc="left", fontsize=11.5, color=INK, x=-0.38, pad=18)
    save(fig, path)


def box(ax, x, y, w, h, title, body, edge, fs=10.5):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
                                fc="#ffffff", ec=edge, lw=2, transform=ax.transAxes,
                                clip_on=False))
    ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
            fontweight="bold", va="top", color=INK)
    ax.text(x + 0.02, y + h - 0.13, body, transform=ax.transAxes, fontsize=fs,
            va="top", color=INK, linespacing=1.45)


def fig_text_panels(path):
    """Design B: abstract vs Table 4 excerpts with a connector naming the conflict."""
    fig = plt.figure(figsize=(7.2, 3.3))
    ax = fig.add_axes([0.02, 0.12, 0.96, 0.84])
    ax.set_axis_off()
    box(ax, 0.0, 0.36, 0.44, 0.60, "Abstract, Results",
        "“rheumatoid arthritis\n(HR 1.21, 95% CI 1.00–1.48;\n p<0.001)”", ORANGE, fs=11)
    box(ax, 0.56, 0.36, 0.44, 0.60, "Table 4, propensity-matched",
        "Rheumatoid arthritis\nCF 223/5901 vs RA 177/5901\nHR 1.21 (1.00–1.48)\np-value 0.055",
        ORANGE, fs=11)
    ax.annotate("", xy=(0.555, 0.66), xytext=(0.445, 0.66), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="<|-|>", color=INK2, lw=2))
    ax.text(0.5, 0.71, "same HR\nand CI", ha="center", fontsize=10, color=INK2,
            transform=ax.transAxes)
    ax.text(0.5, 0.61, "different\np-value", ha="center", fontsize=10, color=INK,
            fontweight="bold", transform=ax.transAxes, va="top")
    ax.text(0.0, 0.18, f"Recomputed from HR 1.21 and CI 1.00–1.48: p = "
            f"{RES['Rheumatoid arthritis'][2]:.3f} (range {P_LO:.3f}–{P_HI:.3f} over rounding).",
            transform=ax.transAxes, fontsize=10.5, color=BLUE, fontweight="bold")
    ax.text(0.0, 0.06, "Orange boxes: text as printed in the paper. Blue: recomputed here.",
            transform=ax.transAxes, fontsize=9.5, color=INK2)
    save(fig, path)


def fig_hr_needed(path):
    """Design C: reported CI vs the widest CI compatible with p<0.001, on a log HR axis."""
    fig, ax = plt.subplots(figsize=(7.2, 2.6))
    rows = [("Reported (abstract and Table 4)\nHR 1.21, 95% CI 1.00–1.48", (l0, u0), ORANGE),
            (f"Widest 95% CI that would give\np<0.001 at HR 1.21: "
             f"{CI_NEED[0]:.2f}–{CI_NEED[1]:.2f}", CI_NEED, BLUE)]
    ax.axvline(1, color=INK2, lw=1)
    ax.text(1.005, 1.55, "HR = 1 (no difference)", color=INK2, fontsize=10, ha="left")
    for i, (lab, (lo, hi), c) in enumerate(rows):
        y = 1 - i
        ax.plot([lo, hi], [y, y], color=c, lw=2)
        ax.plot(h0, y, "o", ms=9, color=c, mec=BG, mew=1.5)
        ax.text(hi * 1.02, y, f"{lo:.2f}–{hi:.2f}", color=INK, fontsize=10, va="center")
    ax.set_yticks([1, 0], [r[0] for r in rows])
    ax.set_xscale("log")
    ax.set_xlim(0.9, 1.7)
    ax.set_xticks([0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6], ["0.9", "1.0", "1.1", "1.2", "1.3",
                                                              "1.4", "1.5", "1.6"])
    ax.minorticks_off()
    ax.set_ylim(-0.5, 1.8)
    ax.set_xlabel("Hazard ratio, CF vs rheumatoid arthritis (propensity-matched, log scale)")
    style_ax(ax)
    save(fig, path)


def fig_z(path):
    """Design D: |z| implied by each reported CI against the 0.05 and 0.001 thresholds."""
    order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
    fig, ax = plt.subplots(figsize=(7.2, 3.0))
    for i, k in enumerate(order):
        y = len(order) - 1 - i
        z = abs(RES[k][1])
        ax.plot([0, z], [y, y], color=BLUE, lw=2)
        ax.plot(z, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5)
        ax.text(z + 0.12, y, f"|z| = {z:.2f}", va="center", fontsize=10)
    yra = 1
    ax.plot(z001, yra + 0.28, "D", ms=8, color=ORANGE)
    ax.text(z001 + 0.12, yra + 0.28, "abstract's p<0.001 needs |z| > 3.29", color=INK,
            fontsize=10, va="center")
    for zz, lab in ((Z975, "p = 0.05"), (z001, "p = 0.001")):
        ax.axvline(zz, color=INK2, lw=1, ls=(0, (4, 3)))
        ax.text(zz + 0.06, 3.55, lab, ha="left", fontsize=10, color=INK2)
    ax.set_yticks(range(4)[::-1], ["vs general population", "vs SLE", "vs rheumatoid arthritis",
                                   "vs HIV"])
    ax.set_xlim(0, 6.8)
    ax.set_ylim(-0.5, 3.8)
    ax.set_xlabel("|z| implied by the reported HR and 95% CI (blue, recomputed)")
    style_ax(ax)
    save(fig, path)


def fig_row_origin(path):
    """Design E: the Table 4 RA row with both columns; the abstract's p matches the wrong one."""
    fig = plt.figure(figsize=(7.2, 3.7))
    ax = fig.add_axes([0.02, 0.1, 0.96, 0.8])
    ax.set_axis_off()
    box(ax, 0.01, 0.58, 0.46, 0.38, "Table 4, RA: unadjusted",
        "HR 0.75 (0.66–0.86)\np-value <0.001", INK2, fs=11)
    box(ax, 0.53, 0.58, 0.46, 0.38, "Table 4, RA: propensity-matched",
        "HR 1.21 (1.00–1.48)\np-value 0.055", INK2, fs=11)
    box(ax, 0.12, 0.04, 0.76, 0.30, "Abstract, Results",
        "“rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)”", ORANGE, fs=10.5)
    ax.annotate("", xy=(0.66, 0.36), xytext=(0.76, 0.565), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
    ax.text(0.73, 0.46, "HR and CI as printed here", fontsize=10, color=INK2,
            transform=ax.transAxes, va="center")
    ax.annotate("", xy=(0.34, 0.36), xytext=(0.24, 0.565), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--"))
    ax.text(0.27, 0.46, "the row's only p<0.001\n(a different analysis, HR < 1)",
            fontsize=10, color=INK, transform=ax.transAxes, va="center", ha="right")
    ax.text(0.01, -0.07, f"Recomputed from the matched HR and CI: p = "
            f"{RES['Rheumatoid arthritis'][2]:.3f}, consistent with Table 4's 0.055 (blue = "
            "recomputed here).", transform=ax.transAxes, fontsize=10, color=BLUE)
    ax.text(0.01, 1.04, "Where the abstract's RA values match Table 4", transform=ax.transAxes,
            fontsize=11.5, color=INK)
    save(fig, path)


# write the chosen figures
fig_pvalue_axis(os.path.join(HERE, "fig1"))
fig_row_origin(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (p-value axis) and fig2.svg/png (Table 4 row vs abstract)")

if "--candidates" in sys.argv:
    cdir = os.path.join(HERE, "candidates")
    os.makedirs(cdir, exist_ok=True)
    for tag, fn in (("A_pvalue_axis", fig_pvalue_axis), ("B_text_panels", fig_text_panels),
                    ("C_ci_needed", fig_hr_needed), ("D_z_thresholds", fig_z),
                    ("E_table_row_origin", fig_row_origin)):
        fn(os.path.join(cdir, tag))
    print("wrote candidates/A-E (.svg, .png)")

Abstract transposes two of four unilateral and bilateral effusion mortality figures given correctly in Results

Eur Respir J (2015) · doi:10.1183/09031936.00217114 · PMC4857137

Summary

This cohort of 308 patients undergoing thoracentesis (247 unilateral, 61 bilateral) reports in its abstract that bilateral effusion, relative to unilateral, carried higher mortality, giving "17% versus 47%" with the 30-day hazard ratio and "36% versus 69%" with the 1-year hazard ratio. The Results instead give unilateral mortality as 17% (30 days) and 47% (1 year) and bilateral mortality as 36% and 69%, and only these figures reproduce the paper's reported totals of 65 and 158 deaths (43/247 + 22/61 and 116/247 + 42/61); no count out of 61 even rounds to 47%. The abstract, including the version of record indexed in PubMed, has therefore transposed 47% and 36%: it overstates bilateral 30-day mortality (47% instead of 36%), understates unilateral 1-year mortality (36% instead of 47%), and makes the absolute gap look 11 percentage points larger at both time points. The hazard ratios and the direction of the finding are unaffected, but this is the comparison the paper presents as its new contribution, and the incorrect 47% figure has since been cited.

What the paper reports

Abstract, Results (PMC author manuscript):

Patients with bilateral pleural effusion, relative to unilateral, were associated with higher risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])

Abstract, Results, as published in Eur Respir J 46(2):495–502 and indexed in PubMed (PMID 25837039):

Patients with bilateral, relative to unilateral, pleural effusion were associated with higher risk of death at 30 days and 1 year (17% versus 47% (hazard ratio (HR) 2.58, 95% CI 1.44-4.63) and 36% versus 69% (HR 2.32, 95% CI 1.55-3.48), respectively).

Results, first paragraph:

A total of 247 unilateral and 61 bilateral thoracenteses were performed. Overall mortality across all groups was 21% (65 of 308 patients) at 30-days and 51% (158 of 308 patients) at one year

Results, second paragraph:

As calculated from Figure 1, thirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, compared with 36% and 69% in patients with bilateral effusion.

Relative to patients with unilateral effusion, unadjusted mortality of patients with bilateral effusion was higher at thirty-days (p=0.0003) and at one-year (p=0.0001).

Results, fifth paragraph (Table 3, row "Bilateral", reads the same: 2.58 (1.44, 4.63), 0.002 | 2.32 (1.55, 3.48), <0.001):

was significantly associated with higher 30-day and 1-year mortality (HR 2.58 [1.44–4.63] p=0.002 and HR 2.32 [1.55–3.48] p<0.001, respectively)

Methods, Statistical Analysis, on what the percentages are:

Unadjusted 30-day and 1-year mortalities were calculated and diagrammed for subgroups according to the following hierarchy: laterality (bilateral versus unilateral) and malignancy (yes versus no).

Discussion, on the weight the paper gives the comparison:

The study is novel in evaluating mortality associated with bilateral effusions.

It is also the first study to report the association between bilateral pleural effusion and mortality.

Abstract, Conclusions:

Bilateral pleural effusion is distinctly associated with high mortality.

What the paper's own numbers show

Method. The group sizes (247 and 61) and the overall death counts (65 and 158 of 308) come from the Results. For each printed percentage I listed every integer number of deaths k for which k/n rounds to that percentage. I then asked which pairing of unilateral and bilateral percentages can reproduce the reported overall totals: the pairing in the Results (unilateral 17%/47%, bilateral 36%/69%), or the pairing in the abstract, where each "x% versus y%" is tied to one time point by the hazard ratio that follows it (30 days: 17% vs 47%; 1 year: 36% vs 69%). For each correct split I recomputed the risk difference and crude risk ratio. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Group sizes Abstract; Results ¶1 247 unilateral, 61 bilateral 247 + 61 = 308
Overall deaths, 30 days Results ¶1 65 of 308 (21%) 21.1%
Overall deaths, 1 year Results ¶1 158 of 308 (51%) 51.3%
Unilateral, 30 days Results ¶2; abstract 17% 43/247 = 17.4% (41–43 deaths round to 17%)
Bilateral, 30 days Results ¶2 36% 22/61 = 36.1% (22 is the only count giving 36%)
Bilateral, 30 days Abstract (30-day pair, with HR 2.58) 47% not attainable: 28/61 = 45.9%, 29/61 = 47.5%; the value is the unilateral 1-year figure
Unilateral, 1 year Results ¶2 47% 116/247 = 47.0%
Unilateral, 1 year Abstract (1-year pair, with HR 2.32) 36% would mean 88–90 deaths; the value is the bilateral 30-day figure
Bilateral, 1 year Results ¶2; abstract 69% 42/61 = 68.9%
Total 30-day deaths implied, Results pairing derived 65 0.17 × 247 + 0.36 × 61 = 64.0 (63–65 with rounding); the only exact split is 43 + 22 = 65
Total 30-day deaths implied, abstract pairing derived 65 0.17 × 247 + 0.47 × 61 = 70.7 (about 69–72); no integer split gives 65
Total 1-year deaths implied, Results pairing derived 158 0.47 × 247 + 0.69 × 61 = 158.2 (157–159); the only exact split is 116 + 42 = 158
Total 1-year deaths implied, abstract pairing derived 158 0.36 × 247 + 0.69 × 61 = 131.0 (130–132); no integer split gives 158
Bilateral minus unilateral, 30 days Abstract vs Results 47 − 17 = 30 pp 36.1 − 17.4 = 18.7 pp (19 pp from the rounded Results figures); crude RR 2.07 (2.76 as printed in the abstract)
Bilateral minus unilateral, 1 year Abstract vs Results 69 − 36 = 33 pp 68.9 − 47.0 = 21.9 pp (22 pp from the rounded Results figures); crude RR 1.47 (1.92 as printed in the abstract)
Adjusted HR, bilateral vs unilateral Abstract; Results ¶5; Table 3 2.58 (1.44–4.63) at 30 days; 2.32 (1.55–3.48) at 1 year identical in all three places; not affected

The four numbers in the abstract are the right four numbers, but 47% and 36% have swapped places. The Results figures, the death counts they imply and the reported overall totals all agree with one another. The abstract's pairing agrees with none of them.

One side observation does not form part of the finding. Pearson chi-square tests on the recomputed counts give unadjusted p = 0.0014 (30 days) and 0.0022 (1 year), and Fisher exact tests give 0.0026 at both time points. The Results report p = 0.0003 and 0.0001. The paper does not name the test, and a log-rank test on time to death could legitimately give different values, so I leave this unresolved.

Figure

The abstract's mortality pairs compared with the Results, with the transposed cells joined

Figure 1. The upper panel quotes the abstract's sentence and lays out its four percentages as the sentence assigns them: each "x% versus y%" pair is followed by its time-point hazard ratio (HR 2.58 is the 30-day HR and 2.32 the 1-year HR in Table 3), and in each pair bilateral is the higher value. The values as printed are shown in orange. The lower panel quotes the Results and shows, in blue, the death counts recomputed from the paper's own numbers: the only integer split that matches both the Results percentages and the reported totals of 65 and 158 deaths. The unilateral 30-day (17%) and bilateral 1-year (69%) cells agree. The bilateral 30-day and unilateral 1-year cells hold each other's values: the abstract's 47% is the unilateral 1-year mortality (116/247), and its 36% is the bilateral 30-day mortality (22/61).

Total deaths implied by each pairing compared with the reported totals

Figure 2. Total deaths implied by applying each pairing's percentages to the 247 unilateral and 61 bilateral patients, on one axis of deaths that starts at zero. Filled blue circles use the Results pairing and open blue circles the abstract's pairing. The ranges in the labels allow for rounding of the printed percentages. Orange lines mark the reported totals (65/308 at 30 days, 158/308 at 1 year). The Results pairing reproduces both totals. The abstract's pairing implies about 71 deaths at 30 days and 131 at 1 year, and neither range contains the reported total.

Why it matters

Explanations considered

Status

Suggested correction

In the Results sentence of the abstract, change

(17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])

to

(36% versus 17%; HR 2.58 CI [1.44–4.63] and 69% versus 47%; HR 2.32 CI [1.55–3.48])

or, keeping the original order, "(17% versus 36% … and 47% versus 69% …)", stated explicitly as unilateral versus bilateral. The version of record needs the same change, with "respectively" kept. No change is needed to the main text, Table 3 or the conclusions. Because the incorrect figure has already been cited, a published correction notice linked to the article would be more useful than a silent update.

Recomputation script (python recompute.py.txt)
"""Recompute the unilateral vs bilateral mortality figures of PMC4857137
(Eur Respir J 2015;46(2):495-502, doi:10.1183/09031936.00217114).

Run:  python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC (author manuscript):
  Results p0: 247 unilateral and 61 bilateral thoracenteses; 65 of 308 deaths at
              30 days, 158 of 308 at 1 year
  Results p1: unilateral 17% (30 d) and 47% (1 y); bilateral 36% (30 d) and 69% (1 y);
              unadjusted p = 0.0003 (30 d) and 0.0001 (1 y)
  Abstract:   "(17% versus 47%; HR 2.58 ...) and (36% versus 69%; HR 2.32 ...)"
  Table 3:    adjusted HR bilateral 2.58 (1.44, 4.63) at 30 d, 2.32 (1.55, 3.48) at 1 y
"""
import os
import textwrap

from scipy.stats import chi2_contingency, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
N_U, N_B = 247, 61                      # Results p0, Abstract
N_ALL = 308
TOTAL = {"30-day": 65, "1-year": 158}   # Results p0
RESULTS_PCT = {"30-day": (17, 36), "1-year": (47, 69)}   # (unilateral, bilateral), Results p1
ABSTRACT_PCT = {"30-day": (17, 47), "1-year": (36, 69)}  # (unilateral, bilateral) as paired in the abstract
HR = {"30-day": (2.58, 1.44, 4.63), "1-year": (2.32, 1.55, 3.48)}  # Abstract, Results p4, Table 3
P_REPORTED = {"30-day": "0.0003", "1-year": "0.0001"}               # Results p1, test not named


def rnd(x):
    """Round half up to an integer percentage, as printed."""
    return int(x + 0.5)


def counts_for(pct, n):
    """All integer counts k (0..n) whose k/n prints as pct when rounded to a whole percent."""
    return [k for k in range(n + 1) if rnd(100 * k / n) == pct]


print("=== 1. Group sizes ===")
print(f"unilateral + bilateral = {N_U} + {N_B} = {N_U + N_B} (reported {N_ALL})")
for t in TOTAL:
    print(f"overall {t} deaths: {TOTAL[t]}/{N_ALL} = {100 * TOTAL[t] / N_ALL:.1f}%")

print("\n=== 2. Which death counts are compatible with each printed percentage ===")
for t in ("30-day", "1-year"):
    for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
        cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
        print(f"{t} {label:8s}: unilateral {pu}% of {N_U} -> deaths {cu}; "
              f"bilateral {pb}% of {N_B} -> deaths {cb if cb else 'NONE (no k/61 rounds to %d%%)' % pb}")
print("k/61 near 47%:", [(k, round(100 * k / N_B, 1)) for k in range(26, 31)])

print("\n=== 3. Implied total deaths under each pairing vs the reported total ===")
RANGES = {}
for t in ("30-day", "1-year"):
    for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
        point = pu / 100 * N_U + pb / 100 * N_B
        cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
        if cb:
            lo, hi = min(cu) + min(cb), max(cu) + max(cb)
        else:  # bilateral percentage not attainable: use the real-valued rounding interval
            lo = (pu - 0.5) / 100 * N_U + (pb - 0.5) / 100 * N_B
            hi = (pu + 0.5) / 100 * N_U + (pb + 0.5) / 100 * N_B
        RANGES[(t, label)] = (point, lo, hi)
        inside = lo <= TOTAL[t] <= hi
        print(f"{t} {label:8s}: {pu/100:.2f}*{N_U} + {pb/100:.2f}*{N_B} = {point:.1f}; "
              f"totals compatible with rounding {lo:.0f}-{hi:.0f}; reported {TOTAL[t]} -> "
              f"{'consistent' if inside else 'INCONSISTENT'}")

print("\n=== 4. Integer splits that reproduce both the percentages and the reported totals ===")
SPLITS = {}
for t in ("30-day", "1-year"):
    for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
        sols = [(u, TOTAL[t] - u) for u in counts_for(pu, N_U)
                if (TOTAL[t] - u) in counts_for(pb, N_B)]
        print(f"{t} {label:8s} pairing {pu}% vs {pb}%: solutions (unilateral, bilateral) = {sols}")
        if label == "Results":
            SPLITS[t] = sols[0]
for t, (u, b) in SPLITS.items():
    print(f"{t}: unilateral {u}/{N_U} = {100*u/N_U:.1f}%, bilateral {b}/{N_B} = {100*b/N_B:.1f}%, "
          f"sum {u + b} (reported {TOTAL[t]})")

print("\n=== 5. Size of the comparison: Results vs the abstract's pairing ===")
for t in ("30-day", "1-year"):
    u, b = SPLITS[t]
    ru, rb = 100 * u / N_U, 100 * b / N_B
    au, ab = ABSTRACT_PCT[t]
    print(f"{t}: from counts  RD = {rb:.1f} - {ru:.1f} = {rb - ru:.1f} pp, crude RR = {rb / ru:.2f}")
    print(f"{t}: Results text RD = {RESULTS_PCT[t][1]} - {RESULTS_PCT[t][0]} = "
          f"{RESULTS_PCT[t][1] - RESULTS_PCT[t][0]} pp, RR = {RESULTS_PCT[t][1] / RESULTS_PCT[t][0]:.2f}")
    print(f"{t}: abstract     RD = {ab} - {au} = {ab - au} pp, RR = {ab / au:.2f}")
    print(f"{t}: abstract overstates the gap by {(ab - au) - (RESULTS_PCT[t][1] - RESULTS_PCT[t][0])} pp")

print("\n=== 6. Hazard ratios (not affected) ===")
for t, (h, lo, hi) in HR.items():
    print(f"{t}: HR {h} ({lo}-{hi}) - identical in Abstract, Results p4 and Table 3")

print("\n=== 7. Side note: unadjusted tests on the recomputed counts (test in paper not named) ===")
for t in ("30-day", "1-year"):
    u, b = SPLITS[t]
    tab = [[b, N_B - b], [u, N_U - u]]
    _, p, _, _ = chi2_contingency(tab, correction=False)
    _, py, _, _ = chi2_contingency(tab, correction=True)
    pf = fisher_exact(tab)[1]
    print(f"{t}: reported p = {P_REPORTED[t]}; Pearson chi-square p = {p:.4f}; "
          f"Yates p = {py:.4f}; Fisher exact p = {pf:.4f}")

# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
                     "svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG})


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)


# ---- fig1: abstract vs Results, two text panels; the swapped cells are joined by crossing connectors
FH = 5.5
fig = plt.figure(figsize=(7.2, FH))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, FH)
ax.axis("off")
COLS = [2.2, 3.45, 4.7, 5.95]           # unilateral 30 d, bilateral 30 d, unilateral 1 y, bilateral 1 y
HEAD = ["Unilateral\n30 days", "Bilateral\n30 days", "Unilateral\n1 year", "Bilateral\n1 year"]
X0, W = 0.12, 6.96


def panel(y_bottom, h):
    ax.add_patch(FancyBboxPatch((X0, y_bottom), W, h, boxstyle="round,pad=0.0,rounding_size=0.06",
                                fc="white", ec="#cfcdc7", lw=1))


# Panel A: abstract
panel(3.2, 2.2)
ax.text(X0 + 0.15, 5.18, "Abstract, Results sentence", fontsize=11.5, weight="bold", color=INK, va="center")
qa = ("\u201cPatients with bilateral pleural effusion, relative to unilateral, were associated with higher "
      "risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44\u20134.63] and 36% versus 69%; "
      "HR 2.32 CI [1.55\u20133.48])\u201d")
ax.text(X0 + 0.15, 4.98, "\n".join(textwrap.wrap(qa, 86)), fontsize=10, color=INK2, style="italic",
        va="top", linespacing=1.3)
for (c0, c1), lab in (((COLS[0], COLS[1]), "30-day pair, HR 2.58"), ((COLS[2], COLS[3]), "1-year pair, HR 2.32")):
    ax.plot([c0 - 0.35, c1 + 0.35], [4.12, 4.12], color=INK2, lw=1)
    ax.text((c0 + c1) / 2, 4.2, lab, ha="center", va="bottom", fontsize=10, color=INK2)
for x, hd in zip(COLS, HEAD):
    ax.text(x, 3.88, hd, ha="center", va="center", fontsize=10, color=INK2, linespacing=1.1)
ax.text(X0 + 0.15, 3.45, "As printed", fontsize=10, color=ORANGE, weight="bold",
        va="center", linespacing=1.1)
A = ["17%", "47%", "36%", "69%"]
for j, (x, v) in enumerate(zip(COLS, A)):
    sw = j in (1, 2)
    ax.text(x, 3.45, v, ha="center", va="center", fontsize=12, color=ORANGE if sw else INK,
            weight="bold" if sw else "normal",
            bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=ORANGE if sw else "none", lw=1.6))

# Panel B: Results
panel(0.12, 1.98)
u30, b30 = SPLITS["30-day"]
u1, b1 = SPLITS["1-year"]
B = [(u30, N_U), (b30, N_B), (u1, N_U), (b1, N_B)]
ax.text(X0 + 0.15, 1.83, "Results,\nparagraph 2", fontsize=11.5, weight="bold", color=INK, va="center",
        linespacing=1.1)
ax.text(X0 + 0.15, 1.33, "Recomputed\nfrom counts", fontsize=10, color=BLUE, weight="bold", va="center",
        linespacing=1.1)
for j, (x, (k, n)) in enumerate(zip(COLS, B)):
    sw = j in (1, 2)
    ax.text(x, 1.62, f"{k}/{n}\n= {100 * k / n:.1f}%", ha="center", va="center", fontsize=11,
            color=BLUE if sw else INK, weight="bold" if sw else "normal", linespacing=1.15,
            bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=BLUE if sw else "none", lw=1.6))
qb = ("\u201cthirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, "
      "compared with 36% and 69% in patients with bilateral effusion\u201d. Counts: the only integer "
      "split matching these percentages and the reported 65 and 158 total deaths.")
ax.text(X0 + 0.15, 0.98, "\n".join(textwrap.wrap(qb, 86)), fontsize=10, color=INK2, style="italic",
        va="top", linespacing=1.3)

# connectors between the panels
ytop, ybot = 3.18, 2.02
for ja, jb in ((1, 2), (2, 1)):  # abstract cell ja holds the value of Results cell jb
    ax.add_patch(FancyArrowPatch((COLS[ja], ytop), (COLS[jb], ybot), arrowstyle="-|>",
                                 mutation_scale=14, lw=2, color=INK, shrinkA=2, shrinkB=2))
ym = (ytop + ybot) / 2
XL = COLS[1] - 0.35
ax.text(XL, ym + 0.27, "47% and 36% transposed", fontsize=11, weight="bold", color=INK, va="center", ha="right")
ax.text(XL, ym - 0.02, "each value sits in the other's cell", fontsize=10, color=INK2, va="center", ha="right")
ax.text(XL, ym - 0.28, "17% and 69% agree", fontsize=10, color=INK2, va="center", ha="right")
save(fig, "fig1")

# ---- fig2: implied total deaths under each pairing vs the reported totals
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.25, right=0.97, top=0.84, bottom=0.17)
rows = [("30-day", "Results"), ("30-day", "Abstract"), ("1-year", "Results"), ("1-year", "Abstract")]
ylab = {("30-day", "Results"): "30 days: Results\n(17% vs 36%)",
        ("30-day", "Abstract"): "30 days: Abstract\n(17% vs 47%)",
        ("1-year", "Results"): "1 year: Results\n(47% vs 69%)",
        ("1-year", "Abstract"): "1 year: Abstract\n(36% vs 69%)"}
ypos = {rows[0]: 4.0, rows[1]: 3.0, rows[2]: 1.4, rows[3]: 0.4}
for r in rows:
    point, lo, hi = RANGES[r]
    y = ypos[r]
    filled = r[1] == "Results"
    ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
    ax.plot(point, y, "o", ms=9, mfc=BLUE if filled else BG, mec=BLUE, mew=2)
    lab = f"{point:.0f} (range {lo:.0f}\u2013{hi:.0f})"
    if hi < TOTAL[r[0]] - 5:   # marker left of the reported line: label on the left
        ax.text(lo - 4, y, lab, va="center", ha="right", fontsize=10, color=BLUE)
    else:
        ax.text(hi + 4, y, lab, va="center", fontsize=10, color=BLUE)
for t, (ya, yb) in (("30-day", (3.0, 4.0)), ("1-year", (0.4, 1.4))):
    ax.plot([TOTAL[t]] * 2, [ya - 0.35, yb + 0.35], color=ORANGE, lw=2)
    ax.text(TOTAL[t], yb + 0.5, f"reported {TOTAL[t]}/308", ha="center", fontsize=10, color=ORANGE)
ax.set_yticks([ypos[r] for r in rows], [ylab[r] for r in rows], fontsize=10)
ax.set_ylim(-0.3, 4.9)
ax.set_xlim(0, 230)
ax.set_xticks(range(0, 226, 25))
ax.set_xlabel("Total deaths implied by the group percentages (247 unilateral, 61 bilateral)",
              fontsize=10, color=INK)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0, colors=INK)
ax.tick_params(axis="x", colors=INK2)
ax.plot([], [], "o", ms=9, mfc=BLUE, mec=BLUE, mew=2, label="Recomputed, Results pairing")
ax.plot([], [], "o", ms=9, mfc=BG, mec=BLUE, mew=2, label="Recomputed, abstract pairing")
ax.plot([], [], color=ORANGE, lw=2, label="Reported total deaths")
fig.legend(loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
           handlelength=1.6, columnspacing=1.0)
save(fig, "fig2")
print("\nfigures written to", HERE)

Dorsal attention network mediation reports a negative indirect effect with an entirely positive confidence interval

Hum Brain Mapp (2023) · doi:10.1002/hbm.26474 · PMC10619388

Summary

The paper reports that the temporal variability of the dorsal attention network (DAN) mediates the link between anhedonia and attention in adolescents with major depressive disorder, with "indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]". The estimate is negative but the interval is entirely positive, and the estimate lies 1.364 below the interval's lower bound, which is 17.8 interval half-widths. The paper's own Figure 7c(C) labels −0.438 as the direct path c′, not as the indirect effect. Its path coefficients give an indirect effect of a × b = −0.509 × 0.263 = −0.134, which also lies outside the printed interval. This result is the only inferential support for the fMRI half of the title claim and for an "Importantly" sentence in the Abstract, so as printed that claim has no valid confidence interval. The paper's other mediation result (EEG Microstate C-to-D transition) is internally consistent in the Results text.

What the paper reports

Results, "Network temporal properties associated with MDD", paragraph 4 (the DAN mediation):

Temporal variability in DAN mediated the relationship between anhedonia and attention (direct effect = -0.271; indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]; Figure 7d).

Results, "Network temporal properties associated with MDD", paragraph 3 (the inputs to that mediation):

Correlation analyses revealed that temporal variability was significantly negatively correlated with anhedonia (r = -0.266, p = 0.049; Figure 7c(A)) and positively correlated with attention scores in adolescents with MDD (r = 0.419, p = 0.019; Figure 7c(B)).

Results, "Characteristics of EEG microstate in adolescents with MDD", paragraph 3 (the second mediation, given for comparison):

The transition rate from Microstates C to D mediated the relationship between anhedonia and attention (direct effect = -0.497; indirect effect = -0.389, 95% confidence interval: [-0.571, -0.032], Figure 3c(H)).

Figure 7, caption:

Correlation analyses indicated that the temporal variability was significantly positively correlated with anhedonia and negatively related with attention scores in MDD patients (c(A,B)). Mediation analysis revealed that the temporal variability of DAN mediated the relationship from anhedonia to attention (c(C)).

Figure 7, panel c(C). The panel is an image; these labels were read from the publisher's graphic (Europe PMC file HBM-44-5749-g009.jpg). The paths are a = −0.509 (anhedonia to DAN variability) and b = 0.263 (DAN variability to attention), and the anhedonia-to-attention arrows carry:

−0.438(c′)

−0.572(c)

Bootstrap estimate [95% CI] = −0.271* [0.926, 1.079]

Figure 3, panel c(H) (file HBM-44-5749-g001.jpg). Here a = 0.416 and b = −0.260, and:

-0.389 (c′)

-0.497 (c)

Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]

The finding is stated as a headline result in the Abstract:

Importantly, the temporal variability of DAN mediated the relationship between anhedonia and attention deficits in the patient group.

and in the Discussion, opening paragraph:

Importantly, the dynamic features of the attentional network (transition probability of C-D and temporal variability of the DAN) mediated the relationship between anhedonia and attentional deficits in the patient group.

Discussion, "Adolescent with MDD and network dynamic attributes":

The temporal variability of DAN mediates the relationship between anhedonia and attention.

The title reads "Attention network mediates the association between anhedonia and attentional deficit". The Methods ("Statistical analyses") say that anhedonia was the independent variable, attention the dependent variable and the C-to-D conversion rate the mediator. They do not describe how the mediation confidence intervals were obtained, and they do not mention the DAN mediation model.

What the paper's own numbers show

Method. For each mediation I checked whether the printed estimate lies inside its printed 95% CI. I computed the implied total effect (c = c′ + ab) and proportion mediated from the text, and the indirect effect a × b and c − c′ from the figure's path labels. I checked the sign that a × b must take given the reported correlations and paths. I tested whether any other DAN quantity the paper prints could be the one the interval describes, and whether the coefficients could be standardized. Finally I worked out the sample sizes that the reported r and p values imply. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
DAN indirect effect vs its CI Results 3.6 p4 −0.438, CI [0.926, 1.079] Outside the CI. 1.364 below the lower bound (17.8 half-widths of 0.0765); CI midpoint 1.0025
Sign-flipped DAN CI Results 3.6 p4 [0.926, 1.079] [−1.079, −0.926] does not contain −0.438 either
DAN implied total effect, proportion mediated Results 3.6 p4 direct −0.271, indirect −0.438 c = −0.709, proportion 0.618. Neither lies in the CI
DAN indirect effect from figure paths Fig 7c(C) a = −0.509, b = 0.263 a × b = −0.134; c − c′ = −0.572 − (−0.438) = −0.134 (the paths agree with each other)
What −0.438 is Results 3.6 p4 vs Fig 7c(C) "indirect effect" in text Labelled c′ (direct path) in the figure
What −0.271 is Results 3.6 p4 vs Fig 7c(C) "direct effect" in text Labelled "Bootstrap estimate" in the figure; outside [0.926, 1.079]
Any printed DAN quantity inside the CI? Text and Fig 7c(C) c, a, b, c′, −0.271, a × b, c′/c (0.766), c/c′ (1.306) None
Sign of the DAN indirect effect Results p3; Fig 7 caption; Fig 7c(C) r = −0.266 and +0.419; caption gives the opposite pair; a < 0, b > 0 a × b < 0 under every source. The negative estimate is the plausible value and the positive CI is not
Standardized-coefficient check derived indirect −0.438 If standardized, a × b would be −0.066; Fig 7 a = −0.509 ≠ r = −0.266. The coefficients are unstandardized, so the CI cannot be rebuilt from the text
C-D indirect effect vs its CI Results 3.2 p3 −0.389, CI [−0.571, −0.032] Inside; CI negative and excludes 0. Total −0.886, proportion 0.439
C-D indirect effect from figure paths Fig 3c(H) a = 0.416, b = −0.260 a × b = −0.108 = c − c′; inside the CI. The figure's "Bootstrap Estimate" 2.241 lies outside its own CI
Sample size behind the DAN correlations Results 3.6 p3 r = −0.266, p = 0.049; r = 0.419, p = 0.019 Consistent only with n = 55 and n = 31 respectively (with n = 188, p would be 2.2 × 10⁻⁴ and 2.2 × 10⁻⁹)

Figure

Reported indirect effects and 95% CIs for the two mediation analyses, with a × b recomputed from the paper's figure path labels

Figure 1. The two mediation results on one axis of unstandardized indirect effect, with the null value 0 as a thin vertical line. Orange circles and bars are the indirect effects and 95% CIs printed in the Results text. Blue diamonds are a × b recomputed from the path coefficients in the paper's Figures 3c(H) and 7c(C); the paper reports no interval for these. For the EEG C-to-D mediation, the printed estimate (−0.389) and the recomputed a × b (−0.108) both lie inside the printed interval. For the DAN mediation, the printed estimate (−0.438) lies 1.364 below an interval that is entirely positive, and the recomputed a × b (−0.134) lies outside it too.

Results text and Figure 7c(C) give different labels to the same DAN numbers

Figure 2. The DAN mediation numbers as printed in the Results text (left) and in the path-diagram labels of the paper's Figure 7c(C) (right), transcribed rather than reproduced. Connector 1: the text calls −0.438 the indirect effect, while the figure labels it c′, the direct path. Connector 2: the text calls −0.271 the direct effect, while the figure labels it the bootstrap estimate. The bottom panel recomputes the indirect effect from the figure's own paths (−0.134). None of the three candidate values lies inside the shared interval [0.926, 1.079].

Why it matters

The title's claim that the "attention network mediates the association between anhedonia and attentional deficit" rests on two analyses: an EEG microstate transition (C to D) and fMRI DAN temporal variability. The DAN result is the only statistical support for the fMRI half of that claim. It appears in the Abstract as an "Importantly" finding and is restated three times in the Discussion. As printed, its 95% CI contradicts its own estimate in sign and location. The paper's figure and text also disagree about which printed number is the indirect effect. A reader cannot tell what the indirect effect is (−0.438, −0.271 or −0.134), what its interval is, or whether that interval excludes zero. The interval is narrow and centred near 1. If it came from a ratio-type quantity, for which 1 is the null, it would include the null. If it belongs to the negative a × b, its true limits are not reported.

The inputs to the mediation are unclear too. The Figure 7 caption reverses the signs of both correlations given in the Results text. One Discussion sentence calls the DAN–attention correlation negative ("the temporal variability of the DAN was significantly negatively correlated with attention scores"), while the Results report r = 0.419. The two r/p pairs fit only n = 55 and n = 31, not the 188 patients enrolled, and the paper does not say which subsample each analysis used. A separate inconsistency: Results paragraph 3 says DAN variability was "significantly higher in adolescents with MDD than in HC (t = 2.582, p = .010)", while the Abstract, the Figure 7 caption and the Discussion say it was weaker.

The EEG C-to-D mediation is unaffected in the text: its estimate lies inside its interval, both negative. It still supports the broader statement that an attention-related dynamic measure mediates the anhedonia–attention link. Its figure, however, prints a "Bootstrap Estimate" of 2.241 outside its own interval, and the figure labels −0.389 as c′ rather than as the indirect effect.

Explanations considered

Status

Suggested correction

  1. Re-run or re-extract the DAN mediation and report the indirect effect (a × b), its 95% CI, the direct effect c′ and the total effect c. Give the method used for the interval (for example the bootstrap type and number of resamples), the software, and whether the coefficients are unstandardized. Update the Results text, the Figure 7c(C) panel, and the Abstract and Discussion sentences that rest on this result.
  2. Make the labels in the Results text consistent with the path diagrams of Figures 3c(H) and 7c(C). The text currently calls the figures' c′ the "indirect effect", and calls the "bootstrap estimate" (DAN) or the total effect c (C-D) the "direct effect". Also correct the C-D panel's bootstrap estimate (2.241), which lies outside its own interval.
  3. Correct the Figure 7 caption's correlation signs, and the Discussion sentence calling the DAN–attention correlation negative, to match the Results (r = −0.266 with anhedonia, r = 0.419 with attention), or correct the Results if the caption is right.
  4. State the number of patients in each correlation and mediation analysis (the r/p pairs imply about 55 and 31). Reconcile Results paragraph 3 ("significantly higher in adolescents with MDD") with the Abstract and Discussion ("significantly weaker").
  5. If a valid interval for the DAN indirect effect includes zero, revise the title, the Abstract and the Discussion so that the mediation claim rests on the EEG result alone.
Recomputation script (python recompute.py.txt)
"""Recompute the mediation results of PMC10619388
(Hum Brain Mapp 2023, doi:10.1002/hbm.26474).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/cand_A ... cand_E) next to this file.

Inputs are copied from the paper as archived in PMC (Wiley JATS) and from the
publisher's figure graphics in the Europe PMC package:
  Results 3.2, p3:  C-D mediation, "direct effect = -0.497; indirect effect = -0.389,
                    95% confidence interval: [-0.571, -0.032]"
  Results 3.6, p3:  DAN correlations r = -0.266 (p = 0.049), r = 0.419 (p = 0.019)
  Results 3.6, p4:  DAN mediation, "direct effect = -0.271; indirect effect = -0.438,
                    95% confidence interval: [0.926, 1.079]"
  Figure 3 c(H) path labels: a = 0.416, b = -0.260, -0.389 (c'), -0.497 (c),
                    "Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]"
  Figure 7 c(C) path labels: a = -0.509, b = 0.263, -0.438 (c'), -0.572 (c),
                    "Bootstrap estimate [95% CI] = -0.271* [0.926, 1.079]"
"""
import os

import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (from the paper)
TEXT = {  # Results text
    "C-D": dict(direct=-0.497, indirect=-0.389, lo=-0.571, hi=-0.032),
    "DAN": dict(direct=-0.271, indirect=-0.438, lo=0.926, hi=1.079),
}
FIG = {   # path-diagram labels in the publisher's figure graphics
    "C-D": dict(a=0.416, b=-0.260, cprime=-0.389, c=-0.497, boot=2.241, lo=-0.571, hi=-0.032),
    "DAN": dict(a=-0.509, b=0.263, cprime=-0.438, c=-0.572, boot=-0.271, lo=0.926, hi=1.079),
}
DAN_R = [(-0.266, 0.049, "anhedonia"), (0.419, 0.019, "attention")]  # Results 3.6 p3

# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
LW, MS = 1.5, 8  # 1.5 pt = 2 px at 96 dpi; markers 8 pt
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
MINUS = "−"


def f(x, d=3):
    s = f"{x:.{d}f}"
    return s.replace("-", MINUS)


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight")
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


# ------------------------------------------------------------------ 1. containment checks
print("1. Estimate-versus-interval checks (Results text)")
for k, t in TEXT.items():
    mid, hw = (t["lo"] + t["hi"]) / 2, (t["hi"] - t["lo"]) / 2
    inside = t["lo"] <= t["indirect"] <= t["hi"]
    total = t["direct"] + t["indirect"]
    print(f"  {k}: indirect {t['indirect']}, CI [{t['lo']}, {t['hi']}]; midpoint {mid:.4f}, "
          f"half-width {hw:.4f}; inside={inside}; CI excludes 0={not (t['lo'] <= 0 <= t['hi'])}")
    print(f"      implied total c' + ab = {total:.3f}; proportion mediated = {t['indirect']/total:.3f}")
    if not inside:
        d = t["lo"] - t["indirect"]
        print(f"      distance to nearest bound = {d:.3f} = {d/hw:.1f} half-widths; "
              f"to far bound = {t['hi'] - t['indirect']:.3f}")
        print(f"      sign-flipped CI [{-t['hi']}, {-t['lo']}] contains estimate: "
              f"{-t['hi'] <= t['indirect'] <= -t['lo']}")

# ------------------------------------------------------------------ 2. figure path arithmetic
print("\n2. Path arithmetic from the figure labels")
AB = {}
for k, g in FIG.items():
    ab, diff = g["a"] * g["b"], g["c"] - g["cprime"]
    AB[k] = ab
    print(f"  {k}: a*b = {g['a']} x {g['b']} = {ab:.4f}; c - c' = {g['c']} - ({g['cprime']}) = {diff:.4f}; "
          f"proportion mediated ab/c = {ab/g['c']:.3f}")
    print(f"      a*b inside printed CI [{g['lo']}, {g['hi']}]: {g['lo'] <= ab <= g['hi']}; "
          f"bootstrap estimate {g['boot']} inside: {g['lo'] <= g['boot'] <= g['hi']}")
dan_vals = {"c (total, Fig 7)": FIG["DAN"]["c"], "a (Fig 7)": FIG["DAN"]["a"],
            "'indirect' text = c' Fig 7": -0.438, "'direct' text = bootstrap est. Fig 7": -0.271,
            "a*b": AB["DAN"], "b (Fig 7)": FIG["DAN"]["b"],
            "c'/c": FIG["DAN"]["cprime"] / FIG["DAN"]["c"], "c/c'": FIG["DAN"]["c"] / FIG["DAN"]["cprime"],
            "text total -0.709": -0.709, "proportion 0.618": 0.618}
print("  DAN candidate quantities and whether each lies in [0.926, 1.079]:")
for name, v in dan_vals.items():
    print(f"      {name:40s} {v:8.3f}  {0.926 <= v <= 1.079}")

# ------------------------------------------------------------------ 3. sign logic
print(f"  exp(a*b) = {np.exp(AB['DAN']):.3f}; exp(-0.438) = {np.exp(-0.438):.3f}; "
      f"inside [0.926, 1.079]: {0.926 <= np.exp(AB['DAN']) <= 1.079}, {0.926 <= np.exp(-0.438) <= 1.079}")
print("\n3. Sign logic")
print("  DAN, Results text: r(anhedonia, var) = -0.266, r(var, attention) = +0.419 -> a*b < 0")
print("  DAN, Fig 7 caption: var positively with anhedonia, negatively with attention -> a*b < 0")
print("  DAN, Fig 7 paths: a = -0.509, b = +0.263 -> a*b < 0")
print("  C-D: a = +0.416, b = -0.260 -> a*b < 0; printed CI is negative (consistent)")

# ------------------------------------------------------------------ 4. standardized-coefficient check
rmx, rmy = -0.266, 0.419
c_tot = TEXT["DAN"]["direct"] + TEXT["DAN"]["indirect"]
b_std = (rmy - rmx * c_tot) / (1 - rmx ** 2)
print(f"\n4. If text coefficients were standardized: r_xy = {c_tot:.3f}, b = {b_std:.3f}, "
      f"a*b = {rmx*b_std:.3f} (printed indirect -0.438) -> not standardized")
print(f"   Fig 7 a = {FIG['DAN']['a']} differs from r = {rmx}: unstandardized path, so no CI can be rebuilt")


# ------------------------------------------------------------------ 5. implied n
def pval(r, n):
    t = r * np.sqrt((n - 2) / (1 - r * r))
    return 2 * stats.t.sf(abs(t), n - 2)


print("\n5. Sample sizes consistent with the reported r and p (r rounded to 3 dp, p to 3 dp)")
for r0, p0, lab in DAN_R:
    ns = set()
    for r in np.linspace(abs(r0) - 0.0005, abs(r0) + 0.0005, 21):
        ns |= {n for n in range(10, 300) if round(pval(r, n), 3) == p0}
    print(f"  DAN variability vs {lab}: r = {r0}, p = {p0} -> n in {sorted(ns)} "
          f"(p at n=188 would be {pval(r0, 188):.3e})")

# ================================================================== figures
NOTE = "Values from the paper's Results text (orange) and recomputed from its own figure path labels (blue)."


def cand_A(paths):
    """Dot-and-interval plot, one axis, both mediations."""
    fig, ax = plt.subplots(figsize=(W, 3.3))
    rows = [("C–D transition (EEG)", "C-D", 1.0), ("DAN temporal variability (fMRI)", "DAN", 0.0)]
    for lab, k, y in rows:
        t = TEXT[k]
        ax.plot([t["lo"], t["hi"]], [y + 0.14] * 2, color=ORANGE, lw=LW * 2, solid_capstyle="butt")
        for xb in (t["lo"], t["hi"]):
            ax.plot([xb, xb], [y + 0.07, y + 0.21], color=ORANGE, lw=LW)
        ax.plot(t["indirect"], y + 0.14, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
        ax.plot(AB[k], y - 0.14, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
        ax.text(t["indirect"], y + 0.30, f"{f(t['indirect'])}", ha="center", va="bottom", fontsize=10, color=INK)
        ax.text(AB[k], y - 0.30, f"a×b = {f(AB[k])}", ha="center", va="top", fontsize=10, color=INK)
        ax.text(-1.55, y, lab, ha="left", va="center", fontsize=10.5, color=INK, fontweight="bold")
    t = TEXT["DAN"]
    ax.text((t["lo"] + t["hi"]) / 2, 0.30, f"[{f(t['lo'])}, {f(t['hi'])}]", ha="center", va="bottom", fontsize=10)
    ax.annotate("", xy=(t["lo"] - 0.02, 0.14), xytext=(t["indirect"] + 0.03, 0.14),
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=1, shrinkA=0, shrinkB=0))
    ax.text((t["lo"] + t["indirect"]) / 2 + 0.15, 0.04, "estimate lies 1.364 below\nthe interval's lower bound",
            ha="center", va="top", fontsize=10, color=INK2)
    ax.axvline(0, color=INK2, lw=0.8, zorder=1)
    ax.text(0.01, 1.62, "null (0)", fontsize=10, color=INK2, ha="left", va="top")
    ax.set_xlim(-1.6, 1.3)
    ax.set_ylim(-0.55, 1.65)
    ax.set_yticks([])
    ax.set_xticks(np.arange(-1.0, 1.25, 0.25))
    ax.set_xticklabels([f(v, 2) for v in np.arange(-1.0, 1.25, 0.25)])
    ax.set_xlabel("Indirect effect (anhedonia → mediator → attention), unstandardized")
    ax.grid(axis="x", color=GRID, lw=0.6)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    h1, = ax.plot([], [], "o-", color=ORANGE, ms=MS, lw=LW * 2, label="Reported in Results text: indirect effect and 95% CI")
    h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed: a × b from the paper's figure path labels (no CI)")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=1, frameon=False, fontsize=10)
    save(fig, *paths)


def cand_B(paths):
    """DAN-only number line: every printed DAN quantity versus the printed CI."""
    fig, ax = plt.subplots(figsize=(W, 3.4))
    t = TEXT["DAN"]
    ax.axvspan(t["lo"], t["hi"], color=ORANGE, alpha=0.18, lw=0)
    ax.plot([t["lo"], t["hi"]], [0, 0], color=ORANGE, lw=LW * 2)
    ax.text((t["lo"] + t["hi"]) / 2, 1.05, "printed 95% CI\n[0.926, 1.079]", ha="center", va="bottom", fontsize=10)
    pts = [  # (value, label, colour, marker, label x, label y)
        (FIG["DAN"]["c"], "c (total), Fig 7", ORANGE, "o", -0.70, 0.55),
        (FIG["DAN"]["a"], "a path, Fig 7", ORANGE, "o", -0.62, -0.55),
        (-0.438, "'indirect' (text)\n= c\u2032 (figure)", ORANGE, "o", -0.40, 1.05),
        (-0.271, "'direct' (text) =\n'bootstrap estimate' (figure)", ORANGE, "o", -0.20, -1.0),
        (AB["DAN"], "a\u00d7b = c \u2212 c\u2032\n(recomputed)", BLUE, "D", 0.10, 0.55),
        (FIG["DAN"]["b"], "b path, Fig 7", ORANGE, "o", 0.36, -0.55),
    ]
    for v, lab, col, mk, tx, ty in pts:
        ax.plot([v, tx], [0, ty * 0.85], color=INK2, lw=0.7, zorder=1)
        ax.plot(v, 0, mk, color=col, ms=MS, mec=BG, mew=1.5, zorder=4)
        ax.text(tx, ty, f"{f(v)}  {lab}", ha="center", va="bottom" if ty > 0 else "top", fontsize=10, color=INK)
    ax.axvline(0, color=INK2, lw=0.8, zorder=1)
    ax.axhline(0, color=GRID, lw=0.8, zorder=0)
    ax.set_xlim(-0.95, 1.25)
    ax.set_ylim(-1.75, 1.85)
    ax.set_yticks([])
    ax.set_xlabel("DAN mediation: every coefficient the paper prints (unstandardized)")
    ax.grid(axis="x", color=GRID, lw=0.6)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported (text or figure labels)")
    h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from reported paths")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False)
    save(fig, *paths)


def box(ax, x, y, w, h, title, body, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
                                fc="white", ec=edge, lw=LW, clip_on=False))
    ax.text(x + 0.02, y + h - 0.035, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
    ax.text(x + 0.02, y + h - 0.11, body, fontsize=10, va="top", color=INK, linespacing=1.5)


def cand_C(paths):
    """Two text panels (Results text vs Figure 7c(C) labels) joined by annotated connectors."""
    fig, ax = plt.subplots(figsize=(W, 4.2))
    ax.set_xlim(-0.02, 1.02)
    ax.set_ylim(0, 1)
    ax.axis("off")
    box(ax, 0.0, 0.52, 0.47, 0.46, "Results, section 3.6 (text)",
        "direct effect = −0.271\nindirect effect = −0.438\n95% CI: [0.926, 1.079]", ORANGE)
    box(ax, 0.53, 0.52, 0.47, 0.46, "Figure 7c(C) (path-diagram labels)",
        "−0.438 (c′)    −0.572 (c)\na = −0.509    b = 0.263\nBootstrap estimate [95% CI]\n= −0.271* [0.926, 1.079]",
        ORANGE)
    for (y0, y1), num in [((0.80, 0.865), "1"), ((0.86, 0.66), "2")]:
        ax.add_patch(FancyArrowPatch((0.40, y0), (0.55, y1), arrowstyle="<->", mutation_scale=12,
                                     color=INK2, lw=1.2))
        ax.text(0.475, (y0 + y1) / 2 + (0.035 if num == "1" else -0.06), num, ha="center", va="center",
                fontsize=10, fontweight="bold", color=INK,
                bbox=dict(boxstyle="circle,pad=0.2", fc=BG, ec=INK2, lw=0.8))
    ax.text(0.5, 0.465, "Conflict 1: −0.438 is called 'indirect' in the text but 'c′' (direct) in the figure.\n"
            "Conflict 2: −0.271 is called 'direct' in the text but the 'bootstrap estimate' in the figure.",
            ha="center", va="top", fontsize=10, color=INK)
    box(ax, 0.0, 0.0, 1.0, 0.30, "Recomputed from the figure's own paths",
        f"a × b = −0.509 × 0.263 = {f(AB['DAN'])}     c − c′ = −0.572 − (−0.438) = "
        f"{f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}\n"
        "Neither −0.438, −0.271 nor −0.134 lies inside [0.926, 1.079].", BLUE)
    save(fig, *paths)


def cand_D(paths):
    """Calculation-flow diagram for the DAN indirect effect."""
    fig, ax = plt.subplots(figsize=(W, 3.0))
    ax.set_xlim(-0.02, 1.02)
    ax.set_ylim(0, 1)
    ax.axis("off")

    def node(x, y, w, txt, col):
        ax.add_patch(FancyBboxPatch((x, y), w, 0.2, boxstyle="round,pad=0.01,rounding_size=0.02",
                                    fc="white", ec=col, lw=LW, clip_on=False))
        ax.text(x + w / 2, y + 0.1, txt, ha="center", va="center", fontsize=10, color=INK)

    def arrow(x0, y0, x1, y1):
        ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.2))

    node(0.00, 0.72, 0.22, "a = −0.509\n(Fig 7c(C))", ORANGE)
    node(0.00, 0.40, 0.22, "b = 0.263\n(Fig 7c(C))", ORANGE)
    node(0.00, 0.08, 0.22, "c = −0.572\nc′ = −0.438", ORANGE)
    node(0.32, 0.56, 0.22, f"a × b = {f(AB['DAN'])}", BLUE)
    node(0.32, 0.08, 0.22, f"c − c′ = {f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}", BLUE)
    arrow(0.22, 0.82, 0.32, 0.68)
    arrow(0.22, 0.50, 0.32, 0.64)
    arrow(0.22, 0.18, 0.32, 0.18)
    node(0.64, 0.56, 0.36, "Text: indirect = −0.438\n95% CI [0.926, 1.079]", ORANGE)
    arrow(0.54, 0.66, 0.64, 0.66)
    arrow(0.43, 0.28, 0.43, 0.56)
    ax.text(0.82, 0.48, "Estimate ≠ a×b (−0.438 vs −0.134)\n"
            "CI contains neither, and is\nentirely positive", ha="center", va="top", fontsize=10, color=INK)
    ax.text(0.45, 0.40, "agree", fontsize=10, color=INK2, ha="left")
    save(fig, *paths)


def cand_E(paths):
    """Sign-check grid: each mediation component's sign, by source."""
    cols = ["X → M\n(a path)", "M → Y\n(b path)", "a×b", "printed\nindirect", "printed\n95% CI"]
    rows = [
        ("DAN\nResults text", ["−\nr = −0.266", "+\nr = 0.419", "−", "−\n−0.438", "+\n[0.926, 1.079]"], [0, 0, 0, 0, 1]),
        ("DAN\nFig 7 caption", ["+\n'positively'", "−\n'negatively'", "−", "", ""], [1, 1, 0, 0, 0]),
        ("DAN\nFig 7 paths", ["−\na = −0.509", "+\nb = 0.263", f"−\n{f(AB['DAN'])}", "", ""], [0, 0, 0, 0, 0]),
        ("C–D\nResults, Fig 3", ["+\na = 0.416", "−\nb = −0.260", f"−\n{f(AB['C-D'])}", "−\n−0.389", "−\n[−0.571, −0.032]"], [0, 0, 0, 0, 0]),
    ]
    fig, ax = plt.subplots(figsize=(W, 4.0))
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    x0, rh = 0.17, 0.165
    widths = [0.16, 0.16, 0.13, 0.13, 0.25]
    lefts = [x0 + sum(widths[:j]) for j in range(5)]
    for j, c in enumerate(cols):
        ax.text(lefts[j] + widths[j] / 2, 0.94, c, ha="center", va="center", fontsize=10, fontweight="bold")
    ax.plot([0, 1], [0.865, 0.865], color=INK2, lw=0.8)
    for i, (lab, vals, bad) in enumerate(rows):
        y = 0.78 - i * rh
        ax.text(0.0, y, lab, ha="left", va="center", fontsize=10, linespacing=1.3)
        ax.plot([0, 1], [y - rh / 2, y - rh / 2], color=GRID, lw=0.8)
        for j, (v, b) in enumerate(zip(vals, bad)):
            if b:
                ax.add_patch(FancyBboxPatch((lefts[j] + 0.008, y - rh / 2 + 0.012), widths[j] - 0.016, rh - 0.024,
                                            boxstyle="round,pad=0,rounding_size=0.01", fc="none", ec=ORANGE, lw=LW))
            ax.text(lefts[j] + widths[j] / 2, y, v, ha="center", va="center", fontsize=10, linespacing=1.3,
                    color=BLUE if j == 2 else INK)
    ax.text(0.0, -0.02, "X = anhedonia, M = mediator, Y = attention.\nOrange outline: a reported sign that conflicts with the other sources.\n"
            "Blue: a×b recomputed from the reported signs or path labels.", fontsize=10, color=INK2, va="bottom")
    save(fig, *paths)


cand_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_dot_interval")])
cand_C([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_C_text_panels")])
cand_B([os.path.join(CAND, "cand_B_number_line")])
cand_D([os.path.join(CAND, "cand_D_calc_flow")])
cand_E([os.path.join(CAND, "cand_E_sign_grid")])
print("\nFigures written to", HERE, "and", CAND)

Paired t-test p-values for one-year cortical thickness change do not match the reported t statistics

Hum Brain Mapp (2009) · doi:10.1002/hbm.20740 · PMC2903209

Summary

The paper's only real-MRI validation (Section IV.B) tests one-year change in mean cortical thickness in 9 patients with Alzheimer's disease (AD) and 13 controls. It reports paired t statistics of −1.200 and −1.495 in AD (df = 8) and −0.274 and +0.175 in controls (df = 12), with p = 0.001, 0.001, 0.009 and 0.001. For those t statistics the two-sided p-values are 0.264, 0.173, 0.789 and 0.864. Table I independently gives t values of about the same size and p-values of 0.19 to 0.84. With the correct p-values none of the four within-group changes is statistically significant. That undoes the paper's statement that both methods "detect a systematic change" in both groups. It also undoes footnote 9's statement that FreeSurfer's increase in controls "is statistically significant", which the paper uses in its comparison with FreeSurfer.

What the paper reports

Section IV.B (Real MRI Data), paragraph 3, sample:

Our dataset includes pairs of scans over a one-year interval from 22 subjects (total of 44 scans), of whom 9 had been diagnosed with Alzheimer’s disease at their first scan, and 13 were age-matched normal subjects.

Section IV.B (Real MRI Data), paragraph 5, the tests in question:

Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD (Alzheimer’s disease) group and in controls. Our method found thickness declined in AD (tpaired = -1.200; p=0.001), and in controls (tpaired = -0.274; p=0.009); FreeSurfer found thickness declined in AD (tpaired = -1.495; p=0.001), and increased in controls (tpaired = +0.175; p=0.001).9

Footnote 9, attached to that sentence (in the PMC text it sits in the back matter):

This increase, which is anatomically not expected, is statistically significant. No increase is found with our proposed method.

Section IV.B, paragraph 6, the baseline/follow-up correlations (the next paragraph):

Our technique gives measures that correlated highly over the 1 year time-interval in AD (p=0.001) and in controls (p=0.009), and so did FreeSurfer (p=0.001 both for AD and for normal subjects).

Section IV.B, paragraph 9, the interpretation:

Both our method and FreeSurfer report a few subjects that show increases (less pronounced with our approach), while on average our proposed method shows decline for both populations (much more significant for the AD subjects, as expected). Even so, FreeSurfer shows a net increase for the control subjects, not very large but still statistically significant. A number of reasons might explain this, including registration issues and segmentation errors, which are minimized in our proposed technique.

Table I ("Statistical Results on Real MRI Data"; relative change in mean thickness over one year), rows for the two groups (columns: proposed algorithm | FreeSurfer; spacing normalised):

Mean change for AD subjects | -1.42 % | -1.15 % Mean change for normal subjects | -0.19 % | +0.20 % SD of change for AD subjects | 3.52 | 2.40 SD of change for normal subjects | 3.39 | 2.82

The Abstract mentions this dataset without giving numbers ("demonstrate its performance on both artificial volumes and real 3D brain MRI data from subjects with Alzheimer’s disease and healthy individuals"). The Conclusion says the authors "presented reasonable results for longitudinal MRI scans of Alzheimer’s disease and normal subjects".

What the paper's own numbers show

Method. The tests compare each subject's baseline and follow-up thickness, so df = n − 1: 8 for AD and 12 for controls. I computed the two-sided p-value for each reported t from the t distribution, along with the one-sided p-value and the |t| that the reported p would require. As an independent check I rebuilt each t from Table I as mean % change / (SD / √n), with a rounding envelope of ±0.005 on the mean and SD. I also used Table I's "all subjects" rows to confirm the group sizes. All values come from recompute.py.txt (scipy.stats).

Quantity Location Reported Recomputed
Proposed method, AD: paired t, df = 8 Sec IV.B para 5 t = −1.200; p = 0.001 two-sided p = 0.264 (one-sided 0.132); p = 0.001 needs |t| ≥ 5.04
FreeSurfer, AD: paired t, df = 8 Sec IV.B para 5 t = −1.495; p = 0.001 two-sided p = 0.173 (one-sided 0.087); p = 0.001 needs |t| ≥ 5.04
Proposed method, controls: paired t, df = 12 Sec IV.B para 5 t = −0.274; p = 0.009 two-sided p = 0.789 (one-sided 0.394); p = 0.009 needs |t| ≥ 3.11
FreeSurfer, controls: paired t, df = 12 Sec IV.B para 5; footnote 9 t = +0.175; p = 0.001; "statistically significant" two-sided p = 0.864 (one-sided 0.432); p = 0.001 needs |t| ≥ 4.32
Same four t, with unlimited df (normal approximation) — — p = 0.230, 0.135, 0.784, 0.861; no choice of df gives the reported p
t from Table I, proposed method, AD (−1.42 %, SD 3.52, n = 9) Table I (text: −1.200) t = −1.210 (rounding range −1.216 to −1.204); p = 0.261
t from Table I, FreeSurfer, AD (−1.15 %, SD 2.40, n = 9) Table I (text: −1.495) t = −1.438 (−1.447 to −1.428); p = 0.189
t from Table I, proposed method, controls (−0.19 %, SD 3.39, n = 13) Table I (text: −0.274) t = −0.202 (−0.208 to −0.196); p = 0.843
t from Table I, FreeSurfer, controls (+0.20 %, SD 2.82, n = 13) Table I (text: +0.175) t = +0.256 (+0.249 to +0.263); p = 0.803
Mean change, all subjects (weighted from group rows) Table I −0.69 % / −0.35 % −0.693 % / −0.352 %
SD of change, all subjects (pooled from group rows) Table I 3.42 / 2.68 3.416 / 2.683
Group separation, (mean normal − mean AD) / SD all Table I 0.36 / 0.50 0.360 / 0.504
p-values of the t-tests vs p-values of the baseline/follow-up correlations Sec IV.B paras 5 and 6 0.001, 0.009, 0.001, 0.001 (para 5); 0.001, 0.009, 0.001, 0.001 (para 6) identical, in the same order; at n = 9 a correlation of r = 0.898 gives two-sided p = 0.001

The Table I checks confirm n = 9 and n = 13. All four printed t values are close to, but slightly outside, the range implied by Table I's rounded means and SDs. They differ from the Table I values by 0.010 and 0.058 (AD) and by 0.072 and 0.081 (controls). The paper does not say whether the tests used percent change or absolute thickness, which could account for those differences. They do not affect the conclusion: every t value the paper's numbers support gives a two-sided p between 0.17 and 0.86.

A separate check on the next paragraph found a smaller inconsistency of the same kind. It does not affect the main finding:

Quantity Location Reported Recomputed
Agreement of change between methods, all subjects (r = 0.572, n = 22) Sec IV.B para 7 p = 0.0028 two-sided p = 0.0054 (one-sided 0.0027)
Agreement of change between methods, controls (r = 0.583, n = 13) Sec IV.B para 7 p = 0.0011 two-sided p = 0.0365 (one-sided 0.0183)
Agreement of change between methods, AD (r = 0.558, n = 9) Sec IV.B para 7 p = 0.0047 two-sided p = 0.1185 (one-sided 0.0592)

Figure

Reported and recomputed p-values for the four paired t-tests of one-year thickness change

Figure 1. The four paired t-tests of one-year change in mean cortical thickness in Section IV.B, on one log-scaled p-value axis. Orange dots are the p-values printed in the paper. Filled blue dots are the two-sided p-values for the printed t statistic and df (8 for 9 AD patients, 12 for 13 controls). Hollow blue rings are the two-sided p-values for t rebuilt from Table I's mean and SD of percent change. The thin vertical line marks p = 0.05. In every row the two recomputed values agree with each other (0.17 to 0.86) and lie well above the printed value, on the other side of the p = 0.05 line. All four changes are non-significant at the conventional level.

The t-test p-values in paragraph 5 and the correlation p-values in paragraph 6 are the same four numbers

Figure 2. The left panel shows the four paired t-tests from Section IV.B paragraph 5, with the printed t, the printed p (orange) and the two-sided p for that t (blue). The right panel shows the p-values that paragraph 6 gives for the baseline/follow-up thickness correlations in the same four method-by-group cells. The printed p-values in the two paragraphs are identical, cell for cell. That identity is a fact about the text. It is consistent with, but does not prove, the explanation that the correlation p-values were entered in place of the t-test p-values (see Explanations considered).

Why it matters

The paper is mainly a methods paper. Its central contribution is defining thickness as a minimum line integral over a gray-matter probability map, together with the algorithm, the synthetic-data validation and the run time, and none of that depends on these p-values. The method-agreement correlation for all 22 subjects (r = 0.572) is also unaffected in substance.

The longitudinal AD/control dataset is the paper's only test on real MRI, and these four tests are its main quantitative result. With the corrected values:

  1. "Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD … group and in controls" is not supported. Neither method detects a statistically significant one-year change in either group (p = 0.17 to 0.86 with n = 9 and n = 13). The mean changes are in the expected direction for AD. With these sample sizes and SDs of change, they do not differ significantly from zero.
  2. The comparative argument against FreeSurfer loses its statistical basis. Footnote 9 and paragraph 9 describe FreeSurfer's mean increase in controls as "statistically significant" and attribute it to "registration issues and segmentation errors, which are minimized in our proposed technique." The increase has p = 0.864 (t = +0.175), or 0.803 from Table I, and is consistent with no change. Neither method's control-group mean differs significantly from zero, so on these numbers the paper cannot conclude that one method is biased and the other is not.
  3. "Much more significant for the AD subjects" compares p-values (0.001 vs 0.009) that are not the p-values of these tests.

The descriptive content of Table I (means, SDs, group separation 0.36 vs 0.50) is internally consistent and not affected.

Explanations considered

Status

Suggested correction

  1. Replace the four p-values in Section IV.B paragraph 5 with the p-values of the paired t-tests. For the printed t statistics these are, two-sided, 0.264 (proposed method, AD), 0.789 (proposed method, controls), 0.173 (FreeSurfer, AD) and 0.864 (FreeSurfer, controls). Also state whether the tests used percent change or absolute thickness, because the printed t values differ slightly from those implied by Table I.
  2. Revise the opening sentence of paragraph 5 ("Both techniques are able to detect a systematic change … both in the AD group and in controls"), footnote 9 ("is statistically significant"), and the paragraph 9 statements "much more significant for the AD subjects" and "not very large but still statistically significant", together with the inference about FreeSurfer's registration and segmentation errors that rests on them.
  3. Check the p-values for the method-agreement correlations in paragraph 7. For r = 0.583 (n = 13) and r = 0.558 (n = 9) the two-sided values are 0.037 and 0.119, not 0.0011 and 0.0047. Also state whether the tests are one- or two-sided (the all-subjects value of 0.0028 matches a one-sided test).
Recomputation script (python recompute.py.txt)
"""Recompute the paired t-test p-values in PMC2903209
(Hum Brain Mapp 2009;30:3188-3199, doi:10.1002/hbm.20740), Section IV.B "Real MRI Data".

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg / fig1.png and fig2.svg / fig2.png next to this file.

Inputs copied from the paper (as archived in PMC and in the publisher's PDF):
  Sec IV.B para 3: 22 subjects scanned twice one year apart; 9 AD, 13 age-matched controls.
  Sec IV.B para 5: four paired t statistics and their p-values.
  Sec IV.B para 6: p-values of the baseline/follow-up thickness correlations.
  Sec IV.B para 7: Pearson r (and p) for agreement of change between the two methods.
  Table I: mean and SD of the one-year percent change in mean thickness, by group.
"""
import os

import numpy as np
from scipy import stats

import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (from the paper)
N_AD, N_NC = 9, 13                          # Sec IV.B para 3
# label: (reported t, n, reported p)        # Sec IV.B para 5
REPORTED = {
    "Proposed method, AD":       (-1.200, N_AD, 0.001),
    "Proposed method, controls": (-0.274, N_NC, 0.009),
    "FreeSurfer, AD":            (-1.495, N_AD, 0.001),
    "FreeSurfer, controls":      (+0.175, N_NC, 0.001),
}
# label: (mean % change, SD of % change, n)  # Table I
TABLE_I = {
    "Proposed method, AD":       (-1.42, 3.52, N_AD),
    "Proposed method, controls": (-0.19, 3.39, N_NC),
    "FreeSurfer, AD":            (-1.15, 2.40, N_AD),
    "FreeSurfer, controls":      (+0.20, 2.82, N_NC),
}
TABLE_I_ALL = {  # method: (mean all, SD all, group separation)
    "Proposed method": (-0.69, 3.42, 0.36),
    "FreeSurfer":      (-0.35, 2.68, 0.50),
}
# Baseline/follow-up correlation p-values, Sec IV.B para 6 (same order as REPORTED)
CORR_P_PARA5 = {
    "Proposed method, AD": 0.001, "Proposed method, controls": 0.009,
    "FreeSurfer, AD": 0.001, "FreeSurfer, controls": 0.001,
}
# Agreement of change between methods, Sec IV.B para 7: (r, n, reported p)
AGREEMENT = {
    "All subjects": (0.572, N_AD + N_NC, 0.0028),
    "Controls":     (0.583, N_NC, 0.0011),
    "AD":           (0.558, N_AD, 0.0047),
}


def p_two(t, df):
    return 2 * stats.t.sf(abs(t), df)


def p_one(t, df):
    return stats.t.sf(abs(t), df)


def t_needed(p, df, sided=2):
    return stats.t.isf(p / sided, df)


def r_pvalue(r, n, sided=2):
    t = r * np.sqrt((n - 2) / (1 - r * r))
    return sided * stats.t.sf(abs(t), n - 2)


def main():
    rows = []
    print("== 1. Reported paired t statistics vs reported p-values (Sec IV.B para 5) ==")
    for lab, (t, n, prep) in REPORTED.items():
        df = n - 1
        p2, p1 = p_two(t, df), p_one(t, df)
        print(f"{lab:27s} t={t:+.3f} df={df:2d}  two-sided p={p2:.3f}  one-sided p={p1:.3f}  "
              f"reported p={prep}  |t| needed for reported p: two-sided {t_needed(prep, df):.2f}, "
              f"one-sided {t_needed(prep, df, 1):.2f}")
        rows.append([lab, t, df, prep, p2])

    print("\n== 2. Even with unlimited df (normal approximation), the reported t cannot give the reported p ==")
    for lab, (t, n, prep) in REPORTED.items():
        print(f"{lab:27s} t={t:+.3f}  two-sided p (df=inf) = {2 * stats.norm.sf(abs(t)):.3f}")

    print("\n== 3. t rebuilt from Table I: mean % change / (SD / sqrt n) ==")
    for i, (lab, (m, sd, n)) in enumerate(TABLE_I.items()):
        t = m / (sd / np.sqrt(n))
        p2 = p_two(t, n - 1)
        # rounding envelope: mean +/- 0.005, SD +/- 0.005
        ts = [(m + dm) / ((sd + ds) / np.sqrt(n)) for dm in (-0.005, 0.005) for ds in (-0.005, 0.005)]
        print(f"{lab:27s} t={t:+.3f} (rounding range {min(ts):+.3f} to {max(ts):+.3f}) df={n - 1:2d}  "
              f"two-sided p={p2:.3f}   reported t={REPORTED[lab][0]:+.3f}  "
              f"|reported - Table I| = {abs(REPORTED[lab][0] - t):.3f}  "
              f"inside rounding range: {min(ts) <= REPORTED[lab][0] <= max(ts)}")
        rows[i].append(t)
        rows[i].append(p2)
    for lab, (t, n, _) in REPORTED.items():
        m_impl = t * TABLE_I[lab][1] / np.sqrt(n)
        print(f"  mean % change implied by reported t and Table I SD, {lab}: {m_impl:+.3f} "
              f"(Table I: {TABLE_I[lab][0]:+.2f})")

    print("\n== 4. Table I internal consistency (confirms n = 9 and 13) ==")
    for meth in TABLE_I_ALL:
        mA, sA, _ = TABLE_I[f"{meth}, AD"]
        mN, sN, _ = TABLE_I[f"{meth}, controls"]
        mall_rep, sdall_rep, gs_rep = TABLE_I_ALL[meth]
        mall = (N_AD * mA + N_NC * mN) / (N_AD + N_NC)
        ss = (N_AD - 1) * sA**2 + (N_NC - 1) * sN**2 + N_AD * (mA - mall)**2 + N_NC * (mN - mall)**2
        sdall = np.sqrt(ss / (N_AD + N_NC - 1))
        gs = (mN - mA) / sdall_rep
        print(f"{meth:16s} mean all={mall:+.3f} (rep {mall_rep:+.2f})  SD all={sdall:.3f} (rep {sdall_rep})  "
              f"group separation={gs:.3f} (rep {gs_rep})")

    print("\n== 5. Exact Wilcoxon signed-rank: smallest attainable two-sided p ==")
    for n in (N_AD, N_NC):
        print(f"n={n}: minimum two-sided p = 2/2^{n} = {2 / 2**n:.4f}")

    print("\n== 6. Para 5 p-values vs para 6 baseline/follow-up correlation p-values ==")
    for lab, (_, _, prep) in REPORTED.items():
        print(f"{lab:27s} para 5 (t-test) p={prep}   para 6 (correlation) p={CORR_P_PARA5[lab]}   "
              f"identical={prep == CORR_P_PARA5[lab]}")
    for n, p in [(N_AD, 0.001), (N_NC, 0.009), (N_NC, 0.001)]:
        tc = t_needed(p, n - 2)
        print(f"  correlation r giving two-sided p={p} at n={n}: r = {tc / np.sqrt(tc**2 + n - 2):.3f}")

    print("\n== 7. Side check: Pearson r p-values for agreement of change (Sec IV.B para 7) ==")
    for lab, (r, n, prep) in AGREEMENT.items():
        print(f"{lab:13s} r={r} n={n}  two-sided p={r_pvalue(r, n):.4f}  one-sided p={r_pvalue(r, n, 1):.4f}  "
              f"reported p={prep}")

    make_figure(rows)
    make_figure2(rows)


def make_figure(rows):
    BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
    ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
    plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
                         "text.color": TXT, "axes.labelcolor": TXT,
                         "xtick.color": SEC, "ytick.color": SEC})
    fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
    ax.set_facecolor(BG)
    y = np.arange(len(rows))[::-1]
    for yi, (lab, t, df, prep, p2, t_tab, p_tab) in zip(y, rows):
        lo, hi = min(prep, p2, p_tab), max(prep, p2, p_tab)
        ax.plot([lo, hi], [yi, yi], color="#b9b7b1", lw=2, zorder=1)
        ax.plot([min(p2, p_tab), max(p2, p_tab)], [yi, yi], color=BLUE, lw=2, zorder=2)
        ax.scatter(prep, yi, s=80, color=ORANGE, zorder=3, edgecolor=ORANGE)
        ax.scatter(p_tab, yi, s=190, facecolor=BG, edgecolor=BLUE, lw=2, zorder=4)
        ax.scatter(p2, yi, s=70, color=BLUE, zorder=5, edgecolor=BLUE)
        ax.annotate(f"reported\np = {prep:g}", (prep, yi), xytext=(0, 9), textcoords="offset points",
                    ha="center", va="bottom", fontsize=10, color=ORANGE, linespacing=0.95)
        ax.annotate(f"{p2:.3f} from t\n{p_tab:.3f} from Table I", (max(p2, p_tab), yi), xytext=(11, 0),
                    textcoords="offset points", ha="left", va="center", fontsize=10, color=BLUE,
                    linespacing=1.0)
    ax.set_yticks(y)
    ax.set_yticklabels([f"{r[0]}\nt = {r[1]:+.3f}, df = {r[2]}" for r in rows], fontsize=10, color=TXT)
    ax.set_xscale("log")
    ax.set_xlim(3e-4, 25)
    ax.set_ylim(-0.6, len(rows) - 0.25)
    ax.axvline(0.05, color=SEC, lw=1.0, zorder=0)
    ax.text(0.05, len(rows) - 0.35, "p = 0.05 ", ha="right", va="top", fontsize=10, color=SEC)
    ax.set_xticks([0.001, 0.01, 0.05, 0.1, 1])
    ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "1"])
    ax.spines["bottom"].set_bounds(3e-4, 1)
    minor = [m * 10.0**e for e in (-4, -3, -2, -1) for m in range(2, 10) if 3e-4 <= m * 10.0**e < 1]
    ax.xaxis.set_minor_locator(matplotlib.ticker.FixedLocator(minor))
    ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_xlabel("Two-sided p-value (log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(SEC)
    ax.tick_params(axis="y", length=0)
    from matplotlib.lines import Line2D
    handles = [
        Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p (Sec IV.B)"),
        Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from reported t"),
        Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
               label="Recomputed from Table I mean, SD, n"),
    ]
    ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=2,
              frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
    fig.tight_layout()
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
    print("\nwrote fig1.svg, fig1.png")


def make_figure2(rows):
    """Two text panels: para 5 t-tests and para 6 correlations share the same four p-values."""
    from matplotlib.patches import FancyBboxPatch
    BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
    ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#cfcdc8"
    fig = plt.figure(figsize=(7.2, 4.1), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 7.2)
    ax.set_ylim(0, 4.1)
    ax.axis("off")
    ax.text(3.6, 3.85, "The four p-values of the t-tests (paragraph 5) repeat, in the same order,\n"
            "the four p-values of the baseline/follow-up correlations (paragraph 6)",
            ha="center", va="center", fontsize=10.5, color=TXT, style="italic", linespacing=1.15)
    for x0, w in ((0.1, 3.9), (4.4, 2.7)):
        ax.add_patch(FancyBboxPatch((x0, 0.55), w, 2.85, boxstyle="round,pad=0,rounding_size=0.08",
                                    fc="white", ec=EDGE, lw=1.2))
    ax.text(0.25, 3.2, "Sec IV.B, paragraph 5", fontsize=12, weight="bold", color=TXT, va="center")
    ax.text(0.25, 2.93, "paired t-tests of one-year change", fontsize=10.5, color=SEC, va="center")
    ax.text(4.55, 3.2, "Sec IV.B, paragraph 6", fontsize=12, weight="bold", color=TXT, va="center")
    ax.text(4.55, 2.93, "baseline vs follow-up correlation", fontsize=10.5, color=SEC, va="center")
    for x, lab, ha in ((0.25, "group", "left"), (2.35, "t", "right"), (2.55, "reported p \u2192\nrecomputed p", "left"),
                       (4.55, "group", "left"), (6.15, "reported p", "left")):
        ax.text(x, 2.6, lab, fontsize=10, color=SEC, ha=ha, va="center")
    for i, (lab, t, df, prep, p2, *_) in enumerate(rows):
        yy = 2.2 - i * 0.43
        short = lab.replace("Proposed method", "Proposed")
        ax.text(0.25, yy, short, fontsize=10.5, color=TXT, va="center")
        ax.text(2.35, yy, f"{t:+.3f}", fontsize=10.5, color=TXT, ha="right", va="center")
        ax.text(2.55, yy, f"{prep:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
        ax.text(3.08, yy, f"\u2192 {p2:.3f}", fontsize=10.5, color=BLUE, weight="bold", va="center")
        ax.text(4.55, yy, short, fontsize=10.5, color=TXT, va="center")
        ax.text(6.15, yy, f"{CORR_P_PARA5[lab]:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
        ax.plot([4.0, 4.4], [yy, yy], color=SEC, lw=1.2, ls=(0, (2, 2)))
    ax.text(0.1, 0.3, "orange = p as printed in the paper", fontsize=10, color=ORANGE, va="center")
    ax.text(0.1, 0.08, "blue = two-sided p for the printed t (df = 8 for AD, 12 for controls), recomputed (scipy)",
            fontsize=10, color=BLUE, va="center")
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
    print("wrote fig2.svg, fig2.png")


if __name__ == "__main__":
    main()

Table 4 brain-volume T values contradict their p values and the table's own means and SDs

Hum Brain Mapp (2022) · doi:10.1002/hbm.25798 · PMC9057088

Summary

Table 4 compares regional brain volumes between 49 healthy children and 53 children with ADHD by a two-independent-samples t test, and in 9 of its 10 rows the printed T value implies a p at least 50 times smaller than the printed p (for whole brain, T = 9.872 sits with p = .995, although no t of that size can have p above .064 at any degrees of freedom). Recomputed from the table's own means and SDs, only the hippocampus row matches its T (7.50 vs 7.401); every other row gives |t| between 0.41 and 1.42, none significant. As a result, lower frontal-lobe volume in ADHD, stated in the Abstract (table p = .008), recomputes to t = 1.42, p = .16, and lower total brain volume, stated in the Results (p < .05), has p = .995 in the table itself and p = .68 on recomputation. The paper's main finding, lower brain iron measured by QSM, rests on other tables and is not affected, but its volume findings and the Discussion built on them are.

What the paper reports

Methods, "Patient information" (group sizes):

53 children with ADHD were included in the study

49 children were included in the study

Methods, "Statistical analysis":

Two independent samples t test was used to compare the brain volume of children between the study group and the control group (p < .05 means the difference is statistically significant).

Table 4, caption and selected rows. The columns are Control group | Study group | T value | p value:

Volume values of brain regions in children [ x ¯ ± s, volume (cm3)]

Frontal | 189.77 ± 27.65 | 182.56 ± 23.53 | 12.871 | .008

Temporal | 138.32 ± 15.25 | 140.47 ± 18.34 | 10.132 | .213

Hippocampus | 12.37 ± 1.28 | 10.24 ± 1.56 | 7.401 | .002

TH | 14.97 ± 2.56 | 14.59 ± 2.39 | 8.601 | .804

Whole brain | 1452.89 ± 179.43 | 1437.96 ± 187.31 | 9.872 | .995

Abstract:

The volume of frontal lobe and hippocampus of children with ADHD was lower than that of healthy children (p < .05).

Results, "Brain volume comparison results":

The brain areas of the frontal lobe and hippocampus of children with ADHD are lower than healthy children (p < .05); the total brain volume of children with ADHD is lower than that of healthy children (p < .05) (Table 4).

Discussion, where the volume results are interpreted:

children with ADHD only have abnormal brain areas such as the frontal lobe and hippocampus (lower than healthy children)

The number of brain areas with abnormal iron content is more than the number of brain areas with abnormal volume

suggesting that the reduced volume of frontal and hippocampal brain regions may be due to decreased iron content in brain regions

the decrease in the volume of the frontal and temporal lobes and other brain regions may be due to

What the paper's own numbers show

Method. For every Table 4 row, I recomputed the two-sample t test from the printed means and SDs with n = 49 (control) and n = 53 (ADHD), using scipy.stats.ttest_ind_from_stats, both pooled-variance (Student) and Welch. I then made three checks. (i) The two-sided p each reported T implies on 100 df, the df of the stated test. (ii) The largest p any reported T could have at all, which occurs at df = 1. (iii) The |t| each reported p implies. As a control, the same procedure applied to Table 1 (age, weight, BMI and serum iron, which the Methods say were tested with the same test on the same groups) reproduces the published T and p values: t = 0.259, −0.904, −0.486 and 0.676 against 0.255, −0.904, −0.486 and 0.676. This confirms the group sizes and that "±" is the SD. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Frontal volume, T / p Table 4 12.871 / .008 t = 1.422, p = .158 (Welch 1.413, p = .161). T = 12.871 implies p = 6.1 × 10⁻²³
Temporal volume, T / p Table 4 10.132 / .213 t = −0.641, p = .523 (ADHD mean is larger). T = 10.132 implies p = 5.1 × 10⁻¹⁷
Hippocampus volume, T / p Table 4 7.401 / .002 t = 7.503, p = 2.6 × 10⁻¹¹. T matches; T = 7.401 implies p = 4.3 × 10⁻¹¹, not .002
Thalamus (TH) volume, T / p Table 4 8.601 / .804 t = 0.775, p = .440. T = 8.601 implies p = 1.1 × 10⁻¹³
Globus pallidus (GP) volume, T / p Table 4 1.354 / .193 t = −1.194, p = .235. The only row where T and p agree with each other (T implies p = .179)
Substantia nigra (SN) volume, T / p Table 4 4.457 / .643 t = −1.158, p = .249. T = 4.457 implies p = 2.2 × 10⁻⁵
Putamen (PU) volume, T / p Table 4 3.871 / .109 t = 1.248, p = .215. T = 3.871 implies p = 1.9 × 10⁻⁴
Red nucleus (RN) volume, T / p Table 4 7.422 / .127 t = 1.128, p = .262. T = 7.422 implies p = 3.9 × 10⁻¹¹
Caudate nucleus (CN) volume, T / p Table 4 3.214 / .566 t = 0.977, p = .331. T = 3.214 implies p = .0018
Whole-brain volume, T / p Table 4 9.872 / .995 t = 0.410, p = .682 (difference 14.93 cm³, SE 36.3). T = 9.872 implies p = 1.9 × 10⁻¹⁶
Largest p possible for |t| = 9.872 Table 4, whole brain .995 .064, even at df = 1
|t| implied by p = .995 on 100 df Table 4, whole brain 9.872 0.006
Whole-brain effect size Table 4 — Cohen's d = 0.081. Reaching t = 9.872 at this d would need about 29 466 children per group
Frontal volume significance Abstract; Results; Table 4 lower in ADHD, p < .05 (table p = .008) not significant, p = .158
Total brain volume significance Results lower in ADHD, p < .05 table p = .995; recomputed p = .682
Hippocampus volume significance Abstract; Results; Table 4 lower in ADHD, p < .05 confirmed, p = 2.6 × 10⁻¹¹
Control check, Table 1 t values Table 1 0.255 / −0.904 / −0.486 / 0.676 0.259 / −0.904 / −0.486 / 0.676

Table 4 therefore has two separate problems. First, the T column and the p column cannot both be correct in 9 of 10 rows. Second, the T column does not follow from the printed means and SDs. In 8 of 10 rows the printed T is 3.1 to 24 times the recomputed |t|. GP is close in size (1.354 vs 1.194) but, like temporal and SN, is printed as positive although the control mean is the smaller one. Only hippocampus agrees (ratio 0.99). The p column is closer to the summary data (all rows except frontal and hippocampus are non-significant in both), but it still does not reproduce them. Frontal is the decisive row: p = .008 is printed, and p = .158 follows from the table's own data.

A related observation. Table 3 (left vs right iron content within the ADHD group, n = 53) shows the same mismatch between its T and p columns. For example, SN has T = 4.872 with p = .993, and no t statistic of 4.872 can have a p above .129. For six of its nine rows, the reported T is also larger than a paired t test on those means and SDs could produce at any left–right correlation. Its conclusion of no left–right difference agrees with the printed p values and with an unpaired recomputation (all |t| ≤ 0.72), so this does not change a finding. It does show that the T columns in more than one table do not correspond to the stated tests. Table 2 (ANCOVA for iron content) is internally consistent: its F values reproduce its p values on F(1, 99), for example F = 1.013 gives p = .317 and F = 0.056 gives p = .813.

Figure

Reported T values in Table 4 against t recomputed from the table's own means and SDs

Figure 1. For each Table 4 row, the orange circle is the printed T value. Table 4 prints all T values without a sign, so they are drawn as positive. The blue diamond is the t statistic recomputed from the same row's means and SDs, with n = 49 controls and 53 children with ADHD, using a pooled-variance two-sample t test; positive means the control group is larger. The shaded band is |t| < 1.98, the region where p > .05 on 100 df. The two markers are offset vertically so that they stay visible where they coincide. Only the hippocampus row agrees. In every other row except GP, the printed T lies far outside the band while the recomputed t lies inside it. That includes frontal, which the Abstract reports as significantly smaller in ADHD.

The three p values for each Table 4 row on one log axis

Figure 2. Three two-sided p values per row on one log axis. The orange circle is the p printed in Table 4. The filled blue diamond is p recomputed from the row's means and SDs. The hollow blue square is the p that the printed T value implies on 100 df. The thin vertical line marks p = .05. If the table were internally consistent, the three markers would coincide in every row. They do so only for GP. For whole brain, the printed T implies p = 1.9 × 10⁻¹⁶, while the printed p is .995 and the recomputed p is .68. For frontal, the printed p (.008) lies on the significant side of .05 and the recomputed p (.158) on the other.

Why it matters

The paper's primary result is lower regional brain iron on quantitative susceptibility mapping in children with ADHD (Tables 2 and 3, and the ROC analysis in Table 5). Table 4 does not bear on it, and this write-up does not question it.

Table 4 is, however, the only source of the paper's volume findings, and two of them do not survive recomputation.

  1. Frontal-lobe volume. The Abstract, the Results and the Discussion all state that frontal volume is lower in ADHD (p < .05). The printed means differ by 7.21 cm³ against a pooled SD of 25.59 cm³ (d = 0.28). This gives t = 1.42, p = .16, which is not significant at the paper's own threshold.
  2. Total brain volume. The Results state that total brain volume is lower in ADHD (p < .05) and cite Table 4. The table itself prints p = .995 for whole brain, and recomputation gives p = .68 for a 1% difference.
  3. The Discussion's interpretation. The Discussion proposes that reduced iron delays development and so reduces the volume of the frontal (and temporal) lobes and hippocampus. It also contrasts the number of regions with abnormal iron against the number with abnormal volume. On the table's own data, only the hippocampal volume difference holds (t = 7.50). The frontal and whole-brain volume differences, and the temporal difference mentioned in the Discussion, have no support in Table 4.

Because the T column does not follow from the summary data in 9 of 10 rows, a reader also cannot use Table 4's T values for meta-analysis or to judge effect sizes. The means and SDs, which reproduce the Table 1 pattern, are the more usable part of the table, assuming they are correct.

Explanations considered

None of these explanations accounts for the table. The published material does not show how the T and p columns were produced.

Status

Suggested correction

  1. Re-run the volume comparisons from the source data with the test stated in the Methods, and replace the T and p columns of Table 4. If the analysis was adjusted (for example for age, sex or intracranial volume), say so in the Methods and the table footnote, and report the matching statistic and degrees of freedom.
  2. If the printed means and SDs are correct, revise the Abstract, the Results ("Brain volume comparison results") and the Discussion: lower frontal-lobe volume (recomputed p = .16) and lower total brain volume (table p = .995, recomputed p = .68) are not supported, and lower hippocampal volume is. Remove or qualify the Discussion passages that attribute reduced frontal and temporal volume to reduced iron.
  3. Check Table 3 the same way. Its T values are inconsistent with its p values, and in six rows with any paired test on the printed means and SDs. Report signed t statistics in Tables 3 and 4, as Table 1 does.
  4. Deposit the data in the figshare record named in the Data Availability Statement, which at the time of checking contained no files.
Recomputation script (python recompute.py.txt)
"""Recompute Table 4 (regional brain volumes) of PMC9057088
(Hum Brain Mapp 2022, doi:10.1002/hbm.25798).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the article as archived in PubMed Central:
  Methods: 53 children with ADHD (study group), 49 healthy controls; brain volume
           compared with a "Two independent samples t test"; data shown as mean ± s.
  Table 1: patient information (used as a control check of n and the SD convention)
  Table 2: iron content, ANCOVA F and p (checked for internal consistency only)
  Table 3: left vs right iron content, ADHD group (T and p columns)
  Table 4: volume of brain regions, mean ± s (cm3), T value, p value
"""
import os

import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
N_C, N_S = 49, 53          # controls, ADHD
DF = N_C + N_S - 2         # 100

# ------------------------------------------------------------------ Table 4 (as printed)
# region, control mean, control s, study mean, study s, reported T, reported p
T4 = [
    ("Frontal",     189.77, 27.65, 182.56, 23.53, 12.871, .008),
    ("Temporal",    138.32, 15.25, 140.47, 18.34, 10.132, .213),
    ("Hippocampus",  12.37,  1.28,  10.24,  1.56,  7.401, .002),
    ("TH",           14.97,  2.56,  14.59,  2.39,  8.601, .804),
    ("GP",            3.62,  0.97,   3.87,  1.13,  1.354, .193),
    ("SN",            0.94,  0.16,   0.99,  0.26,  4.457, .643),
    ("PU",           16.97,  2.89,  16.18,  3.45,  3.871, .109),
    ("RN",            0.72,  0.22,   0.68,  0.13,  7.422, .127),
    ("CN",            9.26,  1.82,   8.96,  1.25,  3.214, .566),
    ("Whole brain", 1452.89, 179.43, 1437.96, 187.31, 9.872, .995),
]

# ------------------------------------------------------------------ Table 1 (control check)
T1 = [  # label, ctrl mean, ctrl s, study mean, study s, reported T, reported p
    ("Age",        8.56, 2.54,  8.44, 2.13,  0.255, .800),
    ("Weight",    28.15, 5.93, 29.23, 6.12, -0.904, .368),
    ("BMI",       17.19, 1.87, 17.38, 2.06, -0.486, .628),
    ("Trace iron", 7.38, 1.13,  7.24, 0.96,  0.676, .501),
]

# ------------------------------------------------------------------ Table 2 (F, p)
T2 = [("Frontal", 24.928, "<.001"), ("Hippocampus", 84.683, "<.001"), ("CN", 87.182, "<.001"),
      ("PU", 39.943, "<.001"), ("GP", 27.913, "<.001"), ("TH", 0.056, ".813"),
      ("Temporal", 1.013, ".317"), ("SN", 28.357, "<.001"), ("RN", 3.504, ".064")]

# ------------------------------------------------------------------ Table 3 (T, p)
T3 = [  # region, left mean, left s, right mean, right s, reported T, reported p
    ("Frontal",      60.18, 13.24,  61.93, 11.67, 1.823, .984),
    ("Temporal",     93.35, 16.41,  94.58, 14.27, 5.257, .092),
    ("Hippocampus",  55.27, 18.32,  56.89, 14.64, 2.874, .143),
    ("TH",          179.23, 23.43, 178.26, 32.54, 3.547, .752),
    ("GP",          393.56, 78.89, 392.47, 89.56, 1.168, .247),
    ("SN",          175.13, 38.03, 176.98, 45.21, 4.872, .993),
    ("PU",           57.65, 13.26,  56.18, 12.22, 2.146, .109),
    ("RN",          114.78, 26.67, 113.89, 33.78, 3.428, .136),
    ("CN",          321.89, 79.27, 322.96, 87.31, 1.224, .143),
]


def p2(t, df):
    """two-sided p for a t statistic"""
    return 2 * stats.t.sf(abs(t), df)


print("=== 0. Control check: Table 1 reproduces with n = 49 / 53 and '±' = SD ===")
for lab, mc, sc, ms, ss, T, p in T1:
    t, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
    print(f"  {lab:10s} recomputed t = {t:6.3f}, p = {pp:.3f}   | reported T = {T:6.3f}, p = {p:.3f}")

print("\n=== 1. Table 4 recomputed from its own means and SDs (two independent samples t test) ===")
hdr = (f"  {'region':12s} {'diff':>8s} {'SE':>7s} {'t_pool':>7s} {'p_pool':>8s} {'t_welch':>7s} "
       f"{'p_welch':>8s} | {'T_rep':>7s} {'p_rep':>6s} {'p(T_rep|df=100)':>15s} {'p(T_rep|df=1)':>13s} "
       f"{'|t| implied by p_rep':>20s}")
print(hdr)
R = []
for r, mc, sc, ms, ss, T, p in T4:
    tp, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
    tw, pw = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=False)
    se = np.sqrt(sc**2 / N_C + ss**2 / N_S)
    pT = p2(T, DF)
    pT1 = p2(T, 1)
    t_imp = stats.t.isf(p / 2, DF)
    R.append(dict(region=r, diff=mc - ms, se=se, t=tp, p=pp, tw=tw, pw=pw, T=T, prep=p, pT=pT,
                  t_imp=t_imp))
    print(f"  {r:12s} {mc - ms:8.2f} {se:7.3f} {tp:7.3f} {pp:8.4f} {tw:7.3f} {pw:8.4f} | "
          f"{T:7.3f} {p:6.3f} {pT:15.2e} {pT1:13.3f} {t_imp:20.3f}")
print("  (diff = control minus ADHD; SE = unpooled standard error of the difference)")
print(f"  Hippocampus recomputed p = {R[2]['p']:.1e}")

print("\n=== 2. Rows where reported T and p contradict each other (df = 100) ===")
for x in R:
    flag = "consistent" if abs(np.log10(x["pT"]) - np.log10(x["prep"])) < 0.3 else "INCONSISTENT"
    print(f"  {x['region']:12s} T={x['T']:6.3f} -> p={x['pT']:.2e}; reported p={x['prep']:.3f}  {flag}")
print("  Largest two-sided p any |t| >= 9.872 can have, at the smallest possible df (1): "
      f"{p2(9.872, 1):.3f}")
print("  Largest two-sided p for |t| = 4.457 (SN) at df = 1: "
      f"{p2(4.457, 1):.3f}")

print("\n=== 3. Significance at alpha = .05: reported vs recomputed ===")
for x in R:
    print(f"  {x['region']:12s} reported p={x['prep']:.3f} ({'sig' if x['prep'] < .05 else 'ns'})  "
          f"recomputed p={x['p']:.3g} ({'sig' if x['p'] < .05 else 'ns'})")

print("\n=== 4. Alternative readings ===")
wb = R[-1]
mc, sc, ms, ss = 1452.89, 179.43, 1437.96, 187.31
print(f"  If '±' were SEM (whole brain): t = {(mc - ms) / np.hypot(sc, ss):.4f}")
sp = np.sqrt(((N_C - 1) * sc**2 + (N_S - 1) * ss**2) / DF)
d = (mc - ms) / sp
print(f"  Whole brain Cohen's d = {d:.4f}; equal n per group needed for t = 9.872 at this d: "
      f"~{2 * (9.872 / d) ** 2:.0f}")
fr = T4[0]
sp_f = np.sqrt(((N_C - 1) * fr[2]**2 + (N_S - 1) * fr[4]**2) / DF)
print(f"  Frontal Cohen's d = {(fr[1] - fr[3]) / sp_f:.3f} (pooled SD {sp_f:.2f} cm3)")
print(f"  Whole-brain difference as % of control mean: {100 * (mc - ms) / mc:.2f}%")
print(f"  Variance-ratio F for whole-brain SDs: {(ss / sc) ** 2:.3f}")
print("  Ratio reported T / recomputed |t|:")
for x in R:
    print(f"    {x['region']:12s} {x['T'] / abs(x['t']):6.2f}  (sign of recomputed t: {'+' if x['t'] > 0 else '-'})")
print("  Could any reordering of the printed columns match? For each recomputed value, "
      "printed values within 15%:")
for x in R:
    tm = [y["region"] for y in R if abs(y["T"] - abs(x["t"])) <= 0.15 * abs(x["t"])]
    pm = [y["region"] for y in R if abs(y["prep"] - x["p"]) <= 0.15 * x["p"]]
    print(f"    {x['region']:12s} T from rows {tm or '-'};  p from rows {pm or '-'}")
print("  Reported T read as F(1,100) (e.g. a variance test), p:")
for x in R:
    print(f"    {x['region']:12s} F={x['T']:6.3f} -> p={stats.f.sf(x['T'], 1, DF):.4f}   reported p={x['prep']:.3f}")
rec_p = [x["p"] for x in R]
rep_p = [x["prep"] for x in R]
rho, _ = stats.spearmanr(rec_p, rep_p)
print(f"  Spearman correlation between recomputed and reported p across the 10 rows: {rho:.2f}")

print("\n=== 5. Table 2 (ANCOVA, iron): F vs p on F(1, 99) ===")
for r, F, p in T2:
    print(f"  {r:12s} F={F:7.3f} -> p={stats.f.sf(F, 1, 99):.3g}   reported p={p}")

print("\n=== 6. Table 3 (left vs right iron, ADHD group n = 53): T vs p ===")
print("  r_needed = left-right correlation a paired t test would need to reach the reported T")
for r, ml, sl, mr, sr, T, p in T3:
    t_ind, _ = stats.ttest_ind_from_stats(ml, sl, 53, mr, sr, 53)
    sd_d = abs(ml - mr) * np.sqrt(53) / T
    r_need = (sl**2 + sr**2 - sd_d**2) / (2 * sl * sr)
    print(f"  {r:12s} T={T:6.3f} reported p={p:.3f}   p(T|df=52)={p2(T, 52):.2e}  "
          f"p(T|df=1)={p2(T, 1):.3f}  independent-samples t={t_ind:6.3f}  r_needed={r_need:.3f}")

# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
                     "axes.edgecolor": INK2, "svg.fonttype": "none"})
labels = [x["region"] for x in R]
y = np.arange(len(R))[::-1]


def style(ax):
    ax.set_facecolor(BG)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.tick_params(axis="y", length=0)
    ax.set_yticks(y)
    ax.set_yticklabels(labels)


# Figure 1: reported T vs recomputed t on one t axis
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
crit = stats.t.isf(0.025, DF)
ax.axvspan(-crit, crit, color="#efeeea", zorder=0, lw=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for yi, x in zip(y, R):
    ax.plot([x["t"], x["T"]], [yi - 0.14, yi + 0.14], color="#c9c8c3", lw=2, zorder=2)
    ax.plot(x["T"], yi + 0.14, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
    ax.plot(x["t"], yi - 0.14, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
    ax.text(14.4, yi, f"{x['T']:.2f} vs {x['t']:.2f}", va="center", ha="left", fontsize=10,
            color=INK2)
ax.text(14.4, len(R) - 0.35, "reported vs\nrecomputed", va="bottom", ha="left", fontsize=10,
        color=INK, fontweight="bold")
ax.text(0, -1.05, f"|t| < {crit:.2f}: p > .05 (df = 100)", ha="center", va="center",
        fontsize=10, color=INK2)
ax.set_xlim(-3, 14)
ax.set_ylim(-1.5, len(R) - 0.4)
ax.set_xlabel("t statistic, control minus ADHD volume")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported T value (Table 4)"),
       Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
              label="t recomputed from Table 4 means and SDs (n = 49 / 53)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.13), ncol=1, frameon=False,
          fontsize=10)
fig.subplots_adjust(left=0.16, right=0.80, top=0.86, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)

# Figure 2: three p values per row on one log axis
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.03, -0.55, "p = .05", ha="right", va="bottom", fontsize=10, color=INK2)
for yi, x in zip(y, R):
    lo = min(x["pT"], x["p"], x["prep"])
    hi = max(x["pT"], x["p"], x["prep"])
    ax.plot([lo, hi], [yi, yi], color="#c9c8c3", lw=2, zorder=2)
    ax.plot(x["pT"], yi + 0.2, "s", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
    ax.plot(x["prep"], yi, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=5)
    ax.plot(x["p"], yi - 0.2, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.set_xlim(1e-24, 3)
ax.set_ylim(-0.7, len(R) - 0.5)
ax.set_xticks([1e-24, 1e-20, 1e-16, 1e-12, 1e-8, 1e-4, 1])
ax.set_xlabel("two-sided p value (log scale)")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p value (Table 4)"),
       Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
              label="p recomputed from Table 4 means and SDs"),
       Line2D([], [], marker="s", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
              label="p implied by the reported T value (t, df = 100)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, 1.2), ncol=1, frameon=False,
          fontsize=10)
fig.subplots_adjust(left=0.16, right=0.97, top=0.82, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Abstract reports low birthweight rising from 9.0% to 10.1%; Table 1 shows 8.2% to 8.3%

Int J Epidemiol (2018) · doi:10.1093/ije/dyy106 · PMC6422062

Summary

The abstract of this four-cohort study from Pelotas, Brazil, states that the prevalence of low birthweight (LBW, < 2500 g) "increased from 9.0% to 10.1%" between 1982 and 2015, and that LBW was "higher for boys". The paper's own Table 1 counts give 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015. No cohort reaches 10.1%, and the abstract's 9.0% matches the 2004 value. The Results, Discussion and Key Messages all describe LBW as stable (P for trend = 0.80, which reproduces exactly from the counts), and Table 2 shows LBW higher for girls in all four cohorts. LBW is one of the two outcomes named in the title, so the abstract reports a change in direction and size, and a sex difference, that the paper's tables and text do not support. The preterm figures in the same abstract sentence check out.

What the paper reports

Abstract, Results (the full sentence):

Preterm births increased from 5.8% (1982) to 13.8% (2015), and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour.

Abstract, Methods:

Four population-based birth cohort studies were carried out in the city of Pelotas in 1982, 1993, 2004 and 2015, with samples ranging from 4231 to 5914 liveborn children. Low birthweight (LBW) was defined as <2500 g

Methods, statistical analysis paragraph:

All analyses were restricted to live, singleton newborns.

Results, first paragraph:

The numbers of liveborn singletons in the four cohorts were 5816, 5168, 4147 and 4164. Birthweight information was missing for 5 (0.09%), 17 (0.33%), 1 (0.02%) and 13 (0.31%) children, respectively.

Table 1, "<2500" row (1982 | 1993 | 2004 | 2015):

<2500 | 8.2 | 9.1 | 9.0 | 8.3

Results, paragraph on Table 1:

The proportion of newborns with low birthweight remained stable at around 8% to 9% throughout the period (P for trend = 0.80).

Results, paragraph on Table 2:

As expected due to the use of the same absolute cutoff for both sexes, prevalence was higher among girls than boys in the four cohorts. There was no evidence of a time trend in low birthweight for either sex.

Table 2, sex rows (% LBW, 95% CI; 1982 | 1993 | 2004 | 2015 | P for linear trend):

Male | 7.2 (6.3; 8.1) | 8.0 (6.9; 9.0) | 8.1 (7.0; 9.3) | 7.6 (6.5; 8.7) | 0.476

Female | 9.3 (8.2; 10.4) | 10.2 (9.1; 11.4) | 9.9 (8.6; 11.2) | 9.1 (7.8; 10.3) | 0.784

Discussion:

One important finding of our study is that—for the whole population—mean birthweight and LBW prevalence remained stable.

Key Messages box (in the PMC archive copy this box follows the Funding statement):

At the population level, mean birthweight remained stable from 1982 to 2015, as did low birthweight.

What the paper's own numbers show

Method. Table 1 gives the number of live singletons in each 500 g birthweight band, plus the number not weighed. For each cohort I added the four bands below 2500 g and divided by the number weighed (band total, which equals the cohort total minus those not weighed). I recomputed the chi-square test for linear trend over the four cohorts (Cochran-Armitage; the Methods name "the chi-square statistic for linear trend"), the 1982–2015 difference with a Wald 95% CI, and Wilson 95% CIs per cohort. As a control I recomputed the preterm percentages from Table 1's gestational-age counts. I compared the abstract's sex statement with Table 2, and searched every LBW value in Tables 1 and 2 for a 9.0 (1982) to 10.1 (2015) series. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
LBW, 1982 Abstract; Table 1 9.0% (abstract); 8.2 (Table 1) (17+40+92+329)/5811 = 478/5811 = 8.23% (95% CI 7.55–8.96)
LBW, 1993 Table 1 9.1 468/5151 = 9.09% (8.33–9.90)
LBW, 2004 Table 1 9.0 372/4146 = 8.97% (8.14–9.88), the value the abstract gives for 1982
LBW, 2015 Abstract; Table 1 10.1% (abstract); 8.3 (Table 1) (15+24+56+250)/4151 = 345/4151 = 8.31% (95% CI 7.51–9.19)
Weighed + not weighed = cohort total Table 1 5816, 5168, 4147, 4164 5811+5, 5151+17, 4146+1, 4151+13: all match
LBW over all singletons, including the unweighed Table 1 — 8.22%, 9.06%, 8.97%, 8.29% (no material change)
Highest LBW in any cohort Table 1 — 9.09% (1993); no cohort rounds to 10.1%
Change 1982 → 2015 Abstract vs Table 1 +1.1 points (9.0 → 10.1), a 12.2% relative rise +0.09 points (95% CI −1.01 to +1.18), a 1.0% relative rise
Linear trend, four cohorts Results P = 0.80 χ² = 0.062, Z = 0.25, P = 0.803 (same with ordinal scores)
LBW by sex Abstract vs Table 2 "higher for boys" (abstract) Table 2: girls higher in all four cohorts, by 2.1, 2.2, 1.8 and 1.5 points (Table 2 P = 0.003, 0.005, 0.044, 0.089)
Any Table 2 row running 9.0 → 10.1 Table 2 — none (girls 9.3 → 9.1; poorest quintile 13.5 → 11.7)
Preterm (< 37 weeks), control Abstract; Table 1 5.8% (1982), 13.8% (2015) 265/4595 = 5.77%; 576/4164 = 13.83%: match

Table 1, the trend P value in the Results, the Discussion and the Key Messages agree with one another. Only the abstract's LBW figures and its sex statement are out of line. The preterm half of the same abstract sentence reproduces exactly.

Figure

The abstract's LBW sentence set against the paper's Table 1, Table 2 and Results text

Figure 1. The abstract's Results sentence on LBW (left, orange: text as printed in the abstract) beside the paper's own tables and Results text (right, blue). The upper connector marks the conflict over direction: the abstract says LBW "increased" from 9.0% to 10.1% (+1.1 points), while Table 1 prints 8.2, 9.1, 9.0 and 8.3 and the Results call LBW stable (P for trend = 0.80). Recomputed from the Table 1 counts, LBW is 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015 (+0.1 points), with trend P = 0.803. The lower connector marks the conflict over sex: the abstract says LBW was higher for boys, while Table 2 gives higher values for girls in every cohort (9.3 vs 7.2 in 1982, 10.2 vs 8.0, 9.9 vs 8.1, 9.1 vs 7.6 in 2015). All values come from the paper or from recompute.py.txt.

LBW by cohort recomputed from Table 1 with 95% CIs, and the abstract's two endpoint values

Figure 2. LBW prevalence by cohort, recomputed from the Table 1 counts (blue circles joined by a line, with Wilson 95% CIs), and the two values the abstract gives for 1982 and 2015 (orange diamonds). The abstract gives endpoints only, so they are not joined. The y-axis starts at zero. The recomputed series is flat at 8.2–9.1% (trend P = 0.80). The abstract's 1982 value (9.0%) lies just above the 1982 CI (7.55% to 8.96%), and its 2015 value (10.1%) lies above the whole 2015 CI (7.51% to 9.19%) and above every cohort.

Why it matters

LBW is one of the two outcomes in the title, and the abstract is the part of the paper most readers, indexers and evidence summaries use. The abstract tells them LBW rose by 1.1 percentage points (12% in relative terms) over 33 years. The paper's own data and its own conclusions say LBW did not change (+0.09 points, P = 0.80). A reader who takes the abstract at face value would cite a rising LBW trend in southern Brazil, which this paper does not show. The sex statement is reversed: the paper explains the female excess as expected "due to the use of the same absolute cutoff for both sexes", and the abstract says the opposite.

The paper's main conclusions are not affected. The preterm increase (5.8% to 13.8%), the stable mean birthweight, the income-group trends in mean birthweight, the adjusted 160 g decline and the caesarean-section interpretation are all stated correctly in the abstract and body. A flat LBW series is also what the abstract's own Background and Conclusions imply ("did not show reductions", "offset the expected improvements"), so the correct figures fit the paper's argument better than the printed ones. The LBW gradients by income and skin colour in the same abstract sentence are broadly consistent with Table 2 and were not examined further here.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the abstract's Results, replace "and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour" with wording that matches Table 1, Table 2 and the Results, for example: "while LBW prevalence remained stable (8.2% in 1982 and 8.3% in 2015; P for trend = 0.80), being higher for girls and for children born to mothers with low income and brown or black skin colour." If the 9.0% and 10.1% figures come from a different population, such as all live births including multiples, the authors could say so, give that population's counts, and explain why it differs from the singleton population that the Methods specify for all analyses.

Recomputation script (python recompute.py.txt)
"""Recompute the low-birthweight (LBW) trend reported in PMC6422062
(Int J Epidemiol 2018, doi:10.1093/ije/dyy106).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/candidate1..5.svg/png) next to this file.

Inputs are copied from the paper as archived in PubMed Central:
  Table 1: birthweight distribution (n per 500 g band, not weighed), <2500 % row,
           gestational-age distribution, number of children
  Table 2: LBW prevalence (95% CI) by sex of the newborn
  Abstract (Results): "LBW prevalence increased from 9.0% to 10.1%, being higher for boys"
  Results text: "remained stable at around 8% to 9% throughout the period (P for trend = 0.80)"
"""
import math
import os

import numpy as np
from scipy.stats import chi2

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (from the paper)
YEARS = [1982, 1993, 2004, 2015]
BANDS = ["<1000", "1000-1499", "1500-1999", "2000-2499",
         "2500-2999", "3000-3499", "3500-3999", ">=4000"]
BW = {  # Table 1, n per birthweight band
    1982: [17, 40, 92, 329, 1361, 2211, 1416, 345],
    1993: [11, 24, 82, 351, 1283, 2040, 1080, 280],
    2004: [24, 29, 70, 249, 1016, 1648, 912, 198],
    2015: [15, 24, 56, 250, 960, 1713, 919, 214],
}
NOT_WEIGHED = {1982: 5, 1993: 17, 2004: 1, 2015: 13}          # Table 1 "Not weighed"
N_CHILDREN = {1982: 5816, 1993: 5168, 2004: 4147, 2015: 4164}  # Table 1 / Results text
T1_LBW = {1982: 8.2, 1993: 9.1, 2004: 9.0, 2015: 8.3}          # Table 1 "<2500" row
GA = {  # Table 1 gestational age: <37, 37-38, 39-41, 42+
    1982: [265, 1007, 2854, 469], 1993: [517, 906, 2661, 518],
    2004: [567, 1244, 2064, 267], 2015: [576, 1562, 1996, 30],
}
T1_PRETERM = {1982: 5.8, 1993: 11.2, 2004: 13.7, 2015: 13.8}
ABS_LBW = (9.0, 10.1)          # Abstract: "increased from 9.0% to 10.1%"
ABS_PRETERM = (5.8, 13.8)      # Abstract: "from 5.8% (1982) to 13.8% (2015)"
RESULTS_P_TREND = 0.80         # Results: "(P for trend = 0.80)"
T2_SEX = {  # Table 2: % LBW (95% CI)
    "Male":   [(7.2, 6.3, 8.1), (8.0, 6.9, 9.0), (8.1, 7.0, 9.3), (7.6, 6.5, 8.7)],
    "Female": [(9.3, 8.2, 10.4), (10.2, 9.1, 11.4), (9.9, 8.6, 11.2), (9.1, 7.8, 10.3)],
}
T2_SEX_P = [0.003, 0.005, 0.044, 0.089]   # Table 2, P for sex difference per cohort
# Every other Table 2 row (1982 and 2015 values), to search for a 9.0 -> 10.1 pair
T2_OTHER = {"Q1": (13.5, 11.7), "Q2": (8.6, 8.0), "Q3": (6.9, 6.8), "Q4": (6.6, 8.4),
            "Q5": (5.7, 6.7), "White": (7.8, 8.5), "Brown": (10.3, 7.7), "Black": (10.3, 8.2),
            "Male": (7.2, 7.6), "Female": (9.3, 9.1)}


def wilson(k, n, z=1.959964):
    p = k / n
    d = 1 + z * z / n
    c = (p + z * z / (2 * n)) / d
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return 100 * (c - h), 100 * (c + h)


# ---------------------------------------------------------------- 1. LBW per cohort
print("== 1. LBW (<2500 g) per cohort, from Table 1 counts ==")
lbw, wt, lbw_pct, lbw_ci = {}, {}, {}, {}
for y in YEARS:
    b = BW[y]
    wt[y] = sum(b)
    lbw[y] = sum(b[:4])
    lbw_pct[y] = 100 * lbw[y] / wt[y]
    lbw_ci[y] = wilson(lbw[y], wt[y])
    print(f"{y}: weighed {wt[y]} (+{NOT_WEIGHED[y]} not weighed = {wt[y] + NOT_WEIGHED[y]}, "
          f"Table 1 N {N_CHILDREN[y]}); LBW n = {'+'.join(map(str, b[:4]))} = {lbw[y]}; "
          f"{lbw[y]}/{wt[y]} = {lbw_pct[y]:.2f}% (Wilson 95% CI {lbw_ci[y][0]:.2f}-{lbw_ci[y][1]:.2f}); "
          f"Table 1 prints {T1_LBW[y]}; over all {N_CHILDREN[y]} children = {100 * lbw[y] / N_CHILDREN[y]:.2f}%")
    assert wt[y] + NOT_WEIGHED[y] == N_CHILDREN[y]
    assert round(lbw_pct[y], 1) == T1_LBW[y]
print(f"Highest cohort value: {max(lbw_pct.values()):.2f}% ({max(lbw_pct, key=lbw_pct.get)})")
print(f"Abstract 9.0% equals the 2004 value ({lbw_pct[2004]:.2f}% -> {round(lbw_pct[2004], 1)}); "
      f"1982 is {lbw_pct[1982]:.2f}%")
print("Cohorts whose LBW rounds to 10.1%:", [y for y in YEARS if round(lbw_pct[y], 1) == 10.1] or "none")

# ---------------------------------------------------------------- 2. change 1982 -> 2015
print("\n== 2. Change 1982 -> 2015 ==")
p1, n1 = lbw[1982] / wt[1982], wt[1982]
p2, n2 = lbw[2015] / wt[2015], wt[2015]
rd = 100 * (p2 - p1)
se = 100 * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
rd_ci = (rd - 1.959964 * se, rd + 1.959964 * se)
print(f"Recomputed change: {rd:+.2f} percentage points (95% CI {rd_ci[0]:+.2f} to {rd_ci[1]:+.2f})")
print(f"Abstract change:   {ABS_LBW[1] - ABS_LBW[0]:+.1f} percentage points (9.0 -> 10.1)")
print(f"Relative change recomputed {100 * (p2 / p1 - 1):+.1f}%; abstract {100 * (ABS_LBW[1] / ABS_LBW[0] - 1):+.1f}%")
print(f"The abstract's +1.1 lies {'inside' if rd_ci[0] <= ABS_LBW[1] - ABS_LBW[0] <= rd_ci[1] else 'outside'} "
      "this interval, so the issue is the printed endpoint values, not sampling error in the difference")

# ---------------------------------------------------------------- 3. trend test
print("\n== 3. Chi-square for linear trend (Cochran-Armitage) over the four cohorts ==")
r = np.array([lbw[y] for y in YEARS], float)
n = np.array([wt[y] for y in YEARS], float)
pbar = r.sum() / n.sum()
for label, x in (("calendar-year scores", np.array(YEARS, float)), ("ordinal scores 0-3", np.arange(4.0))):
    xb = (n * x).sum() / n.sum()
    T = (r * (x - xb)).sum()
    V = pbar * (1 - pbar) * (n * (x - xb) ** 2).sum()
    print(f"{label}: chi2 = {T * T / V:.3f}, Z = {T / math.sqrt(V):.2f}, P = {chi2.sf(T * T / V, 1):.3f} "
          f"(Results prints P = {RESULTS_P_TREND})")
print(f"Pooled LBW over all cohorts: {int(r.sum())}/{int(n.sum())} = {100 * pbar:.2f}%")

# ---------------------------------------------------------------- 4. preterm (control: abstract matches here)
print("\n== 4. Preterm (<37 weeks), from Table 1 gestational-age counts ==")
for y in YEARS:
    g = GA[y]
    print(f"{y}: {g[0]}/{sum(g)} = {100 * g[0] / sum(g):.2f}% (Table 1 prints {T1_PRETERM[y]}); "
          f"gestational age unknown for {N_CHILDREN[y] - sum(g)} of {N_CHILDREN[y]}")
print(f"Abstract preterm endpoints {ABS_PRETERM}: match Table 1 -> "
      f"{(round(100 * GA[1982][0] / sum(GA[1982]), 1), round(100 * GA[2015][0] / sum(GA[2015]), 1)) == ABS_PRETERM}")

# ---------------------------------------------------------------- 5. sex
print("\n== 5. LBW by sex (Table 2) vs abstract 'higher for boys' ==")
for i, y in enumerate(YEARS):
    m, f = T2_SEX["Male"][i][0], T2_SEX["Female"][i][0]
    print(f"{y}: male {m}%, female {f}% -> higher for {'girls' if f > m else 'boys'} "
          f"(difference {f - m:+.1f} points; Table 2 P = {T2_SEX_P[i]})")
print("Girls higher in all four cohorts:",
      all(T2_SEX["Female"][i][0] > T2_SEX["Male"][i][0] for i in range(4)))

# ---------------------------------------------------------------- 6. search for 9.0 -> 10.1
print("\n== 6. Does any Table 1 or Table 2 series run from 9.0 (1982) to 10.1 (2015)? ==")
hits = [k for k, (a, b) in T2_OTHER.items() if abs(a - 9.0) < 0.05 and abs(b - 10.1) < 0.05]
print("Table 2 rows with 1982 = 9.0 and 2015 = 10.1:", hits or "none")
allvals = [v for row in T2_SEX.values() for v, *_ in row] + list(T1_LBW.values()) + \
          [v for pair in T2_OTHER.values() for v in pair]
print("Any Table 1/2 LBW value equal to 10.1:", [v for v in allvals if abs(v - 10.1) < 0.05] or "none")

# ---------------------------------------------------------------- 7. could non-singletons explain 9.0 / 10.1?
print("\n== 7. Abstract Methods: 'samples ranging from 4231 to 5914 liveborn children' (all live births) ==")
print("The paper does not give non-singleton counts per cohort. Minimum number e of extra (non-singleton)")
print("live births, ALL of them LBW, needed to lift each cohort to the abstract value:")
for y, target in ((1982, ABS_LBW[0]), (2015, ABS_LBW[1])):
    t = target / 100
    e_min = (t * wt[y] - lbw[y]) / (1 - t)
    print(f"{y}: to reach {target}% needs e >= {math.ceil(e_min)} extra live births, all LBW")
e82 = 5914 - N_CHILDREN[1982]
need = 0.090 * (wt[1982] + e82) - lbw[1982]
print(f"If the 5914 maximum is the 1982 cohort: {e82} non-singletons; reaching 9.0% needs about "
      f"{need:.0f} of them ({100 * need / e82:.0f}%) to be LBW; all {e82} LBW would give "
      f"{100 * (lbw[1982] + e82) / (wt[1982] + e82):.2f}%")
print("So a different (all-births) denominator cannot be excluded from the paper's numbers alone,")
print("but the Methods restrict all analyses to singletons and no such rates are reported.")


# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
ORANGE_BG, BLUE_BG = "#fdf0ea", "#eaf2fb"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
    "xtick.labelsize": 10, "ytick.labelsize": 10, "legend.fontsize": 10,
})


def save(fig, path_stem):
    fig.savefig(path_stem + ".svg")
    fig.savefig(path_stem + ".png", dpi=200)
    plt.close(fig)


def clean(ax, grid_axis="y"):
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.spines["left"].set_color(GRID)
    ax.spines["bottom"].set_color(SEC)
    ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)
    ax.tick_params(length=0)


def box(ax, x, y, w, h, edge, face, title):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                lw=2, ec=edge, fc=face, transform=ax.transAxes))
    ax.text(x + 0.022, y + h - 0.035, title, transform=ax.transAxes, fontsize=11.5,
            fontweight="bold", va="top", color=TXT)


def txt(ax, x, y, s, **kw):
    kw.setdefault("fontsize", 10.5)
    ax.text(x, y, s, transform=ax.transAxes, va="top", color=kw.pop("color", TXT), linespacing=1.4, **kw)


# ---- candidate 1: two text panels joined by annotated connectors (abstract vs body)
def cand_text_panels(stem):
    fig = plt.figure(figsize=(7.2, 5.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    txt(ax, 0.03, 0.97, "One abstract sentence, two conflicts with the paper's own table and text",
        fontsize=12, fontweight="bold")
    txt(ax, 0.03, 0.915, "LBW = birthweight < 2500 g; live singletons; Pelotas birth cohorts 1982-2015",
        fontsize=10, color=SEC)
    L, R, W, B, H = 0.03, 0.59, 0.38, 0.12, 0.74
    box(ax, L, B, W, H, ORANGE, ORANGE_BG, "Abstract, Results")
    box(ax, R, B, W, H, BLUE, BLUE_BG, "Table 1, Table 2, Results")
    y1, y2 = 0.74, 0.40
    txt(ax, L + 0.022, y1, "\"... and LBW prevalence\nincreased from 9.0%\nto 10.1%,")
    txt(ax, L + 0.022, y2, "being higher for boys and\nfor children born to\nmothers with low income\n"
                           "and brown or black skin\ncolour.\"")
    txt(ax, R + 0.022, y1, f"Table 1, '<2500' row (%):\n1982 {T1_LBW[1982]}  1993 {T1_LBW[1993]}\n"
                           f"2004 {T1_LBW[2004]}  2015 {T1_LBW[2015]}\n"
                           "Results: \"remained stable\nat around 8% to 9% ...\n(P for trend = 0.80)\"")
    txt(ax, R + 0.022, y2, "Table 2, LBW % girls vs boys:\n"
                           + "\n".join(f"{y}  {T2_SEX['Female'][i][0]} vs {T2_SEX['Male'][i][0]}"
                                       for i, y in enumerate(YEARS)))
    for yy, lab in ((y1 - 0.06, "'increased'\n+1.1 pts\nvs stable\n+0.1 pts"),
                    (y2 - 0.06, "'boys'\nvs girls\nhigher in\nall four")):
        ax.add_patch(FancyArrowPatch((L + W, yy), (R, yy), transform=ax.transAxes,
                                     arrowstyle="<->", mutation_scale=14, lw=2, color=SEC))
        ax.text((L + W + R) / 2, yy - 0.025, lab, transform=ax.transAxes, ha="center", va="top",
                fontsize=10, color=TXT, linespacing=1.25)
    txt(ax, 0.03, 0.085, "Orange: text as printed in the abstract. Blue: the paper's tables and Results text;\n"
                         f"Table 1 recomputes from its counts ({lbw[1982]}/{wt[1982]} = {lbw_pct[1982]:.1f}%, "
                         f"{lbw[2015]}/{wt[2015]} = {lbw_pct[2015]:.1f}%; trend P = 0.803).",
        fontsize=10, color=SEC)
    save(fig, stem)


# ---- candidate 2: LBW by cohort, recomputed with 95% CI, vs the abstract's two endpoints
def cand_trend(stem):
    fig, ax = plt.subplots(figsize=(7.2, 4.2))
    fig.subplots_adjust(left=0.10, right=0.76, top=0.84, bottom=0.12)
    clean(ax)
    xs = np.array(YEARS)
    ys = np.array([lbw_pct[y] for y in YEARS])
    lo = ys - np.array([lbw_ci[y][0] for y in YEARS]); hi = np.array([lbw_ci[y][1] for y in YEARS]) - ys
    ax.errorbar(xs, ys, yerr=[lo, hi], fmt="o-", color=BLUE, lw=2, ms=8, capsize=0, elinewidth=2,
                mec=BG, mew=1.5, zorder=3)
    for x_, y_, l0 in zip(xs, ys, lo):
        ax.text(x_, y_ - l0 - 0.2, f"{y_:.1f}", color=TXT, fontsize=10, va="top", ha="center")
    ax.plot([1982, 2015], ABS_LBW, "D", color=ORANGE, ms=9, mec=BG, mew=1.5, zorder=4, ls="none")
    for x_, v in zip((1982, 2015), ABS_LBW):
        ax.text(x_ - 1.0, v, f"{v}", fontsize=10, va="center", ha="right")
    ax.text(2016.8, ABS_LBW[1] + 0.3, "Abstract: \"increased\nfrom 9.0% to 10.1%\"\n(endpoints only)",
            fontsize=10, va="center")
    ax.text(2016.8, lbw_pct[2015] - 0.5, "Table 1 counts,\nrecomputed, 95% CI;\ntrend P = 0.80",
            fontsize=10, va="top")
    ax.set_ylim(0, 12); ax.set_xlim(1978, 2017)
    ax.set_xticks(YEARS)
    ax.set_ylabel("Low birthweight (%)")
    fig.text(0.10, 0.955, "Low birthweight by cohort: the abstract vs the paper's own counts",
             fontsize=12, fontweight="bold", va="top")
    fig.text(0.10, 0.9, "Live singletons weighed at birth; the y-axis starts at zero", fontsize=10,
             color=SEC, va="top")
    ax.legend(handles=[Line2D([], [], color=ORANGE, marker="D", ls="none", ms=9, label="Reported in abstract"),
                       Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed from Table 1")],
              loc="lower left", frameon=False, ncol=2)
    save(fig, stem)


# ---- candidate 3: dumbbell of the 1982 -> 2015 change
def cand_dumbbell(stem):
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    fig.subplots_adjust(left=0.25, right=0.96, top=0.78, bottom=0.2)
    clean(ax, "x")
    rows = [("Abstract", ABS_LBW, ORANGE, "D", "change +1.1 points"),
            ("Table 1, recomputed", (lbw_pct[1982], lbw_pct[2015]), BLUE, "o",
             f"change {rd:+.1f} points\n(95% CI {rd_ci[0]:+.1f} to {rd_ci[1]:+.1f})")]
    for i, (lab, (a, b), c, m, ch) in enumerate(rows):
        yy = 1 - i
        ax.plot([a, b], [yy, yy], color=c, lw=2, zorder=2)
        ax.plot([a], [yy], m, color=c, ms=9, mfc=BG, mew=2, zorder=3)
        ax.plot([b], [yy], m, color=c, ms=9, mec=BG, mew=1.5, zorder=4)
        ax.text(min(a, b) - 0.25, yy, f"1982 {a:.1f}", ha="right", va="center", fontsize=10)
        ax.text(max(a, b) + 0.25, yy, f"2015 {b:.1f}", ha="left", va="center", fontsize=10)
        ax.text(0.3, yy, ch, ha="left", va="center", fontsize=10, color=SEC)
    ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=TXT)
    ax.set_xlim(0, 12.5); ax.set_ylim(-0.6, 1.6)
    ax.set_xlabel("Low birthweight (%); the axis starts at zero")
    fig.text(0.03, 0.95, "Change in low birthweight, 1982 to 2015", fontsize=12, fontweight="bold", va="top")
    fig.text(0.03, 0.88, "Open marker 1982, filled marker 2015", fontsize=10, color=SEC, va="top")
    save(fig, stem)


# ---- candidate 4: sex panel (Table 2) vs the abstract's 'higher for boys'
def cand_sex(stem):
    fig, ax = plt.subplots(figsize=(7.2, 4.0))
    fig.subplots_adjust(left=0.10, right=0.78, top=0.78, bottom=0.12)
    clean(ax)
    off = {"Female": 0.8, "Male": -0.8}
    style = {"Female": dict(color=TXT, marker="o"), "Male": dict(color=SEC, marker="s")}
    for sx, rows in T2_SEX.items():
        xs = np.array(YEARS) + off[sx]
        v = np.array([r_[0] for r_ in rows]); l_ = np.array([r_[1] for r_ in rows]); h_ = np.array([r_[2] for r_ in rows])
        ax.errorbar(xs, v, yerr=[v - l_, h_ - v], fmt=style[sx]["marker"], color=style[sx]["color"],
                    ms=8, elinewidth=2, capsize=0, mfc=style[sx]["color"] if sx == "Female" else BG, mew=2)
        ax.text(2017.5, v[-1], f"{'Girls' if sx == 'Female' else 'Boys'} (Table 2)", va="center", fontsize=10)
    ax.set_xticks(YEARS); ax.set_xlim(1978, 2017); ax.set_ylim(0, 12)
    ax.set_ylabel("Low birthweight (%), 95% CI")
    fig.text(0.10, 0.96, "Table 2: LBW is higher for girls in all four cohorts", fontsize=12,
             fontweight="bold", va="top")
    fig.text(0.105, 0.885, "Abstract: \"LBW prevalence ... being higher for boys\"", fontsize=10.5,
             va="top", bbox=dict(boxstyle="round,pad=0.3", fc=ORANGE_BG, ec=ORANGE, lw=1.5))
    ax.text(1978.3, 1.6, "P, girls vs boys:", fontsize=10, color=SEC, va="center")
    for i, y in enumerate(YEARS):
        ax.text(y, 0.8, f"{T2_SEX_P[i]}", ha="center", fontsize=10, color=SEC, va="center")
    save(fig, stem)


# ---- candidate 5: calculation flow from Table 1 counts to the percentages
def cand_flow(stem):
    fig = plt.figure(figsize=(7.2, 4.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.text(0.03, 0.955, "From Table 1 counts to the LBW percentage", fontsize=12, fontweight="bold", va="top")
    cols = [(0.06, "Table 1 counts\n< 2500 g"), (0.32, "Weighed\nnewborns"),
            (0.53, "LBW %\nrecomputed"), (0.75, "Abstract")]
    for x, h in cols:
        ax.text(x, 0.85, h, fontsize=10.5, fontweight="bold", va="top", color=TXT)
    for j, y in enumerate((1982, 2015)):
        yy = 0.58 - j * 0.27
        ax.text(0.03, yy + 0.1, str(y), fontsize=10.5, fontweight="bold", color=SEC)
        ax.text(0.06, yy, "+".join(map(str, BW[y][:4])) + f"\n= {lbw[y]}", fontsize=10.5, va="center")
        ax.text(0.32, yy, f"{N_CHILDREN[y]} - {NOT_WEIGHED[y]}\n= {wt[y]}", fontsize=10.5, va="center")
        ax.text(0.53, yy, f"{lbw[y]}/{wt[y]}\n= {lbw_pct[y]:.2f}%", fontsize=10.5, va="center",
                bbox=dict(boxstyle="round,pad=0.35", fc=BLUE_BG, ec=BLUE, lw=2))
        ax.text(0.75, yy, f"{ABS_LBW[j]}%", fontsize=12, va="center",
                bbox=dict(boxstyle="round,pad=0.35", fc=ORANGE_BG, ec=ORANGE, lw=2))
        ax.text(0.71, yy, "vs", fontsize=10.5, va="center", ha="center", color=SEC)
        for x0, x1 in ((0.255, 0.305), (0.465, 0.515)):
            ax.annotate("", xy=(x1, yy), xytext=(x0, yy), xycoords="axes fraction",
                        arrowprops=dict(arrowstyle="-|>", color=SEC, lw=1.5))
        ax.text(0.86, yy, f"{ABS_LBW[j] - lbw_pct[y]:+.1f} pts\nvs Table 1", fontsize=10, va="center", color=SEC)
    ax.text(0.03, 0.12, f"Table 1 itself prints {T1_LBW[1982]} and {T1_LBW[2015]}. The abstract's 9.0% "
            f"equals the 2004 value\n({lbw[2004]}/{wt[2004]} = {lbw_pct[2004]:.2f}%); no cohort reaches 10.1% "
            f"(highest: {max(lbw_pct.values()):.1f}% in 1993).", fontsize=10, color=SEC, va="top", linespacing=1.4)
    save(fig, stem)


builders = [cand_text_panels, cand_trend, cand_dumbbell, cand_sex, cand_flow]
for i, fn in enumerate(builders, 1):
    fn(os.path.join(CAND, f"candidate{i}"))
# Default selection for writeup.md until a design is chosen: fig1 = candidate 1, fig2 = candidate 2
cand_text_panels(os.path.join(HERE, "fig1"))
cand_trend(os.path.join(HERE, "fig2"))
print("\nWrote fig1, fig2 and candidates/candidate1-5 (.svg, .png) in", HERE)

Figure 6D calls a TRPV4 knockout effect significant at p = 0.06706, above the 0.05 threshold

J Neurosci (2018) · doi:10.1523/JNEUROSCI.0897-18.2018 · PMC6181316

Summary

The Figure 6D legend says that conventional TRPV4 knockout (TRPV4KO) mice "showed significantly less photoreceptor cell death than WT mice" 24 h after retinal detachment. It marks the comparison with an asterisk and prints p = 0.06706. The Methods state one decision rule for these TUNEL comparisons: "p < 0.05 was considered significant". Since 0.06706 > 0.05, the comparison is not significant under the paper's own rule. Nothing in the paper (a one-sided test, a different threshold, a correction notice) reconciles the two. Of the 16 exact p-values printed in the paper, this is the only one above 0.05. The same comparison is the first result stated in the Abstract (a ~50% reduction in apoptotic photoreceptors). The paper's main conclusion still has other support, including the Müller-glia conditional knockout in the same legend (p = 0.020863, which is significant and consistent).

What the paper reports

Methods, "Statistical analysis":

The Mann–Whitney U test was used to compare unpaired values of TUNEL-positive cell density and proinflammatory cytokine concentration. The data analyses were performed using Excel 2016 (Microsoft) with add-in software Statce14@. p < 0.05 was considered significant.

Figure 6 legend, panel D:

Conventional TRPV4KO mice showed significantly less photoreceptor cell death than WT mice 24 h after retinal detachments (*p = 0.06706, n = 7 or 8 eyes). The TRPV4CKO mice showed significantly less photoreceptor cell death than WT mice (**p = 0.020863, n = 6 eyes). The TRPV4CKO results were perfectly matched with those in TRPV4KO, indicating that Müller glial TRPV4 activation accelerates photoreceptor cell death in RD.

Abstract:

After RD, the number of apoptotic photoreceptors was reduced by ∼50% in TRPV4KO mice relative to wild-type mice, indicating the possible involvement of TRPV4 activation in RD-induced photoreceptor cell death.

Results, TUNEL paragraph. This passage gives the size of the effect but does not call the KO result significant:

The number of dead photoreceptors in TRPV4KO mice was 50% less than that for WT mice (Fig. 6D).

Consistent with those previous reports and our data (Figs. 1–5), the number of dead photoreceptors in TRPV4CKO mice was 50% less than that for WT mice, and the results were perfectly matched with the result in the conventional TRPV4KO (Fig. 6D).

Discussion, second paragraph:

Indeed, our Müller glia-specific conditional KO mice can significantly reduce the RD-induced photoreceptor cell death (Fig. 6), and the reduction ratio of dead photoreceptors was perfectly the same as the conventional TRPV4KO mice (Fig. 6).

What the paper's own numbers show

Method. The paper gives no per-eye TUNEL counts, group summaries or U statistics, so the test cannot be rerun. Two checks are possible. First, compare each printed p with the threshold stated in the Methods. Second, ask whether 0.06706 is a plausible two-sided Mann–Whitney p for the stated sample sizes, and so whether it could be a misprint of a smaller value. For the second check I enumerated the exact null distribution of U for n = 7 vs 8 (6435 rank arrangements, assuming no ties), and for 7 vs 7 and 8 vs 8 because the legend says "7 or 8 eyes". I also computed the normal approximation with and without continuity correction. For context I collected every exact p-value printed in the paper. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed
Significance threshold Methods, Statistical analysis "p < 0.05 was considered significant" α = 0.05
WT vs conventional TRPV4KO, TUNEL+ photoreceptors at 24 h Fig. 6D legend "significantly less", *p = 0.06706, n = 7 or 8 eyes 0.06706 > 0.05 by 0.01706 (1.34 × the threshold); not significant under the stated rule. Equivalent |z| = 1.831, against 1.960 needed
WT vs TRPV4CKO Fig. 6D legend "significantly less", **p = 0.020863, n = 6 eyes 0.020863 < 0.05; significant, consistent
Other exact p-values in the paper legends of Figs. 4, 5, 7, 8, 9, 10 14 values, from 0.000116 to 0.012851 all < 0.05. Of the 16 printed exact p-values, 0.06706 is the only one ≥ 0.05; the next largest is 0.020863
Hypothetical one-sided reading not stated in the paper — 0.06706 / 2 = 0.03353 (would be < 0.05). If 0.06706 were already one-sided, the two-sided p would be 0.13412
Exact two-sided Mann–Whitney p near the printed value, n = 7 vs 8, no ties recomputation 0.06706 U = 11: 0.05408; U = 12: 0.07211. The printed value lies between them, and both are above 0.05
Normal approximations closest to 0.06706, n = 7 vs 8 recomputation 0.06706 U = 12: 0.06408 (no continuity correction), 0.07285 (with); U = 11 with correction: 0.05620
Same check for n = 7 vs 7 and 8 vs 8 recomputation 0.06706 7 vs 7: exact 0.05303 / 0.07284 (U = 9 / 10); 8 vs 8: exact 0.06496 / 0.08298 (U = 14 / 15)
Exact p near a misprint value of 0.006706, n = 7 vs 8 recomputation — U = 5: 0.00591; U = 6: 0.00932. A value near 0.0067 is also within the attainable range, so a misprint cannot be excluded

The printed 0.06706 is not itself an exact no-ties value for any of these sample sizes. Ties in the TUNEL densities, or a normal approximation as statistics add-ins commonly use, would give values between the exact steps. As printed, 0.06706 falls where an ordinary two-sided Mann–Whitney result for about 7–8 eyes per group would fall when the rank separation is just short of significance (U = 11–12 of a possible 56 for 7 vs 8).

Figure

Reported Fig. 6D p-values against the paper's stated threshold and the neighbouring exact Mann-Whitney values

On a linear p-value axis starting at 0, the two orange dots are the p-values printed in the Figure 6D legend. Both comparisons are described there as "significantly less". The vertical line is the threshold stated in the Methods (p < 0.05), and the shaded band is the region the paper counts as significant. WT vs TRPV4CKO (p = 0.020863) lies inside it. WT vs conventional TRPV4KO (p = 0.06706) lies outside it. The blue open squares are recomputed: the two exact two-sided Mann–Whitney p-values on either side of the printed value for n = 7 vs 8 eyes with no ties (U = 11: 0.0541; U = 12: 0.0721). Both are above 0.05. Taken as printed, the knockout result is a two-sided value just short of significance.

All 16 exact p-values printed in the paper, on a log axis, with the 0.05 threshold

All 16 exact p-values printed in the paper's figure legends, sorted and shown on a log scale. Every open dot is a comparison the legends call significant, and each lies below 0.05. The filled dot (Fig. 6D, WT vs TRPV4KO, p = 0.06706) is the only printed value above the threshold, and the legend calls it "significantly less" all the same. Values are copied from the legends; panel letters follow the legends' own labels.

Why it matters

The WT vs conventional TRPV4KO comparison is the first result in the Abstract ("reduced by ∼50% in TRPV4KO mice relative to wild-type mice"). It is also one half of an argument made twice in the paper: the CKO effect is "perfectly matched" with, or "perfectly the same as", the conventional KO effect, and this match is used to assign the effect to Müller glial TRPV4. Under the paper's criterion, the KO side of that comparison is a non-significant trend (p = 0.067, about 7–8 eyes per group), not an established reduction. The direction of the effect does not change. The 50% figure cannot be checked because no group values are given.

The paper's overall conclusion, that Müller glial TRPV4 contributes to photoreceptor death after retinal detachment, is weakened but not overturned. In the same legend, the Müller-glia/astrocyte conditional knockout reaches significance (p = 0.020863, n = 6). The agonist GSK increases photoreceptor death (Fig. 7A, p = 0.002622). The antagonist HC reduces it (Fig. 7B, p = 0.009823). The MCP-1 and macrophage experiments (Figs. 8–10) point the same way. Readers who rely on the global-knockout result as an independent replication, or quote the Abstract's ~50% reduction as statistically established, are relying on a comparison that the paper's own numbers do not support at α = 0.05.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The Figure 6D legend should be made consistent with the Methods. There are two possibilities:

  1. If p = 0.06706 is correct: change the legend to describe the KO vs WT difference as a non-significant trend, for example "TRPV4KO mice showed a trend toward less photoreceptor cell death than WT mice (p = 0.067, n = 7 or 8 eyes)". Remove the asterisk from the figure. Qualify the Abstract's "reduced by ∼50%" sentence and the "perfectly matched" / "perfectly the same" statements to match.
  2. If the printed value is a misprint: give the correct p-value.

In either case, it would help readers if the authors stated the group sizes for each genotype (which group had 7 eyes and which had 8), the test used for Fig. 6D, whether it was one- or two-sided and exact or approximate, and the group summaries (median and range, or the per-eye values) behind the "50% less" figure.

Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the Figure 6D significance claim in PMC6181316
(J Neurosci 2018, doi:10.1523/JNEUROSCI.0897-18.2018).

Run:  python recompute.py.txt              # prints every number, writes fig1/fig2 (.svg, .png)
      python recompute.py.txt --candidates # also writes five candidate figure designs
                                           # to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.

Inputs are copied from the paper as archived in PMC (PMC6181316):
  Methods, "Statistical analysis": Mann-Whitney U test for TUNEL-positive cell density;
      "p < 0.05 was considered significant."
  Figure 6D legend: WT vs conventional TRPV4KO, "*p = 0.06706, n = 7 or 8 eyes";
      WT vs TRPV4CKO, "**p = 0.020863, n = 6 eyes". Both called "significantly less".
  Every other exact p-value printed in the figure legends (Figs. 4, 5, 7, 8, 9, 10).
The paper gives no per-eye values, so the test itself cannot be rerun. What can be
checked is (1) the printed p against the stated threshold and (2) whether the printed
p is a plausible Mann-Whitney value for the stated sample sizes. For (2) the exact
null distribution of U is enumerated (no ties assumed) and the normal approximation
is computed with and without continuity correction.
"""
import os
import sys
from itertools import combinations

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ----------------------------------------------------------------- inputs (from the paper)
ALPHA = 0.05                 # Methods, Statistical analysis
P_KO = 0.06706               # Fig. 6D legend, WT vs conventional TRPV4KO, "n = 7 or 8 eyes"
P_CKO = 0.020863             # Fig. 6D legend, WT vs TRPV4CKO, "n = 6 eyes"

# every exact p-value printed in the paper (location, label, p, significance wording)
ALL_P = [
    ("Fig. 4C", "pressure-evoked currents", 0.00056),
    ("Fig. 5C", "TRPV4-evoked currents", 0.0058),
    ("Fig. 6D", "WT vs TRPV4KO, TUNEL", P_KO),
    ("Fig. 6D", "WT vs TRPV4CKO, TUNEL", P_CKO),
    ("Fig. 7A", "GSK vs control, TUNEL", 0.002622),
    ("Fig. 7B", "HC vs control, TUNEL", 0.009823),
    ("Fig. 8A", "MCP-1 mRNA, RD vs control", 0.000116),
    ("Fig. 8A", "MCP-1 mRNA, vs WT RD", 0.000232),
    ("Fig. 8B", "MCP-1 protein", 0.0010509),
    ("Fig. 8C", "GSK vs control", 0.000247),
    ("Fig. 8C", "Hypo vs control", 0.00944),
    ("Fig. 8C", "GSK+HC vs GSK", 0.001683),
    ("Fig. 8C", "Hypo+HC vs Hypo", 0.00092),
    ("Fig. 9A", "anti-MCP-1 vs PBS, TUNEL", 0.012611),
    ("Fig. 9A", "anti-MCP-1 vs IgG, TUNEL", 0.012851),
    ("Fig. 10", "CD11b+ cells", 0.00632),
]

# ----------------------------------------------------------------- helpers
def exact_U(n1, n2):
    """All C(n1+n2, n1) values of U for group 1 under H0, no ties."""
    N = n1 + n2
    return np.array([sum(i + 1 for i in c) - n1 * (n1 + 1) / 2
                     for c in combinations(range(N), n1)])


def exact_p2(Us, u):
    return min(1.0, 2 * np.mean(Us <= u))


def normal_p2(u, n1, n2, cc=True):
    mu = n1 * n2 / 2
    sd = np.sqrt(n1 * n2 * (n1 + n2 + 1) / 12)
    z = (abs(u - mu) - (0.5 if cc else 0.0)) / sd
    return 2 * stats.norm.sf(z), z


# ----------------------------------------------------------------- 1. threshold check
print("1. Printed p-values against the stated threshold (p < 0.05)")
print(f"   WT vs TRPV4KO : p = {P_KO}  -> below 0.05? {P_KO < ALPHA}; "
      f"exceeds 0.05 by {P_KO - ALPHA:.5f} ({P_KO / ALPHA:.3f} x the threshold)")
print(f"   WT vs TRPV4CKO: p = {P_CKO} -> below 0.05? {P_CKO < ALPHA}")
print(f"   z implied by a two-sided p of {P_KO}: {stats.norm.isf(P_KO / 2):.4f} "
      f"(1.9600 needed for p = 0.05)")

# ----------------------------------------------------------------- 2. other p-values in the paper
print("\n2. All exact p-values printed in the paper")
above = [r for r in ALL_P if r[2] >= ALPHA]
for loc, lab, p in sorted(ALL_P, key=lambda r: r[2]):
    print(f"   {loc:8s} {lab:28s} p = {p:<10g} {'>= 0.05' if p >= ALPHA else '< 0.05'}")
print(f"   {len(ALL_P)} printed p-values; {len(above)} at or above 0.05: "
      f"{[(r[0], r[2]) for r in above]}")
print(f"   largest p other than the KO comparison: "
      f"{max(r[2] for r in ALL_P if r[2] != P_KO)}")

# ----------------------------------------------------------------- 3. hypothetical one-sided readings
print("\n3. One-sided readings (the paper states none)")
print(f"   if 0.06706 were two-sided and halved: {P_KO / 2:.5f} (< 0.05: {P_KO / 2 < ALPHA})")
print(f"   if 0.06706 were already one-sided, two-sided p = {min(1, 2 * P_KO):.5f}")

# ----------------------------------------------------------------- 4. attainable Mann-Whitney p-values
print("\n4. Mann-Whitney p-values attainable for the stated sample sizes (no ties)")
EXACT = {}
for n1, n2 in [(7, 8), (7, 7), (8, 8)]:
    Us = exact_U(n1, n2)
    EXACT[(n1, n2)] = Us
    print(f"   n = {n1} vs {n2}: {len(Us)} rank arrangements")
    for u in range(8, 16):
        pe = exact_p2(Us, u)
        pc, _ = normal_p2(u, n1, n2, True)
        pn, _ = normal_p2(u, n1, n2, False)
        flag = "  <- brackets 0.06706" if (exact_p2(Us, u - 1) < P_KO <= pe or
                                           exact_p2(Us, u) < P_KO <= exact_p2(Us, u + 1)) else ""
        print(f"     U = {u:2d}: exact 2-sided {pe:.5f}; normal cc {pc:.5f}; "
              f"normal no-cc {pn:.5f}{flag}")
U78 = EXACT[(7, 8)]
EX11, EX12 = exact_p2(U78, 11), exact_p2(U78, 12)
NORM = sorted([normal_p2(11, 7, 8, True)[0], normal_p2(12, 7, 8, True)[0],
               normal_p2(12, 7, 8, False)[0]])
print(f"   n = 7 vs 8: 0.06706 lies between exact values {EX11:.5f} (U = 11) and "
      f"{EX12:.5f} (U = 12)")
print(f"   n = 7 vs 8: normal approximations closest to 0.06706 (U = 11 with cc, U = 12 with and "
      f"without cc) span {NORM[0]:.5f} to {NORM[-1]:.5f}")
print(f"   gap between those two exact values: {EX12 - EX11:.5f}; 0.06706 is not itself an "
      f"exact no-ties value")
P_TYPO = 0.006706
print(f"   for reference, a value of {P_TYPO} would lie between exact values "
      f"{exact_p2(U78, 5):.5f} (U = 5) and {exact_p2(U78, 6):.5f} (U = 6)")

U66 = exact_U(6, 6)
print(f"   n = 6 vs 6 (CKO): exact U = 3 -> {exact_p2(U66, 3):.5f}, U = 4 -> "
      f"{exact_p2(U66, 4):.5f}; normal cc U = 3 -> {normal_p2(3, 6, 6, True)[0]:.5f}, "
      f"no-cc U = 4 -> {normal_p2(4, 6, 6, False)[0]:.5f}")

# ----------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#8a8984"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 1.5, 8   # 1.5 pt = 2 px at 96 dpi; markers 8 pt


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)
    print("   wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")


def clean(ax, grid_axis="x"):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis=grid_axis, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)


# ----------------------------------------------------------------- design A: p-value axis
def design_A(path):
    from matplotlib.lines import Line2D
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ax.axvspan(0, ALPHA, color="#eef3fa", zorder=0)
    ax.axvline(ALPHA, color=INK2, lw=1.0)
    ax.text(ALPHA, 3.05, "stated threshold p = 0.05 (Methods)", ha="center", va="bottom",
            fontsize=10, color=INK2)
    ax.text(0.025, 2.85, "significant (p < 0.05)", ha="center", va="center",
            fontsize=10, color=INK2)
    ax.text(0.0775, 2.85, "not significant", ha="center", va="center",
            fontsize=10, color=INK2)
    y = {"cko": 0.4, "mw": 1.2, "ko": 2.1}
    ax.plot(P_CKO, y["cko"], "o", ms=MS, color=ORANGE, zorder=3)
    ax.annotate(f"p = {P_CKO} (**), \"significantly less\"", (P_CKO, y["cko"]),
                xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
    ax.plot(P_KO, y["ko"], "o", ms=MS, color=ORANGE, zorder=3)
    ax.annotate(f"p = {P_KO} (*)\n\"significantly less\"", (P_KO, y["ko"]),
                xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
    ax.hlines(y["mw"], EX11, EX12, color=BLUE, lw=LW, zorder=2)
    ax.plot([EX11, EX12], [y["mw"]] * 2, "s", ms=MS, color=BLUE, zorder=3, mfc=BG, mew=LW)
    for u, pe, ha in ((11, EX11, "right"), (12, EX12, "left")):
        ax.text(pe + (-0.0012 if ha == "right" else 0.0012), y["mw"] + 0.24,
                f"U = {u}: {pe:.4f}", ha=ha, va="bottom", fontsize=10, color=INK2)
    ax.plot([P_KO, P_KO], [y["mw"] + 0.12, y["ko"] - 0.12], color=GRAY, lw=1.0, ls=(0, (2, 2)))
    ax.set_yticks([y["cko"], y["mw"], y["ko"]])
    ax.set_yticklabels(["WT vs TRPV4CKO\n(n = 6 eyes)",
                        "neighbouring exact\nMann-Whitney p,\nn = 7 vs 8",
                        "WT vs TRPV4KO\n(n = 7 or 8 eyes)"], fontsize=10, color=INK)
    ax.set_xlim(0, 0.10)
    ax.set_ylim(0, 3.35)
    ax.set_xlabel("two-sided p value (linear scale from 0)")
    clean(ax)
    h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
                label="reported in the Fig. 6D legend"),
         Line2D([], [], marker="s", ls="-", lw=LW, ms=MS, color=BLUE, mfc=BG, mew=LW,
                label="recomputed: exact two-sided p (no ties)")]
    fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
               bbox_to_anchor=(0.5, 0.0))
    fig.tight_layout(rect=(0, 0.07, 1, 1))
    save(fig, path)


# ----------------------------------------------------------------- design B: all p-values in the paper
def design_B(path):
    rows = sorted(ALL_P, key=lambda r: r[2])
    fig, ax = plt.subplots(figsize=(7.2, 5.4))
    ax.axvline(ALPHA, color=INK2, lw=1.0)
    ax.text(ALPHA * 1.08, 7.5, "stated threshold p = 0.05", ha="left", rotation=90,
            va="center", fontsize=10, color=INK2)
    for i, (loc, lab, p) in enumerate(rows):
        is_ko = p == P_KO
        ax.hlines(i, 1e-4, p, color=GRID, lw=0.8, zorder=1)
        ax.plot(p, i, "o", ms=MS if is_ko else MS - 1, color=ORANGE,
                mfc=ORANGE if is_ko else BG, mew=LW, zorder=3)
        if is_ko:
            ax.text(0.0009, 1.3, "filled dot: p = 0.06706, the\nonly printed p above 0.05, yet\n"
                    "called \"significantly less\"", ha="left", va="center", fontsize=10,
                    color=INK)
    ax.set_yticks(range(len(rows)))
    ax.set_yticklabels([f"{r[0]}  {r[1]}" for r in rows], fontsize=10)
    for t, r in zip(ax.get_yticklabels(), rows):
        t.set_color(INK if r[2] == P_KO else INK2)
        if r[2] == P_KO:
            t.set_fontweight("bold")
    ax.set_xscale("log")
    ax.set_xlim(8e-5, 0.3)
    ax.set_ylim(-0.7, len(rows) - 0.3)
    ax.set_xlabel("p value as printed in the figure legends (log scale)")
    ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1])
    ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1"])
    clean(ax)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, mfc=BG, mew=LW,
                label="reported p, called significant"),
         Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
                label="reported p = 0.06706 (Fig. 6D, KO)")]
    fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
               bbox_to_anchor=(0.5, 0.0))
    fig.tight_layout(rect=(0, 0.05, 1, 1))
    save(fig, path)


# ----------------------------------------------------------------- design C: two text panels
def panel(ax, x, y, w, h, title, body, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
                                fc="#ffffff", ec=edge, lw=LW))
    ax.text(x + 0.02, y + h - 0.04, title, fontsize=10, fontweight="bold", va="top")
    ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", linespacing=1.4)


def design_C(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    panel(ax, 0.01, 0.31, 0.36, 0.66, "Methods, \"Statistical analysis\"",
          "\"The Mann-Whitney U test\nwas used to compare\nunpaired values of\nTUNEL-positive cell\n"
          "density ... p < 0.05 was\nconsidered significant.\"", INK2)
    panel(ax, 0.63, 0.31, 0.36, 0.66, "Figure 6D legend",
          "\"Conventional TRPV4KO\nmice showed significantly\nless photoreceptor cell\n"
          "death than WT mice 24 h\nafter retinal detachments\n(*p = 0.06706, n = 7 or 8\neyes).\"",
          ORANGE)
    ax.annotate("", xy=(0.62, 0.66), xytext=(0.38, 0.66),
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
    ax.text(0.50, 0.70, "conflict", ha="center", va="bottom", fontsize=10, color=INK,
            fontweight="bold")
    ax.text(0.50, 0.62, "0.06706 is not\n< 0.05, but is\ncalled significant", ha="center",
            va="top", fontsize=10, color=INK2)
    ax.add_patch(FancyBboxPatch((0.02, 0.03), 0.96, 0.22,
                                boxstyle="round,pad=0.01,rounding_size=0.02",
                                fc="#eef3fa", ec=BLUE, lw=LW))
    ax.text(0.5, 0.14, f"Recomputed: {P_KO} > {ALPHA} (by {P_KO - ALPHA:.5f}), so not significant "
            "under the paper's rule.\nThe CKO result in the same legend "
            f"(p = {P_CKO}) is below 0.05 and is consistent.", ha="center",
            va="center", fontsize=10)
    ax.text(0.01, 1.0, "gray frame: stated rule;  orange frame: reported claim;  blue frame: recomputed check",
            fontsize=10, color=INK2, va="bottom")
    ax.set_ylim(0, 1.07)
    save(fig, path)


# ----------------------------------------------------------------- design D: exact ladder of attainable p
def design_D(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.4))
    us = np.arange(5, 15)
    pe = [exact_p2(U78, u) for u in us]
    pc = [normal_p2(u, 7, 8, True)[0] for u in us]
    ax.axhline(ALPHA, color=INK2, lw=1.0)
    ax.text(14.6, ALPHA + 0.002, "stated threshold p = 0.05", ha="right", va="bottom",
            fontsize=10, color=INK2)
    ax.plot(us, pe, "s-", color=BLUE, lw=LW, ms=MS, mfc=BG, mew=LW,
            label="recomputed: exact two-sided p, n = 7 vs 8")
    ax.plot(us, pc, "o--", color=BLUE, lw=LW, ms=MS - 2, alpha=0.75,
            label="recomputed: normal approx. with continuity correction")
    ax.axhline(P_KO, color=ORANGE, lw=LW, ls=(0, (1, 2)),
               label=f"reported p = {P_KO} (Fig. 6D, called significant)")
    ax.text(4.6, P_KO + 0.002, f"reported {P_KO}", ha="left", va="bottom", fontsize=10)
    ax.set_xticks(us)
    ax.set_xlabel("Mann-Whitney U statistic (smaller U = larger group difference)")
    ax.set_ylabel("two-sided p value")
    ax.set_ylim(0, 0.13)
    ax.set_xlim(4.5, 14.8)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="y", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.legend(loc="upper left", frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


# ----------------------------------------------------------------- design E: ledger of the two comparisons
def design_E(path):
    fig, ax = plt.subplots(figsize=(7.2, 2.8))
    ax.set_xlim(0, 1)
    ax.set_ylim(-0.12, 1)
    ax.axis("off")
    cols = [0.01, 0.30, 0.47, 0.71, 0.89]
    heads = ["comparison", "printed p\n(reported)", "p < 0.05?\n(recomputed)",
             "legend\nwording", "consistent?"]
    for x, h in zip(cols, heads):
        ax.text(x, 0.90, h, fontsize=10, color=INK2, va="top", fontweight="bold")
    ax.plot([0.01, 0.99], [0.66, 0.66], color=GRID, lw=1.2)
    rows = [("WT vs TRPV4KO\n(n = 7 or 8 eyes)", P_KO, 0.47),
            ("WT vs TRPV4CKO\n(n = 6 eyes)", P_CKO, 0.17)]
    for lab, p, y in rows:
        ok = p < ALPHA
        ax.text(cols[0], y, lab, fontsize=10.5, va="center")
        ax.text(cols[1], y, f"{p}", fontsize=11, va="center", color=INK)
        ax.plot(cols[1] - 0.018, y, "o", ms=MS, color=ORANGE)
        ax.text(cols[2], y, "yes" if ok else "no (> 0.05)", fontsize=10.5, va="center")
        ax.plot(cols[2] - 0.018, y, "s", ms=MS, color=BLUE, mfc=BG, mew=LW)
        ax.text(cols[3], y, "\"significantly\nless\"", fontsize=10.5, va="center")
        ax.text(cols[4], y, "yes" if ok else "NO", fontsize=11, va="center",
                fontweight="normal" if ok else "bold")
    ax.plot([0.01, 0.99], [0.32, 0.32], color=GRID, lw=0.8)
    ax.text(0.01, -0.08, "orange dot: printed in the paper;  blue square: recomputed check", fontsize=10, color=INK2, va="bottom")
    save(fig, path)


print("\n5. Figures")
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))
if "--candidates" in sys.argv:
    cd = os.path.join(HERE, "candidates")
    os.makedirs(cd, exist_ok=True)
    design_A(os.path.join(cd, "A_pvalue_axis"))
    design_B(os.path.join(cd, "B_all_pvalues"))
    design_C(os.path.join(cd, "C_text_panels"))
    design_D(os.path.join(cd, "D_exact_ladder"))
    design_E(os.path.join(cd, "E_ledger"))

Reported renal-cancer P=0.004 is P=0.24 under the paper's own stated Fisher test

JAMA (2010) · doi:10.1001/jama.2010.1877 · PMC3655326

Summary

The paper compares the prevalence of cancers in Cowden or Cowden-like (CS/CSL) patients with germline KILLIN promoter methylation against patients with germline PTEN mutations. The Abstract and the Results both report renal cell carcinoma in 4/45 methylation-positive patients and 6/155 PTEN-mutation-positive patients, with P=0.004. The Methods say these prevalences were compared with a two-tailed Fisher exact test. On those counts that test gives P=0.237 (odds ratio 2.42; exact 95% CI 0.48–10.72), and none of the other standard tests for a 2×2 table gives a value below P=0.09. The renal association is one of the two cancer risks named in the abstract Conclusions, and the Comment bases advice on kidney surveillance and test prioritization on it. The paper's primary finding (methylation in 45/123 patients) and its breast-cancer comparison are not affected.

What the paper reports

Abstract, Results:

Individuals with KILLIN-promoter methylation had 3-fold increased prevalence of breast cancer (35/42 vs 24/64, P<0.0001) and a >2-fold increase of kidney cancer (4/45 vs 6/155, P=0.004) over those with germline PTEN mutations.

Abstract, Conclusions:

Germline KILLIN- methylation is common among CS/CSL patients, and associated with increased risks of breast and renal cancer over PTEN mutation-positive patients. These observations need to be replicated.

Results, paragraph "Prevalent Cancers in Pathogenic PTEN Mutation Positive versus KILLIN Methylation":

Renal cell carcinoma was over-represented in the methylation positive subjects over PTEN mutation positive individuals (4/45 vs 6/155, P=0.004).

Methods, "Statistical Analysis":

The prevalence of component malignancies between KILLIN promoter methylation positive patients and germline pathogenic PTEN mutation positive patients was compared using the Fisher’s 2-tailed exact test with P<0.05 considered to be significant.

Comment, second paragraph:

Patients with CS-like features, especially where breast cancer and/or renal carcinomas are present in the individual and/or family, should be offered KILLIN methylation analysis first

Comment, third paragraph:

In contrast to germline PTEN mutations, germline methylation of the KILLIN promoter confers a significantly higher prevalence of female invasive breast cancer and renal cell carcinomas. The current national practice guidelines for those with PTEN germline mutations includes heightened surveillance of the female breasts and thyroid, but do not have awareness of renal cancer risk. If our observations of two- to three-fold increased risks of renal and/or breast cancer with KILLIN germline methylation over those of PTEN mutation holds, then extra vigilance for the organs-at-risk, breast and kidneys, is warranted.

Comment, "Conclusions":

Among CS/CSL patients, presence of germline KILLIN gene promoter hypermethylation was common and was associated with increased risk of breast and renal cancer compared with PTEN mutation-positive patients.

Every quote was checked verbatim against the article as archived in PubMed Central (the author manuscript). The publisher's version of record gives the same counts and P value in JAMA house style ("4/45 vs 6/155; P = .004").

What the paper's own numbers show

Method. The reported counts give the 2×2 table [[4, 41], [6, 149]]: renal cell carcinoma present or absent among 45 methylation-positive and 155 PTEN-mutation-positive patients. I applied the test the Methods name (two-tailed Fisher exact), then the other common tests for this table, to see whether any of them gives 0.004. I also worked out which counts would be needed to reach P=0.004. As a control, I recomputed the breast-cancer comparison from the same sentence and the methylation prevalence the same way. Every value below is printed by recompute.py.txt (scipy 1.18.1). The paper has no tables holding these counts; they appear only in the text.

Quantity Location Reported Recomputed
Renal prevalence, methylated vs PTEN Abstract Results; Results 4/45 vs 6/155, ">2-fold" 8.9% vs 3.9%; prevalence ratio 2.30 (consistent)
Expected renal cases among the 45 if prevalence were equal derived — 2.25 (observed 4; 10 cases among 200 patients)
Fisher exact, two-tailed (the test the Methods name) Abstract Results; Results P=0.004 P=0.237
Fisher exact, one-tailed same counts — P=0.164
Fisher one-tailed mid-p same counts — P=0.106
Pearson chi-square, uncorrected / Yates-corrected same counts — P=0.174 / 0.331
Barnard / Boschloo exact, two-sided same counts — P=0.212 / 0.208
Exact binomial, 4/45 against a fixed rate of 6/155 (one-sided) same counts — P=0.096
Odds ratio (sample), Wald 95% CI, Wald P same counts not reported 2.42 (0.65–8.99), P=0.186
Odds ratio (conditional MLE), exact 95% CI same counts not reported 2.41 (0.48–10.72)
Renal cases among the 45 needed for two-tailed Fisher P≤0.004 (PTEN group held at 6/155) sensitivity 4 8 (P=0.0038)
Renal cases among the 45 needed for P<0.05 (PTEN group held at 6/155) sensitivity 4 6 (P=0.029)
PTEN-group renal cases needed for P≤0.004 (methylated group held at 4/45) sensitivity 6/155 0/155 (P=0.0023); 1/155 gives P=0.0096
Breast cancer, 35/42 vs 24/64 (control check) Abstract Results; Results P<0.0001 Fisher two-tailed P=3.1×10⁻⁶ (consistent)
Methylation prevalence (control check) Abstract Results 45/123, 37% (95% CI 29–45%) 36.6%; Wald 28.1–45.1%, exact 28.1–45.7% (consistent)

The reported P value is about 1/59 of the value the stated test gives. The ">2-fold" prevalence ratio is correct. Only the P value, and the claim of statistical significance that rests on it, do not follow from the counts.

A smaller point, which does not change any conclusion: the Abstract calls the breast-cancer difference a "3-fold increased prevalence", but 35/42 vs 24/64 is 83.3% vs 37.5%, a prevalence ratio of 2.22 (odds ratio 8.33).

Figure

Reported P value for renal cancer compared with P values recomputed from the paper's counts under eight tests

Figure 1. P values for renal cell carcinoma in 4/45 KILLIN-methylation-positive vs 6/155 PTEN-mutation-positive patients, on one logarithmic axis. The orange point is the value reported in the Abstract and Results (P=0.004). Blue points are recomputed from the same counts; the larger blue point is the two-tailed Fisher exact test named in the Methods (P=0.237), joined to the reported value by a dotted line. The dashed line marks the paper's significance threshold, P=0.05. Every recomputed value lies to the right of the threshold, between 0.096 and 0.331, so the conclusion does not depend on the choice of test.

Two-tailed Fisher P as a function of the number of renal cancers among the 45 methylation-positive patients

Figure 2. Two-tailed Fisher exact P (blue, log scale) for x/45 methylation-positive patients with renal cancer against the reported 6/155 in the PTEN-mutation group, for x from 0 to 12. At the reported count of 4/45 the test gives P=0.237. The reported P=0.004 (orange dashed line) is first reached at 8/45 (P=0.0038), twice the reported count. A count of 6/45 would be needed to cross P=0.05.

Why it matters

Explanations considered

None of these explanations accounts for the reported value.

Status

As of 2026-10-23:

Suggested correction

In the Abstract Results and in the Results paragraph on prevalent cancers, replace "P=0.004" for renal cell carcinoma (4/45 vs 6/155) with the two-tailed Fisher exact value, P=0.24, and preferably give the odds ratio with its exact 95% CI (2.41, 0.48–10.72). Then revise the statements that rest on this result: the abstract Conclusions, the Comment ("confers a significantly higher prevalence of ... renal cell carcinomas", and the kidney-surveillance and testing-priority statements) and the closing Conclusions. They could say that renal cancer was more common in the methylation-positive group (8.9% vs 3.9%) but that the difference was not statistically significant in this sample. If the counts, rather than the P value, are in error, the correction should give the correct counts and describe the 155-patient PTEN-mutation comparison group. Separately, the breast-cancer difference described as "3-fold" corresponds to a prevalence ratio of 2.2 (83.3% vs 37.5%).

Recomputation script (python recompute.py.txt)
"""Recompute the renal-cancer comparison of PMC3655326
(JAMA 2010;304(24):2724-2731, doi:10.1001/jama.2010.1877).

Run:  python recompute.py.txt
Needs numpy, scipy (>= 1.10 for odds_ratio / barnard / boschloo) and matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.

Inputs are copied from the paper's Abstract and Results ("Prevalent Cancers in
Pathogenic PTEN Mutation Positive versus KILLIN Methylation"). The paper has no
tables holding these counts; they appear only in the text.
  Renal cell carcinoma: 4/45 KILLIN-methylated vs 6/155 PTEN-mutation-positive, reported P=0.004
  Female invasive breast cancer: 35/42 vs 24/64, reported P<0.0001
  Methylation prevalence: 45/123, reported 37% (95% CI 29-45%)
Methods: "compared using the Fisher's 2-tailed exact test with P<0.05 considered to be significant"
"""
import os

import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (from the paper)
RENAL = dict(a=4, n1=45, c=6, n2=155, reported_p=0.004)
BREAST = dict(a=35, n1=42, c=24, n2=64)
METH = dict(k=45, n=123)
ALPHA = 0.05

ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"


def table(a, n1, c, n2):
    return [[a, n1 - a], [c, n2 - c]]


def fisher2(a, n1, c, n2):
    return stats.fisher_exact(table(a, n1, c, n2), alternative="two-sided")[1]


def main():
    r = RENAL
    t = table(r["a"], r["n1"], r["c"], r["n2"])
    p1, p2 = r["a"] / r["n1"], r["c"] / r["n2"]
    print("=== Renal cell carcinoma: 4/45 (KILLIN-methylated) vs 6/155 (PTEN mutation+) ===")
    print(f"prevalence {100*p1:.1f}% vs {100*p2:.1f}%; prevalence ratio {p1/p2:.2f}")
    sample_or = stats.fisher_exact(t)[0]
    cond = odds_ratio(t, kind="conditional")
    ci = cond.confidence_interval(0.95)
    print(f"sample odds ratio {sample_or:.2f}; conditional MLE OR {cond.statistic:.2f}, "
          f"exact 95% CI {ci.low:.2f}-{ci.high:.2f}")
    (a_, b_), (c_, d_) = t
    se_log = np.sqrt(1 / a_ + 1 / b_ + 1 / c_ + 1 / d_)
    lo, hi = np.exp(np.log(sample_or) + np.array([-1, 1]) * 1.959964 * se_log)
    p_wald = 2 * stats.norm.sf(abs(np.log(sample_or)) / se_log)
    print(f"sample OR Wald 95% CI {lo:.2f}-{hi:.2f}, Wald P {p_wald:.3f}")
    chi = stats.chi2_contingency(t, correction=False)
    chiy = stats.chi2_contingency(t, correction=True)
    print(f"total events {r['a'] + r['c']} of {r['n1'] + r['n2']}; "
          f"expected in methylated group under no difference {chi.expected_freq[0][0]:.2f} (observed {r['a']})")

    results = {}
    results["Fisher exact, two-sided\n(the test named in Methods)"] = fisher2(r["a"], r["n1"], r["c"], r["n2"])
    results["Fisher exact, one-sided"] = stats.fisher_exact(t, alternative="greater")[1]
    pmf = [stats.hypergeom.pmf(k, 200, 10, 45) for k in range(11)]
    results["Fisher one-sided mid-p"] = sum(pmf[5:]) + 0.5 * pmf[4]
    results["Pearson chi-square"] = chi.pvalue
    results["Yates-corrected chi-square"] = chiy.pvalue
    results["Barnard exact, two-sided"] = stats.barnard_exact(t).pvalue
    results["Boschloo exact, two-sided"] = stats.boschloo_exact(t).pvalue
    results["Binomial 4/45 vs fixed rate 6/155,\none-sided"] = stats.binomtest(4, 45, 6 / 155, alternative="greater").pvalue
    for k, v in results.items():
        print(f"  {k.replace(chr(10), ' '):55s} P = {v:.4f}")
    p_fisher = results["Fisher exact, two-sided\n(the test named in Methods)"]
    print(f"reported P=0.004; recomputed two-sided Fisher P={p_fisher:.3f}, "
          f"ratio {p_fisher / r['reported_p']:.0f}x")
    print(f"smallest recomputed P across all tests: {min(results.values()):.4f}")

    print("\n=== Counts that would give two-sided Fisher P near 0.004 ===")
    sweep_a = {a: fisher2(a, 45, 6, 155) for a in range(0, 13)}
    for a, p in sweep_a.items():
        print(f"  {a:2d}/45 vs 6/155: P = {p:.5f}")
    need = min(a for a, p in sweep_a.items() if p <= r["reported_p"] and a > 4)
    print(f"smallest methylated count (of 45) with P <= 0.004, holding 6/155: {need} (P = {sweep_a[need]:.4f})")
    need05 = min(a for a, p in sweep_a.items() if p < ALPHA and a > 4)
    print(f"smallest methylated count (of 45) with P < 0.05, holding 6/155: {need05} (P = {sweep_a[need05]:.4f})")
    sweep_c = {c: fisher2(4, 45, c, 155) for c in range(0, 7)}
    for c, p in sweep_c.items():
        print(f"  4/45 vs {c}/155: P = {p:.5f}")
    print("  other denominators (from the paper, then hypothetical larger PTEN groups):")
    for n1, n2 in [(42, 64), (45, 64), (42, 155), (45, 500), (45, 1000)]:
        print(f"  4/{n1} vs 6/{n2}: P = {fisher2(4, n1, 6, n2):.4f}")

    print(f"scipy {__import__('scipy').__version__}")
    print("\n=== Breast cancer 35/42 vs 24/64 (reported P<0.0001, '3-fold') ===")
    b = BREAST
    bt = table(b["a"], b["n1"], b["c"], b["n2"])
    bp1, bp2 = b["a"] / b["n1"], b["c"] / b["n2"]
    print(f"prevalence {100*bp1:.1f}% vs {100*bp2:.1f}%; prevalence ratio {bp1/bp2:.2f}; "
          f"sample odds ratio {stats.fisher_exact(bt)[0]:.2f}")
    print(f"two-sided Fisher P = {fisher2(**{k: b[k] for k in ('a','n1','c','n2')}):.1e}")

    print("\n=== Methylation prevalence 45/123 (reported 37%, 95% CI 29-45%) ===")
    ph = METH["k"] / METH["n"]
    se = np.sqrt(ph * (1 - ph) / METH["n"])
    cp = stats.binomtest(METH["k"], METH["n"]).proportion_ci(method="exact")
    print(f"{100*ph:.1f}%; Wald 95% CI {100*(ph-1.96*se):.1f}-{100*(ph+1.96*se):.1f}%; "
          f"Clopper-Pearson {100*cp.low:.1f}-{100*cp.high:.1f}%")

    fig1(results, r["reported_p"])
    fig2(sweep_a, r["reported_p"])


def style(ax):
    ax.set_facecolor(BG)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(TXT2)
    ax.tick_params(colors=TXT2, labelsize=10)
    ax.grid(axis="x", which="major", color=GRID, lw=0.8)
    ax.set_axisbelow(True)


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)


def fig1(results, reported):
    plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans"})
    labels = ["Reported in the paper\n(Abstract and Results)"] + list(results.keys())
    vals = [reported] + list(results.values())
    y = np.arange(len(labels))[::-1]
    fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
    style(ax)
    ax.set_xscale("log")
    ax.set_xlim(0.001, 1.0)
    ax.axvline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
    ax.text(ALPHA * 1.08, y[0] + 0.55, "P = 0.05\n(paper's threshold)", color=TXT2, fontsize=10, va="center")
    # gap between reported and recomputed Fisher value
    ax.plot([vals[0], vals[1]], [y[0], y[1]], color=TXT2, lw=1, ls=":")
    for i, (lab, v) in enumerate(zip(labels, vals)):
        col = ORANGE if i == 0 else BLUE
        ax.plot(v, y[i], "o", ms=10 if i < 2 else 8, color=col, zorder=3)
        ax.text(v * (0.82 if i == 0 else 1.15), y[i], f"{v:.3f}", color=col, fontsize=10,
                va="center", ha="right" if i == 0 else "left", fontweight="bold" if i < 2 else "normal")
    ax.set_yticks(y)
    ax.set_yticklabels(labels, fontsize=10, color=TXT)
    ax.tick_params(axis="y", length=0)
    ax.set_xticks([0.001, 0.004, 0.01, 0.05, 0.1, 0.25, 1.0])
    ax.set_xticklabels(["0.001", "0.004", "0.01", "0.05", "0.1", "0.25", "1"])
    ax.set_xlabel("P value for renal cancer, 4/45 vs 6/155 (log scale)", color=TXT, fontsize=10)
    ax.set_ylim(y[-1] - 0.7, y[0] + 1.1)
    h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Reported by the paper")
    h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Recomputed from the paper's counts")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.3, -0.16), ncol=2,
              frameon=False, fontsize=10, labelcolor=TXT)
    fig.tight_layout()
    save(fig, "fig1")


def fig2(sweep, reported):
    a = np.array(list(sweep.keys()))
    p = np.array(list(sweep.values()))
    fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
    style(ax)
    ax.grid(axis="y", which="major", color=GRID, lw=0.8)
    ax.set_yscale("log")
    ax.plot(a, p, "-", color=BLUE, lw=2, zorder=2)
    ax.plot(a, p, "o", ms=8, color=BLUE, zorder=3, label="Two-sided Fisher P, recomputed (6/155 held fixed)")
    ax.axhline(reported, color=ORANGE, lw=2, ls="--", zorder=1)
    ax.text(12.3, reported * 1.25, "Reported P = 0.004", color=ORANGE, ha="right", va="bottom", fontsize=10)
    ax.axhline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
    ax.text(12.3, ALPHA * 1.15, "P = 0.05", color=TXT2, ha="right", va="bottom", fontsize=10)
    ax.plot(4, sweep[4], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
    ax.annotate(f"Reported count 4/45\nP = {sweep[4]:.3f}", (4, sweep[4]), xytext=(5.6, 0.55),
                fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
    ax.plot(8, sweep[8], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
    ax.annotate(f"8/45 would be needed\nP = {sweep[8]:.4f}", (8, sweep[8]), xytext=(8.6, 0.012),
                fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
    ax.set_xticks(a)
    ax.set_xlim(-0.5, 12.5)
    ax.set_ylim(1e-5, 1.6)
    ax.set_xlabel("Renal cancers among the 45 KILLIN-methylated patients (comparison group fixed at 6/155)",
                  color=TXT, fontsize=10)
    ax.set_ylabel("Two-sided Fisher P (log scale)", color=TXT, fontsize=10)
    ax.legend(loc="lower left", frameon=False, fontsize=10, labelcolor=TXT)
    fig.tight_layout()
    save(fig, "fig2")


if __name__ == "__main__":
    main()

Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1

JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.46336 · PMC11579790

Summary

The Key Points and the abstract's Conclusions and Relevance say that cancer risk was "statistically significantly" higher in people starting Janus kinase inhibitors (JAKis) than in people starting tumor necrosis factor inhibitors (TNFis). The paper's own estimate for that comparison is an adjusted hazard ratio (HR) of 1.36 with a 95% CI of 0.94-1.96. The interval includes 1. Working back from it gives a two-sided P of about 0.10, above the paper's stated threshold of 2-sided P < .05. The propensity-matched estimate (HR 1.79, 0.91-3.56; P about 0.09; shown as 1.8, 0.9-3.6 in eTable 10 of Supplement 1) and the crude rate ratio from Table 2 (1.21, 0.77-1.88) are not significant either, and the paper's Results, Discussion and main-text Conclusions all describe the JAKi result as not significant. The HR, its CI and its direction are reported correctly throughout. What is wrong is the significance label in the two most-read summaries, on the comparison of most safety interest (the JAKi cancer signal from the ORAL Surveillance trial).

What the paper reports

All passages are verbatim from the article as archived in PubMed Central.

The two summary statements:

"In this cohort study consisting of 25 305 individuals who initiated treatment with biologic or targeted synthetic disease-modifying antirheumatic drugs, a statistically significantly higher risk of incident cancer was seen with rituximab, abatacept, and JAKis compared with TNFis." (Key Points, Findings)

"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, individuals initiating rituximab, abatacept, and JAKis demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis in the first 2 years after initiation of biologic or targeted synthetic DMARDs." (Abstract, Conclusions and Relevance)

The estimate those statements rest on, which appears in the abstract Results, the main Results and Table 2:

"Multivariable Cox proportional hazards regression analysis showed that rituximab was associated with a higher risk of incident cancer compared with TNFis (hazard ratio [HR], 1.91; 95% CI, 1.17-3.14), followed by abatacept (HR, 1.47; 95% CI, 1.03-2.11), and JAKis (HR, 1.36; 95% CI, 0.94-1.96)." (Abstract, Results)

"JAKi | 2570 (9) | 276 (169-351) | 22 (0.9) | 94 (59-143) | 1.36 (0.94-1.96)" (Table 2, JAKi row: exposures, median days to diagnosis, crude cancer diagnoses, diagnoses per 10 000 person-years, adjusted HR)

The significance convention:

"A 2-sided P < .05 was considered statistically significant." (Methods, Statistical Analysis)

The body of the paper describes the same result as not significant:

"Also like the primary analysis, we found that those treated with JAKis did not have a statistically significant increase in risk of cancer (HR, 1.79; 95% CI, 0.91-3.56) (eTable 10 and eFigure 4 in Supplement 1)." (Results, propensity-matched analysis paragraph)

"We observed that, among patients with RA, rituximab, abatacept, and JAKi initiations were associated with increased risks of incident cancer after 90 days and within 2 years of initiation, compared with TNFi initiation, although only rituximab and abatacept initiations had statistically significant associations." (Discussion, first paragraph)

"Our findings of increased incidence rate and risk of cancer with JAKi exposure compared with TNFi exposure are consistent with the results of ORAL Surveillance,6 although the results were not statistically significant." (Discussion, paragraph on prior JAKi studies)

"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, we found that new rituximab and abatacept initiators demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with TNFi initiators in the first 2 years after biologic or targeted synthetic DMARD initiation." (Conclusions, main text)

What the paper's own numbers show

Method. Cox-model HRs come with Wald intervals on the log scale, so a reported HR and 95% CI fix the standard error: SE = [ln(upper) − ln(lower)] / (2 × 1.960). Then z = ln(HR) / SE and the two-sided P = 2 × [1 − Φ(|z|)]. This uses only the published numbers; the result is approximate because the bounds are rounded to two decimals. Before relying on it, I checked that the JAKi interval is symmetric about ln(1.36) on the log scale (0.369 below, 0.365 above), as a Wald interval should be. I applied the same calculation to every HR the paper reports, so the comparisons the paper correctly calls significant serve as a check. Separately, I recomputed the crude rates in Table 2 from their event counts. The person-years each rate implies give exact Poisson intervals that match the reported ones to within 1-2 units, so the inputs are internally consistent. From those I formed a crude JAKi/TNFi rate ratio and an exact conditional test. I did the same for the propensity-matched rates in eTable 10 of Supplement 1. All numbers come from recompute.py.txt.

Quantity Location Reported Recomputed
JAKi vs TNFi adjusted HR (95% CI) Abstract Results; Results; Table 2 1.36 (0.94-1.96) CI includes 1; geometric midpoint of the bounds 1.36 (consistent)
JAKi SE(ln HR), z derived from Table 2 not reported SE 0.187; z 1.64 (z ≥ 1.96 needed for P < .05)
JAKi two-sided P derived from Table 2 not reported; Key Points Findings and abstract Conclusions and Relevance say "statistically significantly" 0.10 (not < .05)
Smallest HR that would be significant at this SE derived n/a 1.44
JAKi one-sided P (not the stated convention) derived Methods: "2-sided P < .05" 0.050
JAKi propensity-matched HR (95% CI) Results 1.79 (0.91-3.56), "did not have a statistically significant increase" P 0.094 (not significant; agrees with the text)
JAKi propensity-matched HR (95% CI) Supplement 1, eTable 10 1.8 (0.9-3.6); 22 vs 14 cancers same as the Results value rounded to one decimal; P about 0.10 from the rounded bounds
Propensity-matched crude rates per 10 000 PY, JAKi vs TNFi Supplement 1, eTable 10 100 (63-151), 22 events; 60 (33-101), 14 events imply about 2200 and 2333 PY; exact Poisson CIs 63-151 and 33-101 (exact match); rate ratio 1.67 (0.85-3.26), exact conditional P 0.14
JAKi crude rate per 10 000 PY Table 2 94 (59-143), 22 events implies about 2340 PY; exact Poisson CI 59-142
TNFi crude rate per 10 000 PY Table 2 78 (66-91), 162 events implies about 20 769 PY; exact Poisson CI 66-91
Crude rate ratio JAKi/TNFi derived from Table 2 n/a 1.21 (0.77-1.88); exact conditional P 0.39
Rituximab vs TNFi HR Table 2 1.91 (1.17-3.14) P 0.010 (significant; agrees with the text)
Abatacept vs TNFi HR Table 2 1.47 (1.03-2.11) P 0.035 (significant; agrees with the text)
IL-6i vs TNFi HR Table 2 1.04 (0.57-1.92) P 0.90
Propensity-matched rituximab, abatacept, IL-6i HRs Results 4.37 (1.48-12.93); 3.12 (1.52-6.44); 5.65 (1.11-28.79), all called significant P 0.008; 0.002; 0.037 (agree with the text)

Every HR the paper calls significant has a back-calculated P below .05 and a CI that excludes 1. The JAKi primary estimate is the only one the Key Points and abstract call significant that does not meet either criterion. No JAKi analysis reported anywhere in the paper or in Supplement 1 does.

Figure

Two text panels contrast the Key Points and abstract significance claim for JAKis with the body's estimate and statements

Figure 1. The left panel (Key Points and Abstract) quotes the Key Points Findings and the abstract's Conclusions and Relevance, which group JAKis with rituximab and abatacept as having a "statistically significantly" increased cancer risk. The right panel gives the estimate those statements rest on (Table 2 and Results) and the body's own description of it: the Discussion says only rituximab and abatacept had significant associations, and the main-text Conclusions name only those two drugs. The strip below gives the reported HR and CI (orange) and the z and two-sided P recomputed from that CI (blue). All quotes are verbatim from the PMC version; the recomputed values come from recompute.py.txt.

Forest plot of every reported HR on a log axis with recomputed two-sided P values

Figure 2. Every adjusted HR versus TNFi that the paper reports, primary (Table 2) and propensity-matched (Results), drawn as the reported point estimate and 95% CI (orange) on one log-scale axis. The thin vertical line marks HR = 1. The right-hand columns give the reported values and the two-sided P recomputed from each CI (blue). Both JAKi rows (shaded) cross 1, with P = 0.101 and 0.094; the label on the primary JAKi row marks the estimate that the Key Points and abstract call "statistically significantly" higher. Every row the paper calls significant has P < .05.

Why it matters

The JAKi versus TNFi comparison is the question that motivates the study: the Importance section opens with the increased cancer risk with tofacitinib versus TNFis in ORAL Surveillance. The Key Points Findings sentence is the paper's headline result and is what many readers, abstracting services and news summaries carry forward. As written, it tells a reader that this US claims cohort found a statistically significant JAKi cancer signal. It did not. The point estimate (HR 1.36) is compatible with anything from a 6% lower hazard to a 96% higher hazard, and the paper's other JAKi analyses agree. A reader who takes only the Key Points or the abstract would count this study as independent statistical support for the ORAL Surveillance finding, when the paper's own Discussion calls the JAKi result "not statistically significant".

The rest of the paper is not affected. The HR, its CI and the direction of the JAKi estimate are reported correctly, the rituximab and abatacept results are significant as stated, and the authors' interpretation (channeling bias as a likely explanation) does not depend on the JAKi label. The error is confined to two sentences (one in the Key Points, one in the abstract), which makes it easy to correct.

Explanations considered

None of these accounts for the wording of the Key Points and abstract. Since the body of the paper states the result correctly, those two sentences are inconsistent with the paper itself.

Status

As of 2026-10-23:

Suggested correction

Bring the Key Points Findings sentence and the abstract's Conclusions and Relevance sentence into line with the Results, Discussion and main-text Conclusions. For example:

The numerical results need no change.

Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC11579790 write-up and regenerate its figures.

Paper: "Use of Biologic or Targeted Synthetic Disease-Modifying Antirheumatic Drugs
and Cancer Risk", JAMA Netw Open 2024, doi:10.1001/jamanetworkopen.2024.46336.

All inputs are copied from the article (Table 2 and Results) and from Supplement 1
(eTable 10, page 14, text layer extracted with PyMuPDF). Run:
    python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below). The candidates are for choosing a design only;
once a design is chosen, set CHOSEN, re-run, and send only writeup.md,
fig1/fig2 and this script to the journal (not the cand_* files).
"""
import os
import shutil

import numpy as np
from scipy import stats
from scipy.stats import binomtest

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.transforms import blended_transform_factory

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)

# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Adjusted HR vs TNFi (95% CI). Primary: Table 2, Abstract Results, Results.
# Propensity-matched: Results paragraph beginning "In the propensity analysis".
HR = {
    ("Primary", "Rituximab"): (1.91, 1.17, 3.14),
    ("Primary", "Abatacept"): (1.47, 1.03, 2.11),
    ("Primary", "JAKi"): (1.36, 0.94, 1.96),
    ("Primary", "IL-6i"): (1.04, 0.57, 1.92),
    ("PS-matched", "Rituximab"): (4.37, 1.48, 12.93),
    ("PS-matched", "Abatacept"): (3.12, 1.52, 6.44),
    ("PS-matched", "JAKi"): (1.79, 0.91, 3.56),
    ("PS-matched", "IL-6i"): (5.65, 1.11, 28.79),
}
# Table 2: crude cancer diagnoses, rate per 10 000 person-years (95% CI)
EVENTS = {
    "TNFi": (162, 78, 66, 91),
    "Rituximab": (14, 171, 94, 285),
    "IL-6i": (8, 88, 38, 173),
    "Abatacept": (30, 142, 96, 201),
    "JAKi": (22, 94, 59, 143),
}


# Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi, 2421 per arm per
# eTable 9): crude cancer diagnoses, rate per 10 000 PY (95% CI), adjusted HR
# as printed there (rounded to one decimal).
ETAB10 = {
    "TNFi": (14, 60, 33, 101),
    "JAKi": (22, 100, 63, 151),
}
ETAB10_HR = (1.8, 0.9, 3.6)


def wald_from_ci(hr, lo, hi):
    """Back-calculate SE(log HR), z and P from a log-scale Wald 95% CI."""
    se = (np.log(hi) - np.log(lo)) / (2 * Z975)
    z = np.log(hr) / se
    p2 = 2 * stats.norm.sf(abs(z))
    p1 = stats.norm.sf(z)
    gmid = np.exp((np.log(lo) + np.log(hi)) / 2)
    return se, z, p2, p1, gmid


print("=" * 78)
print("1. Wald back-calculation from each reported HR and 95% CI (log scale)")
print("=" * 78)
RES = {}
for key, (hr, lo, hi) in HR.items():
    se, z, p2, p1, gmid = wald_from_ci(hr, lo, hi)
    RES[key] = dict(hr=hr, lo=lo, hi=hi, se=se, z=z, p2=p2, p1=p1, gmid=gmid)
    print(f"{key[0]:10s} {key[1]:9s} HR {hr:5.2f} ({lo:.2f}-{hi:.2f})  "
          f"geo-mid {gmid:5.2f}  SE {se:.3f}  z {z:5.2f}  "
          f"P two-sided {p2:.3f}  P one-sided {p1:.3f}  CI excludes 1: {lo > 1}")

j = RES[("Primary", "JAKi")]
print("\nJAKi primary, symmetry check on the log scale:")
print(f"  ln(HR) - ln(lower) = {np.log(1.36) - np.log(0.94):.3f}")
print(f"  ln(upper) - ln(HR) = {np.log(1.96) - np.log(1.36):.3f}")
print(f"  SE(log HR) = {j['se']:.4f}; z = {j['z']:.3f}; two-sided P = {j['p2']:.3f}")
print(f"  |z| needed for two-sided P < .05: {Z975:.3f}")
print(f"  Smallest HR significant at this SE: {np.exp(Z975 * j['se']):.2f}")

print("\n" + "=" * 78)
print("2. Table 2 crude rates: implied person-years and exact Poisson 95% CI")
print("=" * 78)
PY = {}
for k, (n, r, lo, hi) in EVENTS.items():
    py = n / r * 1e4
    PY[k] = py
    l = stats.chi2.ppf(0.025, 2 * n) / 2
    u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
    print(f"{k:10s} events {n:3d}  rate {r:3d}  implied PY ~{py:6.0f}  "
          f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f}  (reported {lo}-{hi})")

n1, n0 = 22, 162
irr = 94 / 78
se_irr = np.sqrt(1 / n1 + 1 / n0)
irr_lo, irr_hi = irr * np.exp(-Z975 * se_irr), irr * np.exp(Z975 * se_irr)
pfrac = PY["JAKi"] / (PY["JAKi"] + PY["TNFi"])
p_exact = binomtest(n1, n1 + n0, pfrac).pvalue
print(f"\nCrude IRR JAKi/TNFi = 94/78 = {irr:.2f} "
      f"(Wald 95% CI {irr_lo:.2f}-{irr_hi:.2f}); "
      f"exact conditional (binomial) two-sided P = {p_exact:.2f}")

print("\n" + "=" * 78)
print("3. Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi)")
print("=" * 78)
PY10 = {}
for k, (n, r, lo, hi) in ETAB10.items():
    py = n / r * 1e4
    PY10[k] = py
    l = stats.chi2.ppf(0.025, 2 * n) / 2
    u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
    print(f"{k:10s} events {n:3d}  rate {r:3d}  implied PY ~{py:6.0f}  "
          f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f}  (reported {lo}-{hi})")
e10 = wald_from_ci(*ETAB10_HR)
print(f"eTable 10 adjusted HR {ETAB10_HR[0]} ({ETAB10_HR[1]}-{ETAB10_HR[2]}): "
      f"SE {e10[0]:.3f}, z {e10[1]:.2f}, two-sided P {e10[2]:.3f} "
      f"(bounds rounded to 1 decimal, so approximate)")
pm_ = HR[("PS-matched", "JAKi")]
print(f"  Results text gives {pm_[0]} ({pm_[1]}-{pm_[2]}); rounded to 1 decimal: "
      f"{pm_[0]:.1f} ({pm_[1]:.1f}-{pm_[2]:.1f}) -> matches eTable 10: "
      f"{(round(pm_[0], 1), round(pm_[1], 1), round(pm_[2], 1)) == ETAB10_HR}")
n1m, n0m = ETAB10["JAKi"][0], ETAB10["TNFi"][0]
irr10 = ETAB10["JAKi"][1] / ETAB10["TNFi"][1]
se10 = np.sqrt(1 / n1m + 1 / n0m)
pf10 = PY10["JAKi"] / (PY10["JAKi"] + PY10["TNFi"])
p10 = binomtest(n1m, n1m + n0m, pf10).pvalue
print(f"Crude matched rate ratio 100/60 = {irr10:.2f} (Wald 95% CI "
      f"{irr10 * np.exp(-Z975 * se10):.2f}-{irr10 * np.exp(Z975 * se10):.2f}); "
      f"exact conditional two-sided P = {p10:.2f}")

# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 2.0          # 2 px-ish lines
MS = 8.5          # marker size in points (>= 8)


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)


def fmt_p(p):
    return "<0.001" if p < 0.001 else f"{p:.3f}"


ORDER = ["Rituximab", "Abatacept", "JAKi", "IL-6i"]


# --- Candidate A: forest plot of reported HRs with recomputed P column -------
def cand_A():
    rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
    fig, ax = plt.subplots(figsize=(7.2, 5.2))
    fig.subplots_adjust(left=0.14, right=0.645, top=0.91, bottom=0.11)
    tr = blended_transform_factory(ax.transAxes, ax.transData)
    ys = []
    y = 0
    for i, key in enumerate(rows):
        if i == 4:
            y += 0.8
        ys.append(-y)
        y += 1
    for yy, key in zip(ys, rows):
        r = RES[key]
        is_j = key[1] == "JAKi"
        if is_j:
            ax.axhspan(yy - 0.45, yy + 0.45, color="#f3f1ec", zorder=0)
        ax.plot([r["lo"], r["hi"]], [yy, yy], color=ORANGE, lw=LW, zorder=2,
                solid_capstyle="butt")
        ax.plot(r["hr"], yy, "o", color=ORANGE, ms=MS, zorder=3,
                markeredgecolor=BG, markeredgewidth=1.2)
        ax.text(1.02, yy, f"{r['hr']:.2f} ({r['lo']:.2f}-{r['hi']:.2f})",
                va="center", ha="left", fontsize=10, color=INK, transform=tr)
        ax.text(1.43, yy, fmt_p(r["p2"]), va="center", ha="left", fontsize=10,
                color=BLUE, fontweight="bold" if is_j else "normal",
                transform=tr)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.set_xscale("log")
    ax.set_xlim(0.4, 40)
    ticks = [0.5, 1, 2, 5, 10, 20, 40]
    ax.set_xticks(ticks)
    ax.set_xticklabels([str(t) for t in ticks])
    ax.minorticks_off()
    ax.set_yticks(ys)
    ax.set_yticklabels([k[1] for k in rows])
    ax.set_ylim(ys[-1] - 0.8, 1.6)
    ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Hazard ratio vs TNFi (log scale); vertical line = 1")
    ax.text(0.405, ys[0] + 0.75, "Primary adjusted (Table 2)", fontsize=10,
            color=INK2, fontweight="bold")
    ax.text(0.405, ys[4] + 0.75, "Propensity-matched (Results)", fontsize=10,
            color=INK2, fontweight="bold")
    ax.text(1.02, 1.1, "Reported\nHR (95% CI)", fontsize=10, color=ORANGE,
            fontweight="bold", ha="left", va="center", transform=tr)
    ax.text(1.43, 1.1, "Recomputed\ntwo-sided P", fontsize=10, color=BLUE,
            fontweight="bold", ha="left", va="center", transform=tr)
    note = ax.text(2.15, ys[2], "Key Points/abstract call this\n"
                   "\"statistically significantly\"", fontsize=10,
                   color=INK, va="center", ha="left", linespacing=1.15)
    # check that the label stays inside the plot area and the JAKi band
    fig.canvas.draw()
    bb = note.get_window_extent().transformed(ax.transData.inverted())
    assert bb.x1 < ax.get_xlim()[1] and bb.y0 > ys[2] - 0.45 \
        and bb.y1 < ys[2] + 0.45, bb
    save(fig, "cand_A")


# --- Candidate B: back-calculated z statistics against the +/-1.96 line ------
def cand_B():
    rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
    labels = [f"{d}, {'primary' if a == 'Primary' else 'PS-matched'}"
              for a, d in rows]
    zs = [RES[k]["z"] for k in rows]
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ys = -np.arange(len(rows), dtype=float)
    ys[4:] -= 0.6
    ax.axvspan(-Z975, Z975, color="#efeee9", zorder=0)
    for xv in (-Z975, Z975):
        ax.axvline(xv, color=INK2, lw=0.9, ls=(0, (4, 3)), zorder=1)
    ax.axvline(0, color=INK2, lw=0.8, zorder=1)
    for yy, zz, k in zip(ys, zs, rows):
        ax.plot([0, zz], [yy, yy], color=BLUE, lw=LW, zorder=2,
                solid_capstyle="butt")
        ax.plot(zz, yy, "o", color=BLUE, ms=MS, zorder=3,
                markeredgecolor=BG, markeredgewidth=1.2)
        tx = zz + 0.08 if (zz + 0.08 > Z975 + 0.05 or zz < 1.0) else Z975 + 0.06
        ax.text(tx, yy, f"z = {zz:.2f}, P = {fmt_p(RES[k]['p2'])}",
                va="center", fontsize=10, color=INK)
    ax.set_yticks(ys)
    ax.set_yticklabels(labels)
    ax.set_xlim(-0.6, 4.6)
    ax.set_ylim(ys[-1] - 0.8, 2.0)
    ax.set_xlabel("z = ln(HR) / SE, back-calculated from the reported 95% CI")
    ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.text(0.68, 1.15, "Not significant at the paper's\nstated 2-sided P < .05\n"
            "(|z| < 1.96)", fontsize=10, color=INK2, ha="center", va="center")
    ax.annotate("Key Points/abstract:\n\"statistically\nsignificantly\" higher",
                xy=(j["z"], ys[2] - 0.12), xytext=(2.35, ys[3] - 0.05),
                fontsize=10,
                color=ORANGE, fontweight="bold", va="center",
                arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.0))
    fig.subplots_adjust(left=0.28, right=0.98, top=0.93, bottom=0.14)
    save(fig, "cand_B")


# --- Candidate C: two text panels joined by an annotated connector ----------
def wrap_panel(ax, x, y, w, h, title, body):
    ax.add_patch(FancyBboxPatch((x, y), w, h,
                                boxstyle="round,pad=0.008,rounding_size=0.015",
                                fc="#ffffff", ec=INK2, lw=1.0))
    ax.text(x + 0.02, y + h - 0.04, title, fontsize=11, fontweight="bold",
            va="top", color=INK)
    ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", color=INK,
            linespacing=1.4)


def cand_C():
    fig = plt.figure(figsize=(7.2, 4.6))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    left = ("Key Points, Findings:\n"
            "\"a statistically significantly higher\n"
            "risk of incident cancer was seen with\n"
            "rituximab, abatacept, and JAKis\n"
            "compared with TNFis\"\n\n"
            "Conclusions and Relevance:\n"
            "\"...rituximab, abatacept, and JAKis\n"
            "demonstrated ... statistically\n"
            "significantly increased risks\"")
    right = ("Table 2 and Results:\n"
             "JAKis (HR, 1.36; 95% CI, 0.94-1.96)\n\n"
             "Discussion:\n"
             "\"only rituximab and abatacept\n"
             "initiations had statistically\n"
             "significant associations\"\n\n"
             "Main-text Conclusions name only\n"
             "rituximab and abatacept")
    wrap_panel(ax, 0.03, 0.22, 0.40, 0.70, "Key Points and Abstract", left)
    wrap_panel(ax, 0.57, 0.22, 0.40, 0.70, "Body of the paper", right)
    arr = FancyArrowPatch((0.435, 0.57), (0.565, 0.57), arrowstyle="<->",
                          mutation_scale=14, color=INK2, lw=1.5)
    ax.add_patch(arr)
    ax.text(0.5, 0.64, "conflict", ha="center", fontsize=10, color=INK2,
            fontweight="bold")
    # bottom strip: the numbers that decide it
    ax.text(0.03, 0.13, "Reported JAKi vs TNFi HR:", fontsize=10.5, color=INK2)
    ax.text(0.30, 0.13, "1.36 (95% CI 0.94-1.96), CI includes 1", fontsize=10.5,
            color=ORANGE, fontweight="bold")
    ax.text(0.03, 0.06, "Recomputed from that CI:", fontsize=10.5, color=INK2)
    ax.text(0.30, 0.06, f"z = {j['z']:.2f}, two-sided P = {j['p2']:.2f}  "
            "(paper's threshold: P < .05)", fontsize=10.5, color=BLUE,
            fontweight="bold")
    save(fig, "cand_C")


# --- Candidate D: every JAKi estimate the paper's numbers give, one axis -----
def cand_D():
    pm = RES[("PS-matched", "JAKi")]
    p_irr = 2 * stats.norm.sf(abs(np.log(irr) / se_irr))
    items = [
        ("Adjusted HR, primary\n(Table 2)", j["hr"], j["lo"], j["hi"], ORANGE,
         "reported", j["p2"]),
        ("Adjusted HR, propensity-\nmatched (Results)", pm["hr"], pm["lo"],
         pm["hi"], ORANGE, "reported", pm["p2"]),
        ("Crude rate ratio\n94 / 78 per 10 000 PY", irr, irr_lo, irr_hi, BLUE,
         "recomputed", p_exact),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    tr = blended_transform_factory(ax.transAxes, ax.transData)
    ys = [0, -1, -2]
    for yy, (lab, est, lo, hi, col, kind, p) in zip(ys, items):
        ax.plot([lo, hi], [yy, yy], color=col, lw=LW, solid_capstyle="butt",
                zorder=2)
        ax.plot(est, yy, "o" if kind == "reported" else "D", color=col, ms=MS,
                markeredgecolor=BG, markeredgewidth=1.2, zorder=3)
        ax.text(1.04, yy, f"{est:.2f} ({lo:.2f}-{hi:.2f})", va="center",
                fontsize=10, color=INK, transform=tr)
        ax.text(1.56, yy, f"{p:.2f}", va="center", fontsize=10, color=BLUE,
                transform=tr)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.set_xscale("log")
    ax.set_xlim(0.5, 4)
    ticks = [0.5, 0.75, 1, 1.5, 2, 3, 4]
    ax.set_xticks(ticks)
    ax.set_xticklabels([str(t) for t in ticks])
    ax.minorticks_off()
    ax.set_yticks(ys)
    ax.set_yticklabels([it[0] for it in items])
    ax.set_ylim(-2.7, 1.0)
    ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("JAKi vs TNFi ratio (log scale); vertical line = 1")
    ax.text(1.04, 0.62, "Estimate\n(95% CI)", fontsize=10, color=INK2,
            fontweight="bold", transform=tr, va="center")
    ax.text(1.56, 0.62, "Recomputed\nP", va="center", fontsize=10, color=BLUE,
            fontweight="bold", transform=tr)
    ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
    ax.plot([], [], "D", color=BLUE, ms=MS,
            label="Recomputed from Table 2 rates")
    ax.legend(loc="lower center", bbox_to_anchor=(0.62, 1.02), ncol=2,
              frameon=False, fontsize=10, handlelength=1.0)
    fig.subplots_adjust(left=0.27, right=0.62, top=0.82, bottom=0.17)
    save(fig, "cand_D")


# --- Candidate E: what each section says, drug by drug -----------------------
def cand_E():
    cols = ["Key Points", "Abstract\nConclusions", "Discussion",
            "Main-text\nConclusions", "Recomputed\nP (Table 2)"]
    drugs = ["Rituximab", "Abatacept", "JAKi"]
    S, N, O = "significant", "not significant", "not named"
    says = {
        "Rituximab": [S, S, S, S],
        "Abatacept": [S, S, S, S],
        "JAKi": [S, S, N, O],
    }
    fig, ax = plt.subplots(figsize=(7.2, 2.9))
    ax.set_xlim(-0.5, len(cols) - 0.5)
    ax.set_ylim(-len(drugs) + 0.5, 1.15)
    ax.axis("off")
    for ci, c in enumerate(cols):
        ax.text(ci, 0.85, c, ha="center", va="center", fontsize=10,
                fontweight="bold", color=BLUE if ci == 4 else INK)
    ax.axhspan(-2.45, -1.55, xmin=0, xmax=1, color="#f3f1ec", zorder=0)
    for ri, d in enumerate(drugs):
        y = -ri
        ax.text(-0.52, y, d, ha="right", va="center", fontsize=10.5,
                fontweight="bold", clip_on=False)
        for ci, v in enumerate(says[d]):
            if v == S:
                ax.plot(ci - 0.42, y, "o", color=ORANGE, ms=MS)
            elif v == N:
                ax.plot(ci - 0.42, y, "o", mfc=BG, mec=ORANGE, mew=2, ms=MS)
            ax.text(ci - 0.33, y, v, va="center", fontsize=10,
                    color=INK if v != O else INK2)
        p = RES[("Primary", d)]["p2"]
        ax.text(4, y, f"{p:.3f}", ha="center", va="center", fontsize=10.5,
                color=BLUE, fontweight="bold")
    ax.plot([], [], "o", color=ORANGE, ms=MS,
            label="paper says statistically significant")
    ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=MS,
            label="paper says not significant")
    ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.02), ncol=2,
              frameon=False, fontsize=10)
    fig.subplots_adjust(left=0.15, right=0.99, top=0.97, bottom=0.18)
    save(fig, "cand_E")


cand_A()
cand_B()
cand_C()
cand_D()
cand_E()

# fig1 / fig2 are copies of the chosen candidates
CHOSEN = {"fig1": "cand_C", "fig2": "cand_A"}
for fig_name, cand in CHOSEN.items():
    for ext in ("svg", "png"):
        shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"),
                        os.path.join(HERE, f"{fig_name}.{ext}"))
print("\nFigures written to", HERE, "| chosen:", CHOSEN)

Week-24 HDRS-17 difference reported as P < .001; its t129 = 0.71 gives P = .48

JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.29427 · PMC12406064

Summary

This randomized trial compared measurement-based care (MBC) with standard care for major depressive disorder. For the secondary outcome of change in HDRS-17 score at week 24, it reports a reduction of 18.1 points (95% CI, 16.4–19.6) with MBC and 17.0 points (95% CI, 15.6–18.5) with standard care, with "t129 = 0.71; P < .001". The same values appear in the Abstract, the Results and Table 3, and the Abstract calls the difference "significantly but modestly greater". The printed statistic gives a two-sided P of 0.48: on 129 degrees of freedom, P < .001 needs |t| ≥ 3.37. Rebuilding the test from the printed group CIs gives t = 1.02 and P = 0.31, and the 95% CI of the 1.1-point difference (−1.04 to 3.24) includes zero. As printed, the Abstract presents the trial's prespecified end-of-study outcome for change in symptom severity as a significant benefit of MBC, but the paper's own numbers show no significant difference. The other week-24 efficacy results were already nonsignificant. The primary time-to-event outcomes are not affected.

What the paper reports

Abstract, Results:

After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001).

The sentence just before it in the Abstract reads:

At week 24, there were no significant differences in rates of response or remission between groups.

Results, fourth paragraph (beginning "After week 24, the response rates..."):

After week 24, the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001) (Table 3)

Table 3 (Efficacy Outcomes), last row (measure | MBC group | standard care group | P value):

Change in HDRS-17 score, mean (95% CI), points | 18.1 (16.4-19.6) | 17.0 (15.6-18.5) | <.001

Methods, Statistical Analysis:

Baseline HDRS-17 scores, antidepressant doses, number of clinic visits, changes in HDRS-17 at week 24, and number of adverse effects were compared with independent, 2-tailed t tests or Mann-Whitney tests, as appropriate. The significance threshold was 2-tailed P < .05 for all analyses.

Supplement 1 (trial protocol and statistical analysis plan), Statistical analysis, in both the protocol and the SAP section:

Analysis of covariance will be used to compare the changes in HDRS scores between the two groups.

(The protocol version reads "changes in HDRS and YMRS scores".) In the same documents, independent-sample t tests, Mann-Whitney U tests and χ² tests are planned for sociodemographic and clinical characteristics, discontinuation, response and remission rates, and side effects. The published Methods instead list changes in HDRS-17 at week 24 among the outcomes compared with independent t tests or Mann-Whitney tests.

Table 2, week 24 (MBC vs standard care): "No." 64 vs 67; "HDRS-17 score, mean (SD)" 5.6 (4.2) vs 6.7 (4.7), P value ".14".

In the archived text, "t129" is t with subscript 129 (the degrees of freedom). The publisher's HTML prints the two changes as −18.1 and −17.0 with a true minus sign. All quotes were checked verbatim against the PMC-archived article text.

What the paper's own numbers show

Method. I made three independent checks, all with scipy (see recompute.py.txt).

  1. I converted the printed statistic, t = 0.71 on 129 df, to a two-sided P value.
  2. I rebuilt the between-group test from the printed group means and 95% CIs. I took each group's standard error as (upper − lower) / (2 × t₀.₉₇₅, n−1), using the week-24 sample sizes of 64 and 67 from Table 2. These sizes also match Table 3's percentages (57/64 = 89.1%, 58/67 = 86.6%) and the printed df (64 + 67 − 2 = 129). From those SEs I ran a pooled-variance t test, and as a check a Welch test and a normal approximation. To allow for rounding, I shifted each of the six printed values by ±0.05 and recomputed t at every combination of shifts.
  3. I recomputed the Table 2 week-24 comparison of HDRS-17 levels from the printed means, SDs and sample sizes.
  4. I approximated the analysis the protocol planned, an analysis of covariance (ANCOVA) of the change with baseline HDRS-17 as covariate. From the identity SD(change)² = SD(baseline)² + SD(week 24)² − 2r·SD(baseline)·SD(week 24), the pooled SDs (baseline 4.66 from Table 2, week 24 4.46 from Table 2, change 6.17 from the Table 3 CIs) give the baseline-to-week-24 correlation r. The ANCOVA residual SD is then SD(week 24)·√(1 − r²). This is approximate: Table 2's baseline SDs (5.0 and 4.3) are for all 154 randomized participants, not only the 131 week-24 completers. The completers' baseline means implied by Table 2 and Table 3 (18.1 + 5.6 and 17.0 + 6.7) are both 23.7, so baseline adjustment leaves the 1.1-point difference unchanged.
Quantity Location Reported Recomputed
Two-sided P for t = 0.71, df = 129 Abstract; Results, 4th paragraph P < .001 0.479 (one-sided 0.239); 0.476–0.482 for t from 0.705 to 0.715
|t| needed for two-sided P < .001 at df = 129 derived — 3.37 (for P < .05: 1.98)
Degrees of freedom Abstract; Table 2 week-24 "No." 129 64 + 67 − 2 = 129, consistent with a two-group comparison
Difference in mean change, MBC − standard care Table 3 18.1 vs 17.0 1.1 points
Group SEs implied by the 95% CIs Table 3 CI widths 3.2 and 2.9 0.80 (SD 6.41) and 0.73 (SD 5.94)
Between-group t from the CIs (pooled) Table 3 t129 = 0.71; P < .001 t = 1.02, df 129, P = 0.31 (Welch t = 1.02, df 127, P = 0.31; normal approximation z = 1.00, P = 0.32)
Same, with ±0.05 rounding on all printed values Table 3 — t 0.90 to 1.15; P 0.25 to 0.37
95% CI of the 1.1-point difference derived from Table 3 not reported −1.04 to 3.24 (includes 0)
SE of the difference that P < .001 would need derived — ≤ 0.33; the CIs imply 1.08, about 3.3 times larger
SE and 95% CI implied by the printed t = 0.71 derived — SE 1.55; CI −1.97 to 4.17 (includes 0)
ANCOVA adjusted for baseline (protocol's planned analysis) Supplement 1; Tables 2–3 SDs not reported residual df 128, so F₁,₁₂₈, not t₁₂₉; implied r ≈ 0.08 (0.04 and 0.13 by group); F₁,₁₂₈ ≈ 2.0, P ≈ 0.16
Correlation the ANCOVA would need for P < .001 derived — residual SD ≤ 1.87, i.e. r ≥ 0.91 (for P < .05: r ≥ 0.70)
Week-24 HDRS-17 level, 5.6 (4.2) n = 64 vs 6.7 (4.7) n = 67 Table 2 P = .14 t = −1.41, df 129, P = 0.16 (also nonsignificant)
Within-group change from baseline (one-sample t) from Table 3 CIs — MBC t = 22.6, df 63; standard care t = 23.4, df 66; both P < 10⁻³⁰
Week-4 HDRS-17 level, 8.3 (4.9) vs 12.6 (8.1) Table 2 P < .001 t = −3.72, df 135, P = 0.0003 (consistent)
Weeks 2, 8, 12 HDRS-17 level Table 2 .05, .02, .04 0.056, 0.017, 0.034 (consistent)
Response at week 24, 57/64 vs 58/67 Table 3; Results, 4th paragraph χ² = 0.19; P = .79 Pearson χ² = 0.19, P = 0.66; Fisher exact P = 0.79
Remission at week 24, 47/64 vs 42/67 Table 3; Results, 4th paragraph χ² = 1.74; P = .20 Pearson χ² = 1.74, P = 0.19; Fisher exact P = 0.20
Response at week 12, 58/64 vs 46/68 Table 3 P < .001 Pearson P = 0.0012; Fisher exact P = 0.0013

Every check of the week-24 change comparison gives a P value between about 0.16 and 0.48. None comes near .001. The other HDRS-17 comparisons in Table 2 and the response and remission comparisons in Table 3 reproduce closely; the dose and visit comparisons in Table 2 were not checked. Two of them have small discrepancies that do not change significance: the week-24 rate P values match Fisher's exact test rather than the Pearson χ² printed beside them, and the week-12 response P is 0.0012–0.0013, not < .001. Those two points are outside the scope of this report. The printed t of 0.71 is not exactly reproducible from the printed CIs either (they give t ≈ 1.02). Whichever of the two is used, the result is clearly nonsignificant.

Figure

Two-sided P values for the week-24 HDRS-17 comparison: reported versus recomputed

Figure 1. Two-sided P values for MBC versus standard care at week 24, on one log-scaled axis. The thin vertical line marks the paper's significance threshold of P = .05. Orange marks values printed in the paper and blue marks values recomputed from the paper's own numbers. The top row is the printed P < .001 for the difference in HDRS-17 change; the arrow shows that the printed value is an upper bound. The next three rows recompute the same comparison: from the printed statistic t129 = 0.71 (P = 0.48); from the printed group 95% CIs (P = 0.31; the short bar is the range when each printed value is shifted by ±0.05 for rounding, 0.25–0.37); and as the ANCOVA planned in the protocol, approximated from the printed SDs (P ≈ 0.16 at the implied baseline-to-week-24 correlation r ≈ 0.08). Each row's value is printed to the left of the threshold line, joined to its marker by a dotted leader. The bottom two rows show the closely related week-24 comparison of HDRS-17 levels in Table 2, as printed (P = .14) and as recomputed (P = 0.16). Only the printed P for the change score lies below the threshold.

The 1.1-point difference and its 95% CI, recomputed versus what P < .001 would require

Figure 2. The between-group difference in week-24 HDRS-17 change (MBC minus standard care, 1.1 points), on a points scale with zero (no difference) marked. Blue: the 95% CI of the difference implied by the printed group CIs (−1.04 to 3.24) and by the printed t129 = 0.71 (−1.97 to 4.17). Both include zero. Orange, dashed: the widest 95% CI that a 1.1-point difference could have if the printed P < .001 were correct (0.45 to 1.75, SE ≤ 0.33). That interval is about one third as wide as the one the paper's own CIs imply.

Why it matters

Change in HDRS-17 score at week 24 is a prespecified secondary outcome: the trial's end-of-study outcome for change in symptom severity. The Abstract reports it as the one positive week-24 efficacy finding ("significantly but modestly greater ... P < .001"), immediately after stating that week-24 response and remission rates did not differ. With the P value corrected, no week-24 efficacy outcome favours MBC significantly: response P = .79, remission P = .20, HDRS-17 level P = .14 (Table 2), and HDRS-17 change P ≈ .48 (from the printed t), ≈ .31 (from the printed CIs) or about .16 (the protocol's ANCOVA, approximated from the printed SDs). The difference still favours MBC by 1.1 points, but its 95% CI runs from about −1 to +3 points. That interval is compatible with no difference.

As printed, a reader of the Abstract would conclude that MBC produced both faster improvement and a reliably greater reduction in severity by the end of the study. The paper's own numbers support only the first. The error is repeated in three places (Abstract, Results, Table 3), with no correct value given anywhere in the paper.

Not affected: the primary outcomes, time to response and time to remission (log-rank P = .001 for both; Cox HR 1.53 and 1.80); the week-12 rate differences; and the formal Conclusions, which rest on the time-to-event results. The Key Points do not cite this result.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The authors should check the analysis output for the week-24 change comparison. The P value is 0.48 if the printed t129 = 0.71 is right, and about 0.31 if the printed group CIs are right; whichever value the analysis produces should be reported with the matching statistic. Adding the between-group difference and its 95% CI would let readers check the result directly. Using the printed t:

Abstract, Results. Replace "After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (... t129 = 0.71; P < .001)." with:

After week 24, reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48).

Results, fourth paragraph. Replace "the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (... t129 = 0.71; P < .001)" with:

the reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48)

Table 3, last row. Replace the P value "<.001" with ".48", or with the value from the verified analysis.

With these changes, the Abstract sentence "No differences were observed in other secondary outcomes" would also cover HDRS-17 change. The Conclusions do not need to change.

Recomputation script (python recompute.py.txt)
"""Recompute the week-24 between-group test of HDRS-17 change in PMC12406064
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.29427) from the paper's own
printed numbers, run the cross-checks used in the write-up, and redraw fig1/fig2
in this file's folder.

Run:  python recompute.py.txt      (needs numpy, scipy, matplotlib)
Every input below is copied from the published article (Abstract, Results,
Methods, Table 2, Table 3), except the analysis-plan quote in section 2b, which is
from Supplement 1 (trial protocol and statistical analysis plan).
"""
import os
import itertools
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
# Abstract, Results (fourth paragraph) and Table 3 (last row): change in HDRS-17 score, mean (95% CI)
M1, LO1, HI1 = 18.1, 16.4, 19.6      # MBC
M2, LO2, HI2 = 17.0, 15.6, 18.5      # standard care
T_REP, DF_REP = 0.71, 129            # "t129 = 0.71"
P_REP = 0.001                        # "P < .001"
# Table 2, week 24 "No." row (consistent with Table 3: 57/64 = 89.1%, 58/67 = 86.6%)
N1, N2 = 64, 67
# Table 2: HDRS-17 mean (SD), n, reported P   (MBC first, then standard care)
TABLE2 = {
    "Baseline": ((24.1, 5.0, 76), (23.4, 4.3, 78), None),
    "Wk 2":     ((12.2, 5.3, 68), (14.6, 8.9, 72), ".05"),
    "Wk 4":     ((8.3, 4.9, 66), (12.6, 8.1, 71), "<.001"),
    "Wk 8":     ((7.7, 5.2, 64), (10.4, 7.4, 68), ".02"),
    "Wk 12":    ((6.9, 5.3, 64), (9.2, 6.9, 68), ".04"),
    "Wk 24":    ((5.6, 4.2, 64), (6.7, 4.7, 67), ".14"),
}
# Table 3 / Results (fourth paragraph): response and remission counts, reported chi2 and P
TABLE3 = [
    ("Response wk 12", (58, 64), (46, 68), None, "<.001"),
    ("Response wk 24", (57, 64), (58, 67), 0.19, ".79"),
    ("Remission wk 12", (46, 64), (35, 68), None, ".02"),
    ("Remission wk 24", (47, 64), (42, 67), 1.74, ".20"),
]

R = {}  # numbers reused by the figures


def p2(t, df):
    return 2 * stats.t.sf(abs(t), df)


print("=== 1. The reported statistic: t129 = 0.71 ===")
R["p_from_t"] = p2(T_REP, DF_REP)
R["t_crit_001"] = stats.t.isf(0.0005, DF_REP)
R["t_crit_05"] = stats.t.isf(0.025, DF_REP)
print(f"df check: n1 + n2 - 2 = {N1} + {N2} - 2 = {N1 + N2 - 2} (reported {DF_REP})")
print(f"two-sided P for t = {T_REP}, df = {DF_REP}: {R['p_from_t']:.3f}  (one-sided {R['p_from_t']/2:.3f})")
print(f"|t| needed for two-sided P < .001 at df = {DF_REP}: {R['t_crit_001']:.2f}")
print(f"|t| needed for two-sided P < .05  at df = {DF_REP}: {R['t_crit_05']:.2f}")
print(f"largest two-sided P for t in [0.705, 0.715): "
      f"{p2(0.705, DF_REP):.3f}; smallest: {p2(0.715, DF_REP):.3f}")

print("\n=== 2. Between-group test rebuilt from the reported group 95% CIs ===")
q1, q2 = stats.t.ppf(0.975, N1 - 1), stats.t.ppf(0.975, N2 - 1)
se1, se2 = (HI1 - LO1) / (2 * q1), (HI2 - LO2) / (2 * q2)
sd1, sd2 = se1 * np.sqrt(N1), se2 * np.sqrt(N2)
diff = M1 - M2
print(f"MBC: CI width {HI1-LO1:.1f} -> SE {se1:.3f}, implied SD {sd1:.2f} "
      f"(CI extends {M1-LO1:.1f} below, {HI1-M1:.1f} above the mean)")
print(f"SC : CI width {HI2-LO2:.1f} -> SE {se2:.3f}, implied SD {sd2:.2f} "
      f"(CI extends {M2-LO2:.1f} below, {HI2-M2:.1f} above the mean)")
print(f"difference in mean change: {M1} - {M2} = {diff:.1f} points")
# pooled-variance (Student) t test, df = 129, as the reported subscript implies
sp = np.sqrt(((N1 - 1) * sd1**2 + (N2 - 1) * sd2**2) / (N1 + N2 - 2))
se_p = sp * np.sqrt(1 / N1 + 1 / N2)
t_p = diff / se_p
R["t_ci"], R["p_ci"], R["se_ci"] = t_p, p2(t_p, N1 + N2 - 2), se_p
qd = stats.t.ppf(0.975, N1 + N2 - 2)
R["ci_diff"] = (diff - qd * se_p, diff + qd * se_p)
print(f"Student: pooled SD {sp:.2f}, SE(diff) {se_p:.3f}, t = {t_p:.2f}, df = {N1+N2-2}, "
      f"P = {R['p_ci']:.2f}")
print(f"         95% CI of the difference: {R['ci_diff'][0]:.2f} to {R['ci_diff'][1]:.2f}")
# Welch
se_w = np.sqrt(se1**2 + se2**2)
df_w = se_w**4 / (se1**4 / (N1 - 1) + se2**4 / (N2 - 1))
t_w = diff / se_w
print(f"Welch  : SE(diff) {se_w:.3f}, t = {t_w:.2f}, df = {df_w:.0f}, P = {p2(t_w, df_w):.2f}")
# normal approximation (SE = width / 3.92)
se_z = np.sqrt(((HI1 - LO1) / 3.92) ** 2 + ((HI2 - LO2) / 3.92) ** 2)
z = diff / se_z
print(f"z-approx: SE(diff) {se_z:.3f}, z = {z:.2f}, P = {2*stats.norm.sf(z):.2f}")

# rounding: every printed value may be off by up to 0.05; take all corners
ts = []
for a, b, l1, h1, l2, h2 in itertools.product(*[(-0.05, 0.05)] * 6):
    s1 = ((HI1 + h1) - (LO1 + l1)) / (2 * q1)
    s2 = ((HI2 + h2) - (LO2 + l2)) / (2 * q2)
    ts.append(((M1 + a) - (M2 + b)) / np.sqrt(s1**2 + s2**2))
R["t_rng"] = (min(ts), max(ts))
R["p_rng"] = (p2(max(ts), DF_REP), p2(min(ts), DF_REP))
print(f"rounding (+/-0.05 on all six printed values): t {R['t_rng'][0]:.2f} to {R['t_rng'][1]:.2f}, "
      f"two-sided P {R['p_rng'][0]:.2f} to {R['p_rng'][1]:.2f}")
R["se_needed"] = diff / R["t_crit_001"]
print(f"SE(diff) a 1.1-point difference would need for P < .001: {R['se_needed']:.2f} "
      f"(the CIs imply {se_p:.2f}, about {se_p / R['se_needed']:.1f} times larger)")
R["se_from_t"] = diff / T_REP
R["ci_from_t"] = (diff - qd * R["se_from_t"], diff + qd * R["se_from_t"])
print(f"SE(diff) implied by t = 0.71 and diff = 1.1: {R['se_from_t']:.2f} "
      f"(pooled SD {R['se_from_t'] / np.sqrt(1/N1 + 1/N2):.2f}); "
      f"95% CI {R['ci_from_t'][0]:.2f} to {R['ci_from_t'][1]:.2f}")
R["ci_needed"] = (diff - qd * R["se_needed"], diff + qd * R["se_needed"])
print(f"95% CI of the difference if P were exactly .001: "
      f"{R['ci_needed'][0]:.2f} to {R['ci_needed'][1]:.2f}")

print(f"standardized difference: {diff:.1f} / pooled SD {sp:.2f} = {diff/sp:.2f}")
t_half = diff / (se_p / 2)
print(f"if both group SEs were halved: t = {t_half:.2f}, P = {p2(t_half, DF_REP):.3f}")

print("\n=== 2b. The protocol's planned analysis: ANCOVA adjusted for baseline ===")
# Supplement 1 (protocol and SAP): "Analysis of covariance will be used to compare the
# changes in HDRS scores between the two groups."  The published Methods instead name
# independent t tests or Mann-Whitney tests.  An ANCOVA with group + baseline HDRS-17 would
# report F on 1 and N - 3 df, not t on N - 2 df.
DF_ANC = N1 + N2 - 3
print(f"residual df of a two-group model adjusted for baseline: {N1} + {N2} - 3 = {DF_ANC} "
      f"(printed subscript {DF_REP}); statistic would be F(1, {DF_ANC}), not t{DF_REP}")
(bx1, bsx1, bn1), (bx2, bsx2, bn2), _ = TABLE2["Baseline"]
(wy1, wsy1, _), (wy2, wsy2, _), _ = TABLE2["Wk 24"]
# baseline SDs are for all randomized participants (n 76/78), week-24 SDs for completers
sx = np.sqrt(((bn1 - 1) * bsx1**2 + (bn2 - 1) * bsx2**2) / (bn1 + bn2 - 2))
sy = np.sqrt(((N1 - 1) * wsy1**2 + (N2 - 1) * wsy2**2) / (N1 + N2 - 2))
sd_chg = sp                                   # pooled change SD implied by Table 3 CIs
r_pool = (sx**2 + sy**2 - sd_chg**2) / (2 * sx * sy)
r_g1 = (bsx1**2 + wsy1**2 - sd1**2) / (2 * bsx1 * wsy1)
r_g2 = (bsx2**2 + wsy2**2 - sd2**2) / (2 * bsx2 * wsy2)
print(f"pooled SDs: baseline {sx:.2f} (Table 2, all randomized), week 24 {sy:.2f} (Table 2), "
      f"change {sd_chg:.2f} (from Table 3 CIs)")
print(f"implied baseline-to-week-24 correlation r = (sx^2 + sy^2 - s_change^2) / (2 sx sy) "
      f"= {r_pool:.2f}  (per group: MBC {r_g1:.2f}, standard care {r_g2:.2f})")
# completers' baseline means implied by Table 3 change + Table 2 week-24 mean
cb1, cb2 = M1 + wy1, M2 + wy2
print(f"completers' baseline means implied: MBC {M1} + {wy1} = {cb1:.1f}, "
      f"SC {M2} + {wy2} = {cb2:.1f} (so baseline adjustment shifts the 1.1-point difference "
      f"by {r_pool * sy / sx * abs(cb1 - cb2):.2f}; adjusted difference = {wy2 - wy1:.1f})")
k = np.sqrt(1 / N1 + 1 / N2)


def ancova(r):
    beta = r * sy / sx
    # adjusted difference in reduction = -(Y1 - Y2) + beta * (X1 - X2), completers' baselines
    adj = (wy2 - wy1) + beta * (cb1 - cb2)
    s_res = sy * np.sqrt(1 - r**2) * np.sqrt((N1 + N2 - 2) / DF_ANC)
    t = adj / (s_res * k)
    return s_res, t, p2(t, DF_ANC)


s_res, t_anc, p_anc = ancova(r_pool)
R["p_ancova"], R["r_pool"] = p_anc, r_pool
print(f"ANCOVA at r = {r_pool:.2f}: residual SD {s_res:.2f}, SE(adjusted diff) {s_res*k:.3f}, "
      f"t = {t_anc:.2f}, F(1, {DF_ANC}) = {t_anc**2:.2f}, P = {p_anc:.2f}")
t_need = stats.t.isf(0.0005, DF_ANC)
sres_need = diff / (t_need * k)
r_need = np.sqrt(1 - (sres_need / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
t_need05 = stats.t.isf(0.025, DF_ANC)
r_need05 = np.sqrt(1 - (diff / (t_need05 * k) / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
R["r_need"] = r_need
print(f"for ANCOVA P < .001 (|t| >= {t_need:.2f}, F >= {t_need**2:.1f}): residual SD <= "
      f"{sres_need:.2f}, i.e. r >= {r_need:.2f}; for P < .05: r >= {r_need05:.2f}")
for rr in (0.0, 0.3, 0.5, 0.7, 0.9):
    _, tt, pp = ancova(rr)
    print(f"   r = {rr:.1f}: t = {tt:.2f}, F = {tt**2:.2f}, P = {pp:.3f}")

print("\n=== 3. Alternative reading: within-group change from baseline ===")
for name, m, se, n in [("MBC", M1, se1, N1), ("SC", M2, se2, N2)]:
    tt = m / se
    print(f"{name}: one-sample t = {tt:.1f}, df = {n-1}, P = {p2(tt, n-1):.0e}")

print("\n=== 4. Table 2 cross-checks (Student t from mean, SD, n) ===")
for lbl, ((ma, sa, na), (mb, sb, nb), rep) in TABLE2.items():
    s = np.sqrt(((na - 1) * sa**2 + (nb - 1) * sb**2) / (na + nb - 2))
    tt = (ma - mb) / (s * np.sqrt(1 / na + 1 / nb))
    pp = p2(tt, na + nb - 2)
    tw = (ma - mb) / np.sqrt(sa**2 / na + sb**2 / nb)
    dfw = (sa**2/na + sb**2/nb)**2 / ((sa**2/na)**2/(na-1) + (sb**2/nb)**2/(nb-1))
    if lbl == "Wk 24":
        R["p_wk24_level"] = pp
    print(f"{lbl:8s}: diff {ma-mb:+.1f}, t = {tt:.2f}, df = {na+nb-2}, P = {pp:.4f}; "
          f"Welch P = {p2(tw, dfw):.4f}  (reported {rep})")
print(f"Table 2 means: MBC 24.1 - 5.6 = {24.1-5.6:.1f}, SC 23.4 - 6.7 = {23.4-6.7:.1f} "
      f"(Table 3 change, completers: 18.1 and 17.0)")

print("\n=== 5. Table 3 rates (Pearson chi2, Yates chi2, Fisher exact) ===")
for lbl, (a, na), (b, nb), chi_rep, rep in TABLE3:
    tab = np.array([[a, na - a], [b, nb - b]])
    c0 = stats.chi2_contingency(tab, correction=False)
    c1 = stats.chi2_contingency(tab, correction=True)
    pf = stats.fisher_exact(tab)[1]
    print(f"{lbl:16s}: {a}/{na} = {100*a/na:.1f}% vs {b}/{nb} = {100*b/nb:.1f}%; "
          f"Pearson chi2 = {c0[0]:.2f}, P = {c0[1]:.4f}; Yates chi2 = {c1[0]:.2f}, P = {c1[1]:.4f}; "
          f"Fisher P = {pf:.4f}  [reported chi2 {chi_rep}, P {rep}]")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
                     "text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2,
                     "ytick.color": INK, "axes.edgecolor": INK2, "svg.fonttype": "none"})
LW = 1.5     # 2 px at 96 px/in
MS = 9


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
    plt.close(fig)


def style(ax):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)


# Fig 1: two-sided P values on one log axis
rows = [
    ("Week-24 HDRS-17 change\nas printed (Abstract, Results, Table 3)", P_REP, None, "rep"),
    ("Same test: P from the printed\nstatistic $t_{129}$ = 0.71", R["p_from_t"], None, "rec"),
    ("Same test: rebuilt from the\nprinted group 95% CIs", R["p_ci"], R["p_rng"], "rec"),
    ("Protocol's ANCOVA, approximated\nfrom the printed SDs", R["p_ancova"], None, "rec"),
    ("Week-24 HDRS-17 level\n(Table 2), as printed", 0.14, None, "rep"),
    ("Week-24 HDRS-17 level\n(Table 2), recomputed", R["p_wk24_level"], None, "rec"),
]
labels = ["P < .001",
          f"P = {R['p_from_t']:.2f}",
          f"P = {R['p_ci']:.2f} (rounding range {R['p_rng'][0]:.2f}-{R['p_rng'][1]:.2f})",
          f"P \u2248 {R['p_ancova']:.2f} ($F_{{1,128}}$ \u2248 {t_anc**2:.1f}, r \u2248 {R['r_pool']:.2f})",
          "P = .14",
          f"P = {R['p_wk24_level']:.2f}"]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
style(ax)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1.0)
ys = np.arange(len(rows))[::-1].astype(float)
ys[4:] -= 0.35
ax.axvline(0.05, color=INK2, lw=0.9, zorder=1)
ax.text(0.05, ys[0] + 0.62, "P = .05 (paper's threshold) ", color=INK2, fontsize=10,
        ha="right", va="center")
XLAB = 0.04   # value labels end left of the threshold line
for y, (lab, p, rng, kind), txt in zip(ys, rows, labels):
    c = ORANGE if kind == "rep" else BLUE
    if rng is not None:
        ax.plot(rng, [y, y], color=c, lw=LW, solid_capstyle="round", zorder=2)
    if p == P_REP and kind == "rep":
        ax.annotate("", xy=(1.6e-4, y), xytext=(P_REP, y),
                    arrowprops=dict(arrowstyle="-|>", color=c, lw=LW, mutation_scale=12), zorder=2)
        ax.text(p * 1.35, y, txt, color=INK, fontsize=10, va="center")
    else:
        ax.plot([XLAB * 1.1, (rng[0] if rng else p) / 1.12], [y, y], color=GRID, lw=0.9,
                ls=":", zorder=1)
        ax.text(XLAB, y, txt, color=INK, fontsize=10, ha="right", va="center")
    ax.plot([p], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.9)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_xlabel("Two-sided P value, MBC vs standard care (log scale)")
handles = [Line2D([], [], ls="", marker="o", ms=MS, color=ORANGE, label="Reported in the paper"),
           Line2D([], [], ls="", marker="o", ms=MS, color=BLUE,
                  label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
          fontsize=10, handletextpad=0.3)
save(fig, "fig1")

# Fig 2: the 1.1-point difference and its 95% CI on one axis (HDRS-17 points)
rows2 = [
    ("95% CI implied by the\nprinted group CIs", R["ci_diff"], BLUE, "-"),
    ("95% CI implied by the\nprinted $t_{129}$ = 0.71", R["ci_from_t"], BLUE, "-"),
    ("Widest 95% CI compatible\nwith the printed P < .001", R["ci_needed"], ORANGE, "--"),
]
fig, ax = plt.subplots(figsize=(7.2, 2.9))
style(ax)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(0, 2.75, "no difference ", color=INK2, fontsize=10, ha="right", va="center")
for y, (lab, (lo, hi), c, ls) in zip([2, 1, 0], rows2):
    ax.plot([lo, hi], [y, y], color=c, lw=LW, ls=ls, solid_capstyle="round", zorder=2)
    ax.plot([diff], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
    ax.text(hi + 0.12, y, f"{diff:.1f} ({lo:.2f} to {hi:.2f})", color=INK, fontsize=10, va="center")
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_ylim(-0.6, 3.0)
ax.set_xlim(-3, 7)
ax.set_xlabel("Difference in week-24 HDRS-17 change, MBC minus standard care (points)")
handles = [Line2D([], [], color=ORANGE, lw=LW, ls="--", marker="o", ms=MS,
                  label="What the reported P < .001 requires"),
           Line2D([], [], color=BLUE, lw=LW, marker="o", ms=MS,
                  label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
          fontsize=10, handletextpad=0.3)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)

Abstract reports overall-cohort length-of-stay estimates as the results for hypotensive patients

JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.33266 · PMC12457976

Summary

The abstract states that, among patients with hypotension on admission, splenic angioembolization (SAE) and observation were each associated with a shorter hospital length of stay (HLOS) than open splenectomy (OS): SAE β = -1.44 (95% CI, -1.79 to -1.09; P < .001) and observation β = -1.41 (95% CI, -1.73 to -1.09; P < .001). These six numbers are identical to the paper's Table 2 rows for the total cohort (n = 7567). Table 2's rows for the hypotension subgroup (SBP <90 mm Hg, n = 651) are different: SAE β = -1.65 (-3.24 to -0.06; P = .04) and observation β = -0.13 (-1.58 to 1.32; P = .86). The Results text reports only the SAE figure for this subgroup. For observation, the abstract therefore reports a highly significant benefit of about 1.4 days where the paper's own subgroup analysis, and its inverse-probability-weighted (IPW) sensitivity analysis, show no significant difference. The claim is repeated in words in the main-text Conclusions, and it supports the headline statement that nonoperative management had "favorable outcomes ... even in patients with hypotension on admission".

What the paper reports

Abstract, Results:

Among patients with hypotension, there was no mortality difference, but shorter hospital length of stay was found in the SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) groups.

Results, "Multivariable Analysis for Other Outcomes". This paragraph gives the same two estimates for the total cohort, then reports the hypotension subgroup separately and names only SAE:

Multivariable linear regression analysis of continuous variable outcomes is given in Table 2. SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) showed a negative association with HLOS days.

The same results were observed in normal SBP subgroup. In the hypotension subgroup, SAE was also negatively associated with HLOS days (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) and ICULOS days (β = -1.45; 95% CI, -2.62 to -0.29; P = .01).

Table 2, "SAE or Observation vs OS Outcomes Using Multivariable Linear Regression and IPW". Columns: outcome | No. | mean (SD) | regression β (95% CI) | P value | IPW β (95% CI) | P value. Total block, HLOS days:

SAE | 1547 | 6.35 (4.17) | -1.44 (-1.79 to -1.09) | <.001 | -1.77 (-2.24 to -1.30) | <.001

Observation | 4521 | 6.04 (5.00) | -1.41 (-1.73 to -1.09) | <.001 | -1.60 (-2.05 to -1.14) | <.001

Table 2, SBP <90 mm Hg block, HLOS days (block total 651; OS row 323 | 9.40 (8.31) | 0 [Reference]):

SAE | 124 | 7.48 (4.80) | -1.65 (-3.24 to -0.06) | .04 | -1.67 (-2.89 to -0.46) | .007

Observation | 204 | 8.74 (8.01) | -0.13 (-1.58 to 1.32) | .86 | -0.48 (-2.00 to 1.04) | .54

Main-text Conclusions:

Furthermore, in patients with hypotension on admission, nonoperative management (either with SAE or observation) was associated with no increased risk of complications or mortality and shorter LOS.

Key Points, "Findings" (the abstract's Conclusions and Relevance uses the same closing phrase):

In this cohort study of 7567 patients, nonoperative management (angioembolization or observation) was associated with favorable outcomes when compared with surgery in isolated severe blunt splenic injury, even in patients with hypotension on admission.

What the paper's own numbers show

Method. I compared the abstract's two "hypotension" estimate-and-CI triples with every HLOS row in Table 2 (total, SBP <90, SBP ≥90 and failure blocks, regression and IPW columns). I then checked each row's internal consistency with a Wald approximation from the printed interval: SE = (upper − lower)/(2 × 1.96), z = β/SE, two-sided P = 2Φ(−|z|). I checked that the block sizes add up, and I compared CI widths with the group sizes. As a further check I computed the unadjusted mean differences and Welch t tests from the Table 2 means and SDs. Patient-level data are not available, so the adjusted models cannot be refitted. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
HLOS β, SAE vs OS, "among patients with hypotension" Abstract -1.44 (-1.79 to -1.09), P < .001 Identical to the Table 2 Total row; matches no SBP <90 row
HLOS β, observation vs OS, "among patients with hypotension" Abstract -1.41 (-1.73 to -1.09), P < .001 Identical to the Table 2 Total row; matches no SBP <90 row
HLOS β, SAE, hypotension subgroup Table 2, SBP <90; Results -1.65 (-3.24 to -0.06), P = .04 SE 0.811, z = -2.03, P = 0.042: consistent with itself
HLOS β, observation, hypotension subgroup Table 2, SBP <90 -0.13 (-1.58 to 1.32), P = .86 SE 0.740, z = -0.18, P = 0.861; CI includes 0
IPW HLOS β, SAE / observation, hypotension subgroup Table 2, SBP <90, IPW -1.67 (-2.89 to -0.46), P = .007 / -0.48 (-2.00 to 1.04), P = .54 P = 0.007 / 0.536; observation CI includes 0
HLOS β, total cohort, SAE / observation Table 2, Total -1.44 (-1.79 to -1.09) / -1.41 (-1.73 to -1.09), both P < .001 z = -8.06 / -8.64, both P < 10⁻¹⁵
HLOS β, normotension subgroup, SAE / observation Table 2, SBP ≥90 -1.44 (-1.80 to -1.09) / -1.53 (-1.86 to -1.20) Close to, but not identical with, the abstract (observation -1.53 vs -1.41)
Abstract's failure-subgroup HLOS figures (control) Abstract vs Table 2, Failure 2.50 (1.27-3.73); 0.71 (0.07-1.35) Identical to the Table 2 Failure rows: the abstract's other subgroup result is transcribed correctly
Group sizes, HLOS blocks Table 2 7567; 651; 6916 1499 + 1547 + 4521 = 7567; 323 + 124 + 204 = 651; 1176 + 1423 + 4317 = 6916; 651 + 6916 = 7567
CI width, abstract vs SBP <90 row Abstract; Table 2 0.70 vs 3.18 (SAE); 0.64 vs 2.90 (observation) The subgroup intervals are 4.5 times as wide as the abstract's
Residual SD implied by each CI, SE/√(1/n_arm + 1/n_OS) Table 2 HLOS SD 7.69 (SBP <90), 5.27 (total) SBP <90 rows: 7.68 and 8.27; total rows: 4.93 and 5.48. The abstract's CIs placed at the SBP <90 group sizes would imply 1.69 and 1.83, so covariates would have to explain about 95% of the HLOS variance in that subgroup, while the table's own subgroup rows show almost no such reduction
ICULOS β, observation, hypotension subgroup Table 2, SBP <90 -0.49 (-1.59 to 0.61), P = .38 P = 0.383; CI includes 0
Unadjusted mean HLOS difference vs OS, hypotension subgroup Table 2 means (SD) OS 9.40 (8.31), SAE 7.48 (4.80), observation 8.74 (8.01) SAE -1.92 days (Welch P = 0.003); observation -0.66 days (Welch P = 0.364)

Each Table 2 row is internally consistent: the printed P values follow from the printed intervals, and the block sizes add to the cohort total. The table is not misaligned. The discrepancy lies between the abstract sentence and the subgroup rows it describes. The CI-width check points the same way. The abstract's intervals are as narrow as one would expect from about 3000-6000 patients per comparison, not from the 447 (SAE) and 527 (observation) patients in the hypotension comparisons.

Figure

The abstract's "hypotension" length-of-stay estimates are the Table 2 total-cohort values; the Table 2 hypotension rows differ

Figure 1. The abstract's sentence on patients with hypotension (top, orange outline) quoted next to the two Table 2 blocks it could refer to. All six numbers, for both arms, match the Total-cohort block (n = 7567) exactly. The SBP <90 mm Hg block (n = 651), which is the subgroup the sentence names, gives a different SAE estimate with an interval 4.5 times as wide, and an observation estimate of -0.13 days whose 95% CI includes 0. The blue markers give the P values recomputed from each printed interval with a Wald approximation. They agree with the table's own P values (.04 and .86), so the table rows are internally consistent.

Forest plot of the abstract's hypotension HLOS estimates, the Table 2 SBP <90 rows and the Table 2 total-cohort rows on one axis

Figure 2. The same estimates on one axis, in days of hospital stay relative to open splenectomy. The orange squares are the abstract's "hypotension" values, the blue circles are the Table 2 SBP <90 rows, and the hollow gray circles are the Table 2 total-cohort rows. The orange and gray intervals coincide. For observation, the subgroup interval (blue) crosses the null line at 0, and the abstract's (orange) does not. The right-hand column gives each printed P value, with the Wald P recomputed from the printed CI for the subgroup rows.

Why it matters

Whether nonoperative management, and observation in particular, is reasonable for hemodynamically unstable patients with high-grade splenic injury is a clinically consequential question. The abstract is the part of the paper most readers rely on. In the hypotension subgroup, Table 2 and the Results show no difference in mortality (HR 0.95 for SAE, 1.08 for observation) or in any complication (OR 0.71, P = .30; OR 1.15, P = .62). That leaves length of stay as the only outcome on which the subgroup shows a benefit. For observation, that benefit is not present in the paper's own subgroup analysis: β = -0.13 days (95% CI, -1.58 to 1.32; P = .86), with IPW giving -0.48 days (P = .54). The unadjusted difference is also small and not significant (-0.66 days, P = .36). The main-text Conclusions ("nonoperative management (either with SAE or observation) was associated with ... shorter LOS") and the Key Points' "favorable outcomes ... even in patients with hypotension on admission" both rest partly on the abstract's version.

For SAE the direction is unchanged, but the abstract overstates the precision. The subgroup estimate is -1.65 days with a CI reaching almost to 0 (-0.06) and P = .04, not P < .001 with a 0.7-day-wide interval.

The paper's other main findings do not depend on this sentence. These are lower morbidity and shorter stay in the overall cohort, the same in the normotension subgroup, no mortality difference, and the failure-subgroup results. A null result does not show that observation has no effect on length of stay in hypotensive patients. It shows only that this analysis does not support the significant effect the abstract reports.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the Abstract, Results, replace the hypotension sentence with the Table 2 SBP <90 mm Hg values, for example: "Among patients with hypotension, there was no mortality difference; hospital length of stay was shorter in the SAE group (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) but not significantly different in the observation group (β = -0.13; 95% CI, -1.58 to 1.32; P = .86)." The main-text Conclusions should then be revised: in patients with hypotension, shorter LOS was found for SAE only. The phrase "even in patients with hypotension on admission" in the Key Points and in the Abstract's Conclusions and Relevance should be checked against the subgroup results as corrected. In that subgroup, mortality and complications did not differ, and the LOS benefit applies to SAE only.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12457976 (JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.33266)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (fig1, fig2 and three alternative designs, candidate3-5) as SVG and 200-dpi PNG
# in this script's own folder. Every input below is copied from the article as
# archived in PubMed Central (abstract and Table 2).
import os
import math
import numpy as np
from scipy.stats import norm, ttest_ind_from_stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs
# Abstract, Results: "Among patients with hypotension ... SAE (...) and observation (...)"
ABSTRACT = {"SAE": (-1.44, -1.79, -1.09, "<.001"),
            "Observation": (-1.41, -1.73, -1.09, "<.001")}

# Table 2, HLOS days rows: (n, mean, SD, beta, lo, hi, P, IPW beta, IPW lo, IPW hi, IPW P)
T2 = {
    "Total": {"OS": (1499, 8.38, 6.53),
              "SAE": (1547, 6.35, 4.17, -1.44, -1.79, -1.09, "<.001", -1.77, -2.24, -1.30, "<.001"),
              "Observation": (4521, 6.04, 5.00, -1.41, -1.73, -1.09, "<.001", -1.60, -2.05, -1.14, "<.001")},
    "SBP <90": {"OS": (323, 9.40, 8.31),
                "SAE": (124, 7.48, 4.80, -1.65, -3.24, -0.06, ".04", -1.67, -2.89, -0.46, ".007"),
                "Observation": (204, 8.74, 8.01, -0.13, -1.58, 1.32, ".86", -0.48, -2.00, 1.04, ".54")},
    "SBP >=90": {"OS": (1176, 8.10, 5.93),
                 "SAE": (1423, 6.25, 4.10, -1.44, -1.80, -1.09, "<.001", -1.80, -2.31, -1.28, "<.001"),
                 "Observation": (4317, 5.91, 4.77, -1.53, -1.86, -1.20, "<.001", -1.83, -2.30, -1.35, "<.001")},
    "Failure": {"OS": (1499, 8.38, 6.53),
                "SAE": (105, 10.76, 6.07, 2.50, 1.27, 3.73, "<.001", 2.44, 1.14, 3.75, "<.001"),
                "Observation": (608, 8.88, 6.96, 0.71, 0.07, 1.35, ".03", 1.20, 0.19, 2.21, ".02")},
}
# Table 2, SBP <90, ICULOS days (used for the Conclusions' "shorter LOS")
ICU_HYPO = {"SAE": (-1.45, -2.62, -0.29, ".01"), "Observation": (-0.49, -1.59, 0.61, ".38")}
SUBGROUP_HLOS_N = {"Total": 7567, "SBP <90": 651, "SBP >=90": 6916}
SUBGROUP_HLOS_SD = {"Total": 5.27, "SBP <90": 7.69, "SBP >=90": 4.93}


def wald(b, lo, hi):
    se = (hi - lo) / (2 * Z975)
    z = b / se
    return se, z, 2 * norm.sf(abs(z))


def fmt_p(p):
    return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"


# ---------------------------------------------------------------- 1. Wald check of every row
print("1. Wald check, SE = (upper - lower) / (2 x 1.96), z = beta / SE, two-sided P")
rows = [("Abstract 'hypotension'", a, *ABSTRACT[a]) for a in ("SAE", "Observation")]
for g in T2:
    for a in ("SAE", "Observation"):
        r = T2[g][a]
        rows.append((f"Table 2 {g} MLR", a, r[3], r[4], r[5], r[6]))
        rows.append((f"Table 2 {g} IPW", a, r[7], r[8], r[9], r[10]))
for a in ("SAE", "Observation"):
    rows.append(("Table 2 SBP <90 ICULOS MLR", a, *ICU_HYPO[a]))
WALD = {}
for src, arm, b, lo, hi, p in rows:
    se, z, pc = wald(b, lo, hi)
    WALD[(src, arm)] = (se, z, pc)
    print(f"   {src:28s} {arm:11s} beta={b:6.2f} CI=({lo:5.2f},{hi:5.2f}) printed P={p:6s} "
          f"SE={se:.3f} z={z:6.2f} recomputed P={fmt_p(pc):8s} CI excludes 0: {lo * hi > 0}")

# ---------------------------------------------------------------- 2. which block do the abstract numbers match?
print("\n2. Does the abstract's 'hypotension' triple (beta, lower, upper) equal a Table 2 HLOS row?")
for g in T2:
    for col, idx in (("MLR", (3, 4, 5)), ("IPW", (7, 8, 9))):
        m = [tuple(T2[g][a][i] for i in idx) == ABSTRACT[a][:3] for a in ("SAE", "Observation")]
        print(f"   {g:9s} {col}: SAE match={m[0]!s:5s} Observation match={m[1]}")
print("   (control) Abstract failure-subgroup HLOS: SAE 2.50 (1.27-3.73), observation 0.71 (0.07-1.35)"
      f" = Table 2 Failure rows: {(T2['Failure']['SAE'][3:6] == (2.50, 1.27, 3.73)) and (T2['Failure']['Observation'][3:6] == (0.71, 0.07, 1.35))}")

# ---------------------------------------------------------------- 3. sample sizes
print("\n3. Sample sizes of the HLOS blocks")
for g in ("Total", "SBP <90", "SBP >=90"):
    s = sum(T2[g][a][0] for a in ("OS", "SAE", "Observation"))
    print(f"   {g:9s}: {T2[g]['OS'][0]} + {T2[g]['SAE'][0]} + {T2[g]['Observation'][0]} = {s} "
          f"(printed {SUBGROUP_HLOS_N[g]})")
print(f"   651 + 6916 = {651 + 6916}")
for g in ("Total", "SBP <90"):
    for a in ("SAE", "Observation"):
        print(f"   patients in the {g} {a} vs OS comparison: {T2[g][a][0]} + {T2[g]['OS'][0]} = {T2[g][a][0] + T2[g]['OS'][0]}")
print(f"   normotensive share of the total cohort: 6916 / 7567 = {6916 / 7567:.1%}")

# ---------------------------------------------------------------- 4. CI widths
print("\n4. CI widths: abstract vs the hypotension rows")
CIW = {}
for a in ("SAE", "Observation"):
    wa = ABSTRACT[a][2] - ABSTRACT[a][1]
    r = T2["SBP <90"][a]
    wh = r[5] - r[4]
    CIW[a] = (wa, wh)
    print(f"   {a:11s}: abstract width {wa:.2f}; Table 2 SBP <90 width {wh:.2f}; ratio {wh / wa:.1f}")

# unadjusted SE of a difference in means and the residual SD each CI implies
print("\n5. Residual SD implied by each CI, SE / sqrt(1/n_arm + 1/n_OS)  (rough: ignores covariates' effect on precision)")
IMPLIED = {}
for g in ("Total", "SBP <90", "SBP >=90"):
    for a in ("SAE", "Observation"):
        r, os_ = T2[g][a], T2[g]["OS"]
        k = math.sqrt(1 / r[0] + 1 / os_[0])
        se = wald(r[3], r[4], r[5])[0]
        IMPLIED[(g, a)] = (k, se, se / k)
        print(f"   Table 2 {g:9s} {a:11s}: k={k:.4f} SE={se:.3f} implied residual SD={se / k:.2f} "
              f"(block SD of HLOS {SUBGROUP_HLOS_SD[g]})")
for a in ("SAE", "Observation"):
    r, os_ = T2["SBP <90"][a], T2["SBP <90"]["OS"]
    k = math.sqrt(1 / r[0] + 1 / os_[0])
    se = wald(*ABSTRACT[a][:3])[0]
    sd_raw = math.sqrt(((r[0] - 1) * r[2] ** 2 + (os_[0] - 1) * os_[2] ** 2) / (r[0] + os_[0] - 2))
    r2 = 1 - (se / k) ** 2 / sd_raw ** 2
    IMPLIED[("Abstract", a)] = (k, se, se / k)
    print(f"   Abstract CI placed in the SBP <90 subgroup, {a:11s}: k={k:.4f} SE={se:.3f} implied residual "
          f"SD={se / k:.2f}; pooled SD of the two arms {sd_raw:.2f}; covariates would need R^2={r2:.2f}")

# ---------------------------------------------------------------- 6. unadjusted differences
print("\n6. Unadjusted mean HLOS difference vs OS (Table 2 means and SDs), Welch t test")
for g in ("SBP <90", "Total"):
    os_ = T2[g]["OS"]
    for a in ("SAE", "Observation"):
        r = T2[g][a]
        t, p = ttest_ind_from_stats(r[1], r[2], r[0], os_[1], os_[2], os_[0], equal_var=False)
        print(f"   {g:7s} {a:11s}: {r[1]:.2f} - {os_[1]:.2f} = {r[1] - os_[1]:+.2f} days; Welch t={t:.2f}, P={fmt_p(p)}")

# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
                     "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
                     "savefig.facecolor": BG, "lines.linewidth": 2, "svg.fonttype": "none"})
M = "−"  # minus sign


def n(x, d=2):
    return f"{x:.{d}f}".replace("-", M)


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
    plt.close(fig)
    print(f"   wrote {name}.svg and {name}.png")


def box(ax, x, y, w, h, edge, lw=2):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc="#ffffff", ec=edge, lw=lw, transform=ax.transAxes))


print("\n7. Figures")
pS, pO = (T2["SBP <90"]["SAE"], T2["SBP <90"]["Observation"])
tS, tO = (T2["Total"]["SAE"], T2["Total"]["Observation"])
wS = WALD[("Table 2 SBP <90 MLR", "SAE")]
wO = WALD[("Table 2 SBP <90 MLR", "Observation")]

# ---------------- fig1: text panels (design 1)
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
box(ax, 0.02, 0.72, 0.96, 0.26, ORANGE)
ax.text(0.04, 0.95, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.04, 0.89,
        "\u201cAmong patients with hypotension, there was no mortality difference, but shorter\n"
        "hospital length of stay was found in the SAE (\u03b2 = " + n(-1.44) + "; 95% CI, " + n(-1.79) + " to " + n(-1.09) +
        "; P < .001)\nand observation (\u03b2 = " + n(-1.41) + "; 95% CI, " + n(-1.73) + " to " + n(-1.09) +
        "; P < .001) groups.\u201d", fontsize=10.5, va="top", linespacing=1.45)
for x0, title, lines in (
        (0.02, "Table 2, Total cohort, n = 7567",
         [("SAE", f"{n(tS[3])} ({n(tS[4])} to {n(tS[5])}), P < .001", None),
          ("Observation", f"{n(tO[3])} ({n(tO[4])} to {n(tO[5])}), P < .001", None)]),
        (0.52, "Table 2, SBP <90 mm Hg, n = 651",
         [("SAE", f"{n(pS[3])} ({n(pS[4])} to {n(pS[5])}), P = .04", f"recomputed P = {wS[2]:.3f}"),
          ("Observation", f"{n(pO[3])} ({n(pO[4])} to {n(pO[5])}), P = .86",
           f"recomputed P = {wO[2]:.2f}; CI includes 0")])):
    box(ax, x0, 0.095, 0.46, 0.395, GRAY)
    ax.text(x0 + 0.02, 0.465, title, fontsize=11, weight="bold", va="top")
    ax.text(x0 + 0.02, 0.41, "HLOS \u03b2 vs open splenectomy, days (95% CI)", fontsize=10, color=INK2, va="top")
    for i, (arm, est, rec) in enumerate(lines):
        y = 0.355 - i * 0.13
        ax.text(x0 + 0.02, y, arm, fontsize=10.5, va="top", weight="bold")
        ax.text(x0 + 0.02, y - 0.042, est, fontsize=10.5, va="top")
        if rec:
            ax.plot([x0 + 0.03], [y - 0.1], "o", ms=8, color=BLUE, clip_on=False)
            ax.text(x0 + 0.05, y - 0.1, rec, fontsize=10, color=INK2, va="center")
# connectors
ax.annotate("", xy=(0.25, 0.495), xytext=(0.25, 0.715), xycoords="data",
            arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
ax.text(0.27, 0.605, "identical, digit for digit\n(all six numbers)", fontsize=10, va="center", color=INK)
ax.annotate("", xy=(0.75, 0.495), xytext=(0.75, 0.715), xycoords="data",
            arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, linestyle=(0, (4, 3))))
ax.text(0.77, 0.605, "the subgroup the sentence\nnames: numbers differ", fontsize=10, va="center", color=INK)
# legend
h = [Line2D([], [], color=ORANGE, lw=2, label="claim as reported in the abstract"),
     Line2D([], [], marker="o", ms=8, color=BLUE, lw=0, label="P recomputed from the printed 95% CI")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.005), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")


# ---------------- fig2: forest plot on one axis (design 2)
def forest(ax, entries, xlim):
    ax.axvline(0, color=INK2, lw=0.8, zorder=1)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for y, (lab, b, lo, hi, style, note) in entries.items():
        col = {"abs": ORANGE, "sub": BLUE, "tot": GRAY}[style]
        ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="round", zorder=2)
        ax.plot([b], [y], "s" if style == "abs" else "o", ms=9, color=col,
                mfc=col if style != "tot" else BG, mew=2, zorder=3)
        ax.text(xlim[1] + 0.08, y, note, va="center", fontsize=10, color=INK, clip_on=False)
    ax.set_yticks(list(entries.keys()), [e[0] for e in entries.values()], fontsize=10.5, color=INK)
    ax.tick_params(axis="y", length=0)
    ax.set_xlim(*xlim)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)


fig, ax = plt.subplots(figsize=(7.2, 4.3))
ent = {}
y = 7.6
for arm, sub, tot in (("SAE", pS, tS), ("Observation", pO, tO)):
    a = ABSTRACT[arm]
    w = WALD[("Table 2 SBP <90 MLR", arm)]
    ent[y] = (f"{arm}: abstract, ‘hypotension’", a[0], a[1], a[2], "abs", "P < .001")
    ent[y - 1] = (f"{arm}: Table 2, SBP <90 (n = 651)", sub[3], sub[4], sub[5], "sub",
                  f"P = {sub[6]}  (recomputed {w[2]:.3f})")
    ent[y - 2] = (f"{arm}: Table 2, total (n = 7567)", tot[3], tot[4], tot[5], "tot", "P < .001")
    y -= 4
forest(ax, ent, (-3.6, 1.6))
ax.set_ylim(0.9, 8.3)
ax.set_xlabel("Adjusted difference in hospital length of stay vs open splenectomy, days (β, 95% CI)")
ax.text(0.05, 8.25, "longer stay →", fontsize=10, color=INK2, va="bottom")
ax.text(-0.05, 8.25, "← shorter stay", fontsize=10, color=INK2, va="bottom", ha="right")
h = [Line2D([], [], marker="s", ms=9, color=ORANGE, lw=2, label="abstract (reported)"),
     Line2D([], [], marker="o", ms=9, color=BLUE, lw=2, label="Table 2 hypotension rows, P recomputed from CI"),
     Line2D([], [], marker="o", ms=9, color=GRAY, mfc=BG, mew=2, lw=2, label="Table 2 total cohort")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.32, -0.16), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")

# ---------------- candidate3: z statistics on one axis (design 3)
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ax.axvspan(-1.96, 1.96, color="#efeeea", zorder=0)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0, 4.75, "|z| < 1.96: P > .05", ha="center", fontsize=10, color=INK2,
        bbox=dict(fc="#efeeea", ec="none", pad=1))
yl = []
for i, arm in enumerate(("SAE", "Observation")):
    za = WALD[("Abstract 'hypotension'", arm)][1]
    zs = WALD[("Table 2 SBP <90 MLR", arm)][1]
    y0 = 3.6 - i * 2.2
    ax.plot([za], [y0], "s", ms=10, color=ORANGE)
    ax.text(za + 0.25, y0, f"z = {n(za)}", va="center", fontsize=10)
    ax.plot([zs], [y0 - 0.8], "o", ms=10, color=BLUE)
    ax.text(zs + 0.3, y0 - 0.8, f"z = {n(zs)}, P = {WALD[('Table 2 SBP <90 MLR', arm)][2]:.2f}", va="center", fontsize=10,
            bbox=dict(fc="#efeeea", ec="none", pad=1), zorder=4)
    yl += [(y0, f"{arm}: abstract"), (y0 - 0.8, f"{arm}: Table 2, SBP <90")]
ax.set_yticks([a for a, _ in yl], [b for _, b in yl], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(-9.5, 4.5); ax.set_ylim(0.3, 5.1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.set_xlabel("Wald z = β / SE, with SE = (upper − lower) / 3.92 from each printed 95% CI")
h = [Line2D([], [], marker="s", ms=10, color=ORANGE, lw=0, label="from the abstract's CI (reported)"),
     Line2D([], [], marker="o", ms=10, color=BLUE, lw=0, label="from the Table 2 hypotension CI (recomputed)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "candidate3")

# ---------------- candidate4: lookup grid (design 4)
fig = plt.figure(figsize=(7.2, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
cols = [0.03, 0.36, 0.68]
ax.text(cols[0], 0.93, "Table 2 HLOS row", weight="bold", fontsize=10.5)
ax.text(cols[1], 0.93, "SAE β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[2], 0.93, "Observation β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[0], 0.82, "Abstract, ‘hypotension’", fontsize=10.5, color=INK)
for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
    a = ABSTRACT[arm]
    ax.plot([c - 0.012], [0.835], "s", ms=8, color=ORANGE, transform=ax.transAxes)
    ax.text(c + 0.005, 0.82, f"{n(a[0])} ({n(a[1])} to {n(a[2])})", fontsize=10.5)
ax.plot([0.02, 0.98], [0.78, 0.78], color=INK2, lw=0.8, transform=ax.transAxes)
y = 0.70
for g, lab in (("Total", "Total, n = 7567"), ("SBP <90", "SBP <90, n = 651"), ("SBP >=90", "SBP ≥90, n = 6916")):
    for col, idx, cl in (("MLR", (3, 4, 5), "regression"), ("IPW", (7, 8, 9), "IPW")):
        ax.text(cols[0], y, f"{lab}, {cl}", fontsize=10.5, color=INK if g != "SBP <90" else INK, weight="bold" if g == "SBP <90" else "normal")
        for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
            r = T2[g][arm]
            tri = tuple(r[i] for i in idx)
            same = tri == ABSTRACT[arm][:3]
            ax.text(c + 0.005, y, f"{n(tri[0])} ({n(tri[1])} to {n(tri[2])})", fontsize=10.5)
            if same:
                ax.add_patch(FancyBboxPatch((c - 0.01, y - 0.025), 0.28, 0.075, boxstyle="round,pad=0,rounding_size=0.01",
                                            fc="none", ec=ORANGE, lw=2, transform=ax.transAxes))
        y -= 0.1
ax.text(0.03, 0.06, "Orange outline: identical to the abstract\u2019s \u2018hypotension\u2019 numbers. Only the Total-cohort\n"
        "regression rows match; the SBP <90 rows (bold), which the sentence describes, do not.", fontsize=10, color=INK2)
save(fig, "candidate4")

# ---------------- candidate5: CI width vs sample size (design 5)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
labels = {("Total", "SAE"): "Total SAE", ("Total", "Observation"): "Total Obs",
          ("SBP >=90", "SAE"): "SBP≥90 SAE", ("SBP >=90", "Observation"): "SBP≥90 Obs",
          ("SBP <90", "SAE"): "SBP<90 SAE", ("SBP <90", "Observation"): "SBP<90 Obs"}
off = {("Total", "SAE"): (6, 6), ("Total", "Observation"): (6, -12), ("SBP >=90", "SAE"): (-70, 6),
       ("SBP >=90", "Observation"): (-70, -12), ("SBP <90", "SAE"): (8, 0), ("SBP <90", "Observation"): (8, -4)}
for key, lab in labels.items():
    k, se, _ = IMPLIED[key]
    ax.plot([k], [2 * Z975 * se], "o", ms=9, color=BLUE, zorder=3)
    if key[0] == "SBP <90":
        ax.annotate(lab, (k, 2 * Z975 * se), xytext=off[key], textcoords="offset points", fontsize=10, va="center")
ax.annotate("Total and SBP \u226590 rows\n(SAE and observation, 4 points)", (0.035, 0.72), xytext=(0.035, 1.35),
            ha="center", fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
for arm, dy in (("SAE", 8), ("Observation", -12)):
    k, se, _ = IMPLIED[("Abstract", arm)]
    ax.plot([k], [2 * Z975 * se], "s", ms=9, color=ORANGE, zorder=3)
    ax.annotate(f"abstract ‘hypotension’ {arm}", (k, 2 * Z975 * se), xytext=(8, dy),
                textcoords="offset points", fontsize=10, va="center")
ax.set_xlim(0, 0.13); ax.set_ylim(0, 3.5)
ax.set_xlabel(r"$\sqrt{1/n_{\rm arm} + 1/n_{\rm OS}}$, from the Table 2 group sizes")
ax.set_ylabel("Width of the printed 95% CI, days")
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
h = [Line2D([], [], marker="o", ms=9, color=BLUE, lw=0, label="Table 2 HLOS rows (width from the printed CI)"),
     Line2D([], [], marker="s", ms=9, color=ORANGE, lw=0, label="abstract CI, placed at the hypotension group sizes")]
ax.legend(handles=h, loc="upper left", frameon=False, fontsize=10)
save(fig, "candidate5")

Risk-difference confidence intervals for the primary outcome are about 23 to 31 times too narrow

JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.25965 · PMC13434869

Summary

The Abstract and the Results report the trial's primary-outcome effect sizes, the absolute differences in first-pass intubation success between direct laryngoscopy (DL), Macintosh-blade video laryngoscopy (VLM) and hyperangulated video laryngoscopy (VLH), with 95% confidence intervals 0.22 to 0.33 percentage points wide, for example 4.66 (95% CI, 4.45-4.78) for VLM vs DL (written "DL vs VLM" in the paper). The paper's own Table 2 counts (625/799, 673/812, 711/812) reproduce every point estimate and, to within 0.03, the paper's own z statistics (exactly with a pooled standard error), but with the method the paper names (an unpooled z approach), they give intervals 6.9 to 7.7 points wide: 0.80 to 8.52, 5.69 to 12.99 and 1.24 to 8.12. The numbers needed to treat in the Discussion carry the same narrow intervals (VLM 21.5, reported 20.9-22.2; from the counts, 11.7-125.2). The direction and statistical significance of every comparison are unchanged, but the published intervals overstate the precision of the trial's headline result 23- to 31-fold. That matters most for VLM, where the data are compatible with a gain of anywhere from 0.8 to 8.5 points.

What the paper reports

Differences between success rates were tested applying the z test for unpooled variance and reported as absolute differences with corresponding 95% CIs.

Abstract, Main Outcomes and Measure

Unadjusted absolute risk differences for first-pass intubation success were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.

Abstract, Results

In 2-sided z tests, VLM and VLH were significantly more effective in facilitating first-pass success than the DL (VLM vs DL: z = 2.36, P = .02; VLH vs DL: z = 4.98, P < .001). VLH was in turn superior to VLM (z = 2.66; P = .008). Unadjusted absolute risk differences for a successful first attempt were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.

Results, Primary End Point and Crossover Between Groups

Relative risks (RRs) for successful first-attempt intubation were significantly higher for both VLM and VLH over DL (VLM vs DL: RR, 1.06 [95% CI, 1.01-1.11]; VLH vs DL: RR, 1.12 [95% CI, 1.07-1.17]) and for VLM over VLH (RR, 1.06 [95% CI, 1.01-1.10]) (eTable 2 in Supplement 1).

Results, same paragraph

Numbers needed to treat were 21.5 (95% CI, 20.9-22.2) for VLM and 10.7 (95% CI, 10.6-10.8) for VLH.

Discussion

Table 2 ("First-Pass Success Rates of Tracheal Intubations Using Different Laryngoscopes"), mITT population, row "Yes": DL 625 (78.2), VLM 673 (82.9), VLH 711 (87.6); row "Total": 799, 812, 812.

What the paper's own numbers show

Method. For each pair of arms I took the Table 2 counts and computed the risk difference, its unpooled standard error √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂), and the 95% Wald interval RD ± 1.96·SE, which is the interval that goes with the unpooled z test the paper says it used. As checks I also computed the Newcombe (hybrid Wilson score) and Agresti–Caffo intervals, the z statistics (unpooled and pooled), the log-scale Wald interval for the relative risk, and the NNT interval obtained by inverting the risk-difference limits. Finally, I worked backwards from the reported intervals: dividing a reported width by 2 × 1.96 gives the standard error that interval implies, and dividing the reported risk difference by the reported z gives a second implied standard error. Comparisons are labelled higher-success arm vs lower-success arm (VLM vs DL, VLH vs DL, VLH vs VLM); the paper writes them as DL vs VLM, DL vs VLH and VLM vs VLH, with the same positive values. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
First-pass success, DL / VLM / VLH Table 2 625/799 (78.2), 673/812 (82.9), 711/812 (87.6) 78.22%, 82.88%, 87.56%
RD, VLM vs DL (pp) Abstract; Results 4.66 4.66
95% CI, VLM vs DL Abstract; Results 4.45 to 4.78 (width 0.33) Wald 0.80 to 8.52 (width 7.72); Newcombe 0.79 to 8.52; Agresti–Caffo 0.79 to 8.51
RD, VLH vs DL (pp) Abstract; Results 9.34 9.34
95% CI, VLH vs DL Abstract; Results 9.22 to 9.46 (width 0.24) Wald 5.69 to 12.99 (width 7.31); Newcombe 5.68 to 12.99; Agresti–Caffo 5.66 to 12.97
RD, VLH vs VLM (pp) Abstract; Results 4.68 4.68
95% CI, VLH vs VLM Abstract; Results 4.57 to 4.79 (width 0.22) Wald 1.24 to 8.12 (width 6.89); Newcombe 1.23 to 8.13; Agresti–Caffo 1.22 to 8.12
Recomputed width ÷ reported width derived — 23.4×, 30.4×, 31.3×
z, VLM vs DL Results 2.36 (P = .02) unpooled 2.37 (P = .018); pooled 2.36
z, VLH vs DL Results 4.98 (P < .001) unpooled 5.01 (P = 5.4 × 10⁻⁷); pooled 4.98
z, VLH vs VLM Results 2.66 (P = .008) unpooled 2.66 (P = .0077); pooled 2.66
SE implied by reported RD ÷ reported z (pp) derived 1.975, 1.876, 1.759 unpooled SE from counts 1.970, 1.864, 1.757
SE implied by reported CI width ÷ 3.92 (pp) derived 0.084, 0.061, 0.056 —
z that the reported CIs would imply derived — 55.4, 152.6, 83.4
Per-arm sample size at which a Wald CI would be as narrow as reported derived — about 441,000, 746,000 and 796,000 (trial: 799–812)
Reported CI, distance below / above point Abstract; Results VLM vs DL −0.21 / +0.12; others −0.12 / +0.12 and −0.11 / +0.11 Wald is symmetric (±3.86, ±3.65, ±3.44)
RR, VLM vs DL Results; eTable 2 1.06 (1.01–1.11) 1.060 (1.010–1.112)
RR, VLH vs DL Results; eTable 2 1.12 (1.07–1.17) 1.119 (1.070–1.171)
RR, VLH vs VLM Results; eTable 2 1.06 (1.01–1.10) 1.056 (1.014–1.100)
NNT, VLM vs DL Discussion 21.5 (20.9–22.2) 21.5 (11.7–125.2)
NNT, VLH vs DL Discussion 10.7 (10.6–10.8) 10.7 (7.7–17.6)
NNT limits from inverting the reported RD CIs derived — VLM 20.9–22.5; VLH 10.6–10.8

Two independent routes give the same standard errors. The counts give 1.97, 1.86 and 1.76 points, and the paper's own risk differences divided by its own z statistics give 1.98, 1.88 and 1.76 points. The reported intervals imply standard errors of 0.06 to 0.08 points, about 23 to 31 times smaller. The relative-risk intervals in the same paragraph and in eTable 2 match the counts to the printed decimal, so the precision the paper reports on the ratio scale agrees with the wide intervals, not the narrow ones. The NNT intervals match the reciprocals of the reported risk-difference limits (exactly for VLH; for VLM the upper limit is 22.2 against 22.5), so they inherit the same error.

Figure

Reported vs recomputed 95% CIs for the three first-pass success risk differences

Figure 1. Each row is one of the three primary-outcome comparisons. Orange diamonds show the risk difference and 95% CI reported in the Abstract and Results. The reported intervals are narrower than the marker, so the bars are hidden behind it; the right-hand labels give their limits. Blue circles with capped bars show the 95% CI recomputed from the Table 2 counts with an unpooled Wald interval, the method the paper states. The solid gray line marks no difference and the dashed line marks the −5 point noninferiority margin. Every recomputed interval lies to the right of both, so significance and noninferiority are unchanged, but each is 23 to 31 times wider than reported. The axis is linear and includes zero.

Reported vs recomputed 95% CIs for the numbers needed to treat

Figure 2. Numbers needed to treat with VLM or VLH instead of DL to gain one additional first-pass success, as reported in the Discussion (orange diamonds) and as obtained by inverting the recomputed Wald risk-difference limits (blue circles). The axis is logarithmic because the NNT is a reciprocal. For VLM, the interval the counts support runs from about 12 to about 125 intubations, against a reported 20.9 to 22.2.

Why it matters

Explanations considered

Two smaller observations, which do not affect the main finding:

Status

As of 2026-10-23:

Sources: the article in PubMed Central (PMC13434869) and at doi:10.1001/jamanetworkopen.2026.25965; Supplements 1, 2 and 4 from the Europe PMC supplementary files; ClinicalTrials.gov NCT05228288; Crossref and Europe PMC metadata.

Suggested correction

Replace the risk-difference intervals in the Abstract and in the Results (Primary End Point and Crossover Between Groups) with intervals computed from the Table 2 counts by the stated method. With the unpooled Wald interval these are:

and replace the Discussion's NNT intervals with 21.5 (95% CI, 11.7–125.2) for VLM and 10.7 (95% CI, 7.7–17.6) for VLH. If a score-based interval is preferred, the Newcombe limits (0.79–8.52, 5.68–12.99, 1.23–8.13) differ only in the second decimal. Optionally, the authors could also report the noninferiority z statistics against the −5 point margin, state whether the reported z statistics used pooled or unpooled variance, and change "for VLM over VLH" to "for VLH over VLM".

Recomputation script (python recompute.py.txt)
# Recomputation for PMC13434869 (JAMA Netw Open 2026; doi:10.1001/jamanetworkopen.2026.25965)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the paper's own numbers only.
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)  # 1.959964

# ---- Inputs from the paper -------------------------------------------------
# Table 2 (mITT): first-pass success, No./total
mitt = {"DL": (625, 799), "VLM": (673, 812), "VLH": (711, 812)}
# eTable 1 (Supplement 1, per-protocol): first-pass success
pp = {"DL": (590, 758), "VLM": (663, 799), "VLH": (704, 804)}
# Abstract and Results: risk differences (pp) with 95% CI; Results: z statistics
# keys are (lower-success arm, higher-success arm)
reported = {
    ("DL", "VLM"): dict(rd=4.66, lo=4.45, hi=4.78, z=2.36, label="VLM vs DL"),
    ("DL", "VLH"): dict(rd=9.34, lo=9.22, hi=9.46, z=4.98, label="VLH vs DL"),
    ("VLM", "VLH"): dict(rd=4.68, lo=4.57, hi=4.79, z=2.66, label="VLH vs VLM"),
}
# Results / eTable 2: relative risks (mITT and per-protocol)
rr_rep = {("DL", "VLM"): (1.06, 1.01, 1.11), ("DL", "VLH"): (1.12, 1.07, 1.17),
          ("VLM", "VLH"): (1.06, 1.01, 1.10)}
rr_rep_pp = {("DL", "VLM"): (1.07, 1.01, 1.12), ("DL", "VLH"): (1.12, 1.07, 1.18),
             ("VLM", "VLH"): (1.06, 1.01, 1.10)}
# Discussion: numbers needed to treat (vs DL)
nnt_rep = {"VLM": (21.5, 20.9, 22.2), "VLH": (10.7, 10.6, 10.8)}
NI_MARGIN = -0.05  # Methods: noninferiority margin, 5 percentage points


def wilson(x, n, z=Z):
    p = x / n
    c = (p + z * z / (2 * n)) / (1 + z * z / n)
    h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
    return c - h, c + h


def newcombe(xb, nb, xa, na):
    """Newcombe hybrid score interval for pb - pa."""
    pb, pa = xb / nb, xa / na
    l1, u1 = wilson(xb, nb)
    l2, u2 = wilson(xa, na)
    d = pb - pa
    return d - np.sqrt((pb - l1) ** 2 + (u2 - pa) ** 2), d + np.sqrt((u1 - pb) ** 2 + (pa - l2) ** 2)


def agresti_caffo(xb, nb, xa, na):
    pb, pa = (xb + 1) / (nb + 2), (xa + 1) / (na + 2)
    d = pb - pa
    se = np.sqrt(pb * (1 - pb) / (nb + 2) + pa * (1 - pa) / (na + 2))
    return d - Z * se, d + Z * se


def compare(data, a, b):
    xa, na = data[a]
    xb, nb = data[b]
    pa, pb = xa / na, xb / nb
    rd = pb - pa
    se = np.sqrt(pa * (1 - pa) / na + pb * (1 - pb) / nb)       # unpooled (stated method)
    ppool = (xa + xb) / (na + nb)
    se_pool = np.sqrt(ppool * (1 - ppool) * (1 / na + 1 / nb))   # pooled, for comparison
    rr = pb / pa
    se_lrr = np.sqrt(1 / xb - 1 / nb + 1 / xa - 1 / na)
    return dict(pa=pa, pb=pb, rd=rd, se=se, se_pool=se_pool,
                wald=(rd - Z * se, rd + Z * se), newc=newcombe(xb, nb, xa, na),
                ac=agresti_caffo(xb, nb, xa, na),
                z=rd / se, z_pool=rd / se_pool, z_ni=(rd - NI_MARGIN) / se,
                rr=rr, rr_ci=(rr * np.exp(-Z * se_lrr), rr * np.exp(Z * se_lrr)))


print("=== Proportions (Table 2, mITT) ===")
for k, (x, n) in mitt.items():
    print(f"  {k}: {x}/{n} = {100 * x / n:.2f}%")

res = {}
print("\n=== Risk differences, mITT (percentage points) ===")
for (a, b), r in reported.items():
    c = compare(mitt, a, b)
    res[(a, b)] = c
    rep_w = r["hi"] - r["lo"]
    w = 100 * (c["wald"][1] - c["wald"][0])
    se_rep_ci = rep_w / (2 * Z)          # SE implied by reported CI width (pp)
    se_rep_z = r["rd"] / r["z"]          # SE implied by reported RD and z (pp)
    print(f"{r['label']}: p = {100*c['pa']:.2f}% vs {100*c['pb']:.2f}%")
    print(f"  RD            reported {r['rd']:.2f}   recomputed {100*c['rd']:.2f}")
    print(f"  95% CI        reported {r['lo']:.2f} to {r['hi']:.2f} (width {rep_w:.2f})")
    print(f"     Wald (unpooled)     {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f} (width {w:.2f})")
    print(f"     Newcombe            {100*c['newc'][0]:.2f} to {100*c['newc'][1]:.2f}")
    print(f"     Agresti-Caffo       {100*c['ac'][0]:.2f} to {100*c['ac'][1]:.2f}")
    print(f"  Recomputed width / reported width = {w / rep_w:.1f}x")
    print(f"  Reported CI distances from point: -{r['rd']-r['lo']:.2f} / +{r['hi']-r['rd']:.2f}")
    print(f"  SE: recomputed {100*c['se']:.3f} pp; implied by reported z {se_rep_z:.3f} pp; "
          f"implied by reported CI {se_rep_ci:.4f} pp")
    print(f"  z implied by reported CI = {r['rd']/se_rep_ci:.1f}")
    print(f"  z superiority: reported {r['z']:.2f}; unpooled {c['z']:.2f} "
          f"(2-sided P = {2*stats.norm.sf(c['z']):.2g}, 1-sided P = {stats.norm.sf(c['z']):.2g}); "
          f"pooled {c['z_pool']:.2f}")
    print(f"  z noninferiority vs -5 pp margin (unpooled) = {c['z_ni']:.2f}")
    rr = rr_rep[(a, b)]
    print(f"  RR reported {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f}); recomputed "
          f"{c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f})")
    # sample size per arm at which the Wald CI would be as narrow as reported
    n_needed = (np.mean([mitt[a][1], mitt[b][1]])) * (100 * c['se'] / se_rep_ci) ** 2
    print(f"  Per-arm n for a CI this narrow at these rates: about {n_needed:,.0f}")

print("\n=== Further derived quantities ===")
for (a, b), r in reported.items():
    c = res[(a, b)]
    w_ratio = 100 * (c["wald"][1] - c["wald"][0]) / (r["hi"] - r["lo"])
    half = (r["hi"] - r["lo"]) / 2
    conf = 2 * stats.norm.cdf(half / (100 * c["se"])) - 1
    print(f"  {r['label']}: Wald half-width +/-{100*Z*c['se']:.2f} pp; inverse-variance weight ratio "
          f"(reported/correct) = {w_ratio**2:.0f}x; confidence level of an interval of the reported "
          f"width at the true SE = {100*conf:.1f}%")

print("\n=== Per-protocol (eTable 1): risk differences and RRs ===")
for (a, b), r in reported.items():
    c = compare(pp, a, b)
    rr = rr_rep_pp[(a, b)]
    print(f"{r['label']}: RD {100*c['rd']:.2f} pp, Wald {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}; "
          f"RR {c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f}) vs eTable 2 {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f})")

print("\n=== Numbers needed to treat (vs DL), CI by inverting the Wald RD limits ===")
nnt = {}
for arm, key in (("VLM", ("DL", "VLM")), ("VLH", ("DL", "VLH"))):
    c = res[key]
    est, lo, hi = 1 / c["rd"], 1 / c["wald"][1], 1 / c["wald"][0]
    nnt[arm] = (est, lo, hi)
    r = nnt_rep[arm]
    print(f"  {arm}: reported {r[0]} ({r[1]}-{r[2]}); recomputed {est:.1f} ({lo:.1f}-{hi:.1f}); "
          f"inverting the reported RD CI gives {100/reported[key]['hi']:.1f}-{100/reported[key]['lo']:.1f}")

# ---- Figures -----------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def style(ax):
    ax.set_facecolor(BG)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(INK2)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)


# Figure 1: risk differences, reported vs recomputed 95% CI, one linear axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
style(ax)
keys = list(reported)
for i, k in enumerate(keys):
    y = len(keys) - 1 - i
    r, c = reported[k], res[k]
    yr, yc = y + 0.17, y - 0.17
    ax.errorbar(r["rd"], yr, xerr=[[r["rd"] - r["lo"]], [r["hi"] - r["rd"]]], fmt="D", ms=8,
                color=ORANGE, mec=ORANGE, elinewidth=2, capsize=4, capthick=2, zorder=3)
    ax.errorbar(100 * c["rd"], yc, xerr=[[100 * (c["rd"] - c["wald"][0])], [100 * (c["wald"][1] - c["rd"])]],
                fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4, capthick=2, zorder=3)
    ax.text(14.6, yr, f"reported {r['lo']:.2f} to {r['hi']:.2f}", va="center", ha="left",
            fontsize=10, color=INK)
    ax.text(14.6, yc, f"recomputed {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}", va="center",
            ha="left", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.axvline(-5, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(0.15, 2.62, "no difference", fontsize=10, color=INK2, va="bottom")
ax.text(-4.85, 2.62, "noninferiority\nmargin, −5", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels([reported[k]["label"] for k in reversed(keys)], fontsize=10.5, color=INK)
ax.set_xlim(-6, 14.3)
ax.set_ylim(-0.6, 3.15)
ax.set_xlabel("Difference in first-pass success, percentage points (95% CI)")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
                 label="Reported (Abstract and Results)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
                 label="Recomputed from Table 2 counts (unpooled Wald)")
ax.set_xticks([-5, 0, 5, 10])
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
           fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(left=0.14, right=0.71, top=0.97, bottom=0.27)
save(fig, "fig1")

# Figure 2: NNT vs DL, reported vs recomputed 95% CI, log axis
fig, ax = plt.subplots(figsize=(7.2, 2.7), facecolor=BG)
style(ax)
ax.set_xscale("log")
arms = ["VLM", "VLH"]
for i, arm in enumerate(arms):
    y = len(arms) - 1 - i
    r, c = nnt_rep[arm], nnt[arm]
    yr, yc = y + 0.17, y - 0.17
    ax.errorbar(r[0], yr, xerr=[[r[0] - r[1]], [r[2] - r[0]]], fmt="D", ms=8, color=ORANGE,
                elinewidth=2, capsize=4, capthick=2, zorder=3)
    ax.errorbar(c[0], yc, xerr=[[c[0] - c[1]], [c[2] - c[0]]], fmt="o", ms=8, color=BLUE,
                elinewidth=2, capsize=4, capthick=2, zorder=3)
    ax.text(195, yr, f"reported {r[0]} ({r[1]} to {r[2]})", va="center", fontsize=10)
    ax.text(195, yc, f"recomputed {c[0]:.1f} ({c[1]:.1f} to {c[2]:.1f})", va="center", fontsize=10)
ax.set_xlim(5, 180)
ax.set_xticks([5, 10, 20, 50, 100])
ax.set_xticklabels(["5", "10", "20", "50", "100"])
ax.set_yticks(range(len(arms)))
ax.set_yticklabels([f"{a} vs DL" for a in reversed(arms)], fontsize=10.5, color=INK)
ax.set_ylim(-0.6, 1.6)
ax.set_xlabel("Number needed to treat (95% CI), log scale")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
                 label="Reported (Discussion)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
                 label="Recomputed: 1 / Wald RD limits")
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
           fontsize=10)
fig.subplots_adjust(left=0.14, right=0.63, top=0.96, bottom=0.38)
save(fig, "fig2")
print("\nFigures written to", HERE)

Reported HbA1c intervention effect of 0.02% lies outside its own 95% confidence interval of 0.03%–0.14%

JAMA Netw Open (2023) · doi:10.1001/jamanetworkopen.2022.50375 · PMC9856961

Summary

This secondary analysis of a cluster randomized trial (Texas Sprouts, 16 elementary schools) reports that, compared with control schools, the intervention produced a "0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)". The 0.02 figure appears in the Key Points, and the full estimate with its CI and P value appears in the Abstract, the Results text and Table 3. In each of those three places the estimate 0.02 lies below the lower limit of its own 95% CI. The four reported quantities (estimate 0.02, SE 0.01, CI 0.03 to 0.14, P = .005) cannot all be correct. The CI and the P value agree with each other and point to an effect of about 0.085 percentage points. The estimate and SE agree with neither, and the unadjusted difference in group changes in Table 3 (−0.03 vs +0.06, so 0.09) agrees with the CI. HbA1c is the only outcome behind the paper's conclusion that the intervention "improved glucose control". The Limitations say the improvement "may not be clinically relevant", and then add that "even small reductions in HbA1c levels reflect clinical improvements in glucose control". The improvement the paper reports, 0.02, is about one quarter of the size the paper's other numbers imply.

What the paper reports

Key Points, "Findings":

In this secondary analysis of a cluster randomized clinical trial of 695 children, a gardening, nutrition, and cooking intervention resulted in reductions of 0.02% for hemoglobin A1c and 6.40 mg/dL for low-density lipoprotein cholesterol levels.

Abstract, "Results":

Compared with control schools, children from Texas Sprouts schools had a 0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)

Results, paragraph on intervention effects:

Compared with children in the control schools, children in the Texas Sprouts intervention had a 0.02% reduction in mean HbA1c (95% CI, 0.03%-0.14%; P = .005)

Table 3, HbA1c row. Columns: intervention children, preintervention mean (SE), change mean (SE) | control children, preintervention mean (SE), change mean (SE) | differences in changes, mean (SE) [95% CI] | P value, intervention effect. (The minus sign is a Unicode minus in the publisher's XML.)

Hemoglobin A1c, %c | 236 | 5.27 (0.01) | -0.03 (0.02) | 221 | 5.22 (0.01) | 0.06 (0.02) | 0.02 (0.01) [0.03 to 0.14] | .005

Table 3, footnotes b and c:

Generalized linear mixed models with the identity link for continuous variables were used to test differences in metabolic parameters between the intervention and the control estimates, with schools as clusters fixed for their intervention effect and children nested within schools.

Children at 5 intervention schools and 5 control schools were analyzed for hemoglobin A1c.

Methods, "Statistical Analysis":

Data were analyzed using SPSS statistical software version 28 (IBM) and a 2-sided type I error level of α = .05 was used as the threshold for statistical significance.

The conclusions that rest on this result. Abstract, "Conclusions and Relevance":

In this cluster RCT, Texas Sprouts improved glucose control and reduced low-density lipoprotein cholesterol in high-risk youth.

Discussion, first paragraph:

To our knowledge, Texas Sprouts was the first cluster RCT to show that a school-based gardening, nutrition, and cooking intervention can improve glucose control and lower LDL cholesterol in elementary schoolchildren.

Discussion, paragraph on body composition:

The current intervention resulted in reductions in LDL cholesterol and improvements in HbA1c, independent of changes in body composition.

Discussion, "Limitations":

Another limitation is that the improvement in HbA1c was rather small, albeit significant, and may not be clinically relevant. However, health care professionals believe that even small reductions in HbA1c levels reflect clinical improvements in glucose control.

Conclusions:

In conclusion, the present RCT showed that a school-based gardening, nutrition, and cooking intervention resulted in a small, albeit significant, improvement in glycemic control and a reduction in LDL cholesterol in predominately low-income and racially diverse elementary schoolchildren.

What the paper's own numbers show

Method. The paper states an identity-link linear mixed model, so each 95% CI should be symmetric about its estimate, with half-width equal to a fixed multiplier times the SE. To find that multiplier I divided the CI half-width by the SE in the other rows of Table 3. In the seven rows without other problems it is 2.138 to 2.206 (median 2.148, not 1.96), which matches a t reference with about 14 degrees of freedom. I therefore used t(14) for the HbA1c checks, and give normal-reference values as well. For each pair of reported quantities (estimate, SE, CI, P) I worked out what one implies for the other. As an independent check I took the unadjusted difference between the two group changes in Table 3, with an SE from the two group SEs. That SE ignores clustering, so it is only a rough check. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Estimate inside its own 95% CI? Abstract, Results, Table 3 0.02 with CI 0.03 to 0.14 No: 0.02 is 0.01 below the lower limit
Centre of the reported CI Table 3 (estimate) 0.02 (0.03 + 0.14) / 2 = 0.085
SE implied by the CI width Table 3 0.01 0.055 / 2.145 = 0.026 (t14); 0.028 (normal)
CI implied by estimate 0.02 and SE 0.01 Table 3 0.03 to 0.14 −0.001 to 0.041 (t14); 0.000 to 0.040 (normal)
P implied by estimate/SE (0.02/0.01 = 2.0) Abstract, Results, Table 3 .005 .065 (t14); .046 (normal)
P implied by the reported CI Abstract, Results, Table 3 .005 .0051 (t14); .0025 (normal); .004 to .007 for df 8 to 20
Estimate needed for P = .005 if SE were 0.01 Table 3 0.02 0.033 (t14); 0.028 (normal)
Unadjusted difference in changes, intervention − control Table 3 change columns: −0.03 (0.02) vs 0.06 (0.02) 0.02 −0.09, i.e. a reduction of 0.09; SE 0.028, CI −0.145 to −0.035 (ignores clustering)
Sign of the difference column Table 3, rows with clear effects (e.g. LDL changes −8.43 vs −3.53, difference −6.40) HbA1c CI 0.03 to 0.14 (positive) In those rows the column reads intervention − control, so a reduction should be negative: −0.14 to −0.03, which matches −0.145 to −0.035
Reported effect vs CI centre Key Points, Abstract 0.02 The CI centre is 4.25 times larger; the unadjusted difference is 4.5 times larger

Three things follow. First, the estimate cannot belong to the reported interval, so at least one of them is wrong. Second, the CI and the P value fit together: under the multiplier the rest of Table 3 implies, the CI gives P = .0051, against the reported .005. The estimate 0.02 and SE 0.01 fit neither of them, and they are also much smaller than the unadjusted difference. Third, the CI's sign does not follow the convention the table uses in its rows with clear effects. In those rows (fasting glucose, total, LDL and non-HDL cholesterol) the column reads intervention − control, so an HbA1c reduction would be negative. With the sign restored, the HbA1c CI matches the unadjusted difference closely (−0.14 to −0.03 against −0.145 to −0.035). The most coherent reading of the paper's own numbers is an HbA1c reduction of about 0.085 to 0.09 percentage points, with the estimate and SE cells in error. The exact model estimate cannot be rebuilt from these rounded summaries, because elsewhere in Table 3 the model estimates differ from the unadjusted differences by up to about 2 SEs (for example, non-HDL cholesterol −6.13 against −3.83, about 2.2 SE). In fasting insulin and HDL the model estimate and the unadjusted difference differ in sign (+0.61 against −0.42; +0.43 against −0.31), which fits adjustment gaps of that size. So the sign convention is inferred from the rows with clear effects, not shown for every row.

Related observations in Table 3. These do not change the HbA1c finding, but they bear on how the table can be read:

Figure

Dot-and-interval plot on one axis of the HbA1c effect: the reported estimate 0.02 sits outside its reported CI 0.03 to 0.14; recomputed intervals from the reported estimate and SE, from the reported CI, and from the unadjusted group changes

Figure 1. HbA1c effect in Table 3, on one axis in percentage points, with positive values meaning lower HbA1c in intervention schools than in control schools, as the paper's text reads "0.02% reduction (95% CI, 0.03%-0.14%)". The top row (orange) is what the paper reports: the dot (0.02) sits outside the bar (0.03 to 0.14). The blue rows are recomputed from the paper's own numbers. The second row is the interval that the reported estimate and SE would give (−0.001 to 0.041, P = .065), which does not match the reported CI or P. The third row is the centre and SE implied by the reported CI (0.085, SE 0.026, P = .0051), which matches the reported P. The fourth row is the unadjusted difference in the two group changes in Table 3 (−0.03 vs +0.06), shown as a reduction of 0.09 with CI 0.035 to 0.145. Its SE ignores clustering. The vertical line marks no effect. Intervals use the t(14) multiplier that the other rows of Table 3 imply.

Position of each reported estimate within its own reported 95% CI, for every row of Table 3

Figure 2. Where each reported estimate sits within its own reported 95% CI, for all nine outcomes in Table 3, scaled so that the lower limit is 0, the centre 0.5 and the upper limit 1. Seven rows sit at the centre, as an identity-link model requires. Triglycerides sits off-centre (its lower limit, −15.42, is not symmetric about 1.52). HbA1c is the only estimate outside its interval.

Why it matters

Explanations considered

Status

Suggested correction

  1. Report the HbA1c intervention effect from the fitted model with an estimate, SE, 95% CI and P value that agree with one another. Correct the Key Points, Abstract, Results text and Table 3 together. Use the sign convention of the Table 3 rows with clear effects (intervention − control, so a reduction is negative) in the table, and state the magnitude in the text.
  2. If, as the CI and P value suggest, the effect is about 0.08 to 0.09 percentage points, revisit the Limitations and Conclusions statements that the improvement was small and "may not be clinically relevant", which accompany the reported 0.02.
  3. State which test the "P value, intervention effect" column reports. For fasting glucose, HOMA-IR, total, LDL, HDL and non-HDL cholesterol the printed P values do not follow from the displayed differences and CIs (for example, fasting glucose has a CI of 1.60 to 6.84 but P = .13). If the P values are wrong, the statements that there were "no intervention effects" on glucose and other lipids need to be revisited too.
  4. Check the triglycerides interval (−15.42 to 6.47), which is not symmetric about its estimate 1.52, and remove the minus signs printed on SEs, as in "−6.40 (−1.20)".
Recomputation script (python recompute.py.txt)
"""Recompute the HbA1c intervention effect reported in PMC9856961
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2022.50375), Table 3.

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
  fig1.svg/png, fig2.svg/png               (the figures used in writeup.md)
  candidates/cand_A..E.svg/png             (five alternative designs for the main figure)

All inputs are copied from Table 3 of the article as archived in PMC (identical in the
publisher's version of record). Columns: intervention change, control change,
"Differences in changes, mean (SE) [95% CI]", "P value, intervention effect".
In the rows with clear effects (fasting glucose, total, LDL and non-HDL cholesterol) the
table's difference column reads intervention minus control (e.g. LDL: changes -8.43 vs
-3.53, difference -6.40), so a reduction in HbA1c would be negative there. In fasting
insulin and HDL the model estimate and the unadjusted difference differ in sign (section
4b), so the convention is inferred from the rows with clear effects.
"""
import math
import os

import numpy as np
from scipy import stats
from scipy.optimize import brentq

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (Table 3)
# name: (intervention change, control change, estimate, SE, CI low, CI high, reported P)
# SEs printed with a minus sign in the table (e.g. "-6.40 (-1.20)") are entered as positive.
ROWS = {
    "Fasting glucose":   (6.53, 2.34, 4.22, 1.22, 1.60, 6.84, 0.13),
    "Fasting insulin":   (0.84, 1.26, 0.61, 0.80, -1.16, 2.37, 0.46),
    "HOMA-IR":           (0.68, 0.57, 0.39, 0.18, -0.001, 0.77, 0.84),
    "Total cholesterol": (-7.35, -3.23, -5.72, 1.24, -8.39, -3.04, 0.12),
    "LDL cholesterol":   (-8.43, -3.53, -6.40, 1.20, -8.97, -3.82, 0.048),
    "Triglycerides":     (5.92, 1.42, 1.52, 2.54, -15.42, 6.47, 0.39),
    "HDL cholesterol":   (0.28, 0.59, 0.43, 0.34, -0.30, 1.17, 0.74),
    "Non-HDL chol.":     (-7.62, -3.79, -6.13, 1.05, -8.38, -3.89, 0.08),
    "Hemoglobin A1c":    (-0.03, 0.06, 0.02, 0.01, 0.03, 0.14, 0.005),
}
GROUP_SE_A1C = (0.02, 0.02)      # SEs of the intervention and control HbA1c changes
N_A1C = (236, 221)               # children with HbA1c, intervention / control
Z = stats.norm.ppf(0.975)

I_CH, C_CH, EST, SE, LO, HI, P_REP = ROWS["Hemoglobin A1c"]

# ------------------------------------------------------------------ colours / style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "axes.linewidth": 0.8,
})


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg", bbox_inches="tight")
    fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


def fp(p):
    """P value in the paper's style: no leading zero (.065, .0051)."""
    d = 4 if p < 0.01 else 3
    return f"{p:.{d}f}".lstrip("0")


def p2(z, df=None):
    z = abs(z)
    return 2 * (stats.t.sf(z, df) if df else stats.norm.sf(z))


def hdr(t):
    print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)


# ================================================================== 1. HbA1c row
hdr("1. HbA1c row as reported (Table 3; same numbers in Abstract and Results)")
print(f"estimate {EST}  SE {SE}  95% CI [{LO}, {HI}]  P {P_REP}")
print(f"Is the estimate inside its own CI?  {LO <= EST <= HI}   "
      f"(estimate is {LO - EST:.2f} below the lower limit)")
mid = (LO + HI) / 2
half = (HI - LO) / 2
print(f"CI midpoint = {mid:.3f}; CI half-width = {half:.3f}")

# ================================================================== 2. multiplier used elsewhere in Table 3
hdr("2. CI half-width / SE in the other rows of Table 3 (what multiplier the table uses)")
mults = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
    m = (hi - lo) / 2 / s
    sym = abs((hi - e) - (e - lo)) / ((hi - lo) / 2)
    mults[k] = m
    print(f"{k:18s} half-width/SE = {m:5.3f}   midpoint {(lo + hi) / 2:8.3f}  vs estimate {e:7.3f}")
reg = [m for k, m in mults.items() if k not in ("Triglycerides", "Hemoglobin A1c")]
m_med = float(np.median(reg))
print(f"\nThe 7 regular rows: multiplier {min(reg):.3f} to {max(reg):.3f}, median {m_med:.3f} "
      f"(normal 1.960)")
df_med = brentq(lambda d: stats.t.ppf(0.975, d) - m_med, 2, 1000)
df_lo = brentq(lambda d: stats.t.ppf(0.975, d) - max(reg), 2, 1000)
df_hi = brentq(lambda d: stats.t.ppf(0.975, d) - min(reg), 2, 1000)
print(f"t quantile equal to the median multiplier: df = {df_med:.1f} "
      f"(range of rows: df {df_lo:.1f} to {df_hi:.1f})")
DF = 14
TQ = stats.t.ppf(0.975, DF)
print(f"Used below: t reference with df = {DF}, t(0.975) = {TQ:.3f}")

# ================================================================== 3. what the reported numbers imply
hdr("3. HbA1c: reconstructing each reported quantity from the others")
ci_from_est_t = (EST - TQ * SE, EST + TQ * SE)
ci_from_est_z = (EST - Z * SE, EST + Z * SE)
print(f"CI from estimate 0.02 and SE 0.01, t({DF}):  [{ci_from_est_t[0]:.4f}, {ci_from_est_t[1]:.4f}]")
print(f"CI from estimate 0.02 and SE 0.01, normal: [{ci_from_est_z[0]:.4f}, {ci_from_est_z[1]:.4f}]")
p_est_t = p2(EST / SE, DF)
p_est_z = p2(EST / SE)
print(f"P from 0.02/0.01 (ratio {EST / SE:.2f}): t({DF}) P = {p_est_t:.3f}; normal P = {p_est_z:.4f}"
      f"   (reported {P_REP})")
se_ci_t = half / TQ
se_ci_z = half / Z
print(f"SE implied by the CI width: t({DF}) {se_ci_t:.4f}; normal {se_ci_z:.4f}   (reported {SE})")
p_ci_t = p2(mid / se_ci_t, DF)
p_ci_z = p2(mid / se_ci_z)
print(f"P implied by the CI (midpoint {mid:.3f}): t({DF}) ratio {mid / se_ci_t:.2f}, P = {p_ci_t:.4f}; "
      f"normal ratio {mid / se_ci_z:.2f}, P = {p_ci_z:.4f}   (reported {P_REP})")
for d in (8, 10, 12, 20):
    q = stats.t.ppf(0.975, d)
    print(f"   sensitivity, df={d:2d}: P implied by the CI = {p2(mid / (half / q), d):.4f}")
z_rep = stats.norm.isf(P_REP / 2)
t_rep = stats.t.isf(P_REP / 2, DF)
# just under / just over 0.025, so the result does not depend on the rounding rule
print(f"Rounding check: estimate 0.0249 and lower limit 0.0251 (round to 0.02 / 0.03) would put "
      f"the interval {0.0251 - 0.0249:.4f} below and {HI - 0.0249:.4f} above the estimate")
print(f"Estimate needed for P = .005 with SE 0.01: {t_rep * SE:.3f} (t{DF}) / {z_rep * SE:.3f} (normal)")

# ================================================================== 4. raw difference in group changes
hdr("4. Unadjusted difference in mean changes (Table 3 change columns)")
raw = I_CH - C_CH
se_raw = math.hypot(*GROUP_SE_A1C)
ci_raw = (raw - Z * se_raw, raw + Z * se_raw)
print(f"intervention change {I_CH:+.2f}, control change {C_CH:+.2f}: "
      f"difference (I - C) = {raw:+.2f}")
print(f"SE from the two group SEs (ignores clustering) = {se_raw:.4f}; "
      f"95% CI [{ci_raw[0]:.3f}, {ci_raw[1]:.3f}]; normal P = {p2(raw / se_raw):.4f}")
print(f"Reported CI with the table's sign convention restored: [{-HI:.2f}, {-LO:.2f}]")
print(f"Ratio of CI midpoint to the reported 0.02: {mid / EST:.2f}; of |raw difference|: "
      f"{abs(raw) / EST:.2f}")

hdr("4b. Model estimate vs unadjusted difference in the other rows (how far they can differ)")
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
    print(f"{k:18s} unadjusted I-C {i - c:+7.2f}   model estimate {e:+7.2f}   "
          f"gap {e - (i - c):+6.2f}  ({(e - (i - c)) / s:+5.2f} SE)")

# ================================================================== 5. sign convention
hdr("5. Sign convention of the difference column")
print("Rows with clear effects (CI excludes zero), then the two rows where the model estimate and"
      " the unadjusted difference differ in sign, then HbA1c:")
for k in ("Fasting glucose", "Total cholesterol", "LDL cholesterol", "Non-HDL chol.",
          "Fasting insulin", "HDL cholesterol", "Hemoglobin A1c"):
    i, c, e, s, lo, hi, p = ROWS[k]
    print(f"{k:18s} I {i:+.2f}  C {c:+.2f}  I-C {i - c:+.2f}  table estimate {e:+.2f}  CI [{lo}, {hi}]")

# ================================================================== 6. P column across Table 3
hdr("6. Reported P vs P implied by each row's own estimate/SE and CI")
print(f"{'row':18s} {'est in CI':>9s} {'P est/SE t14':>12s} {'P CI t14':>10s} {'P est/SE z':>10s} "
      f"{'P CI z':>9s} {'reported':>8s}")
P_ROWS = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
    m = (lo + hi) / 2
    sct, scz = (hi - lo) / 2 / TQ, (hi - lo) / 2 / Z
    P_ROWS[k] = (p2(e / s, DF), p2(m / sct, DF), p)
    print(f"{k:18s} {str(lo <= e <= hi):>9s} {p2(e / s, DF):12.4g} {p2(m / sct, DF):10.4g} "
          f"{p2(e / s):10.4g} {p2(m / scz):9.4g} {p:8.3g}")
i, c, e, s, lo, hi, p = ROWS["Triglycerides"]
print(f"\nTriglycerides: lower limit that would be symmetric about {e} given the upper limit "
      f"{hi}: {2 * e - hi:.2f} (printed {lo}); estimate +/- {TQ:.3f} x SE: "
      f"[{e - TQ * s:.2f}, {e + TQ * s:.2f}]")

# ================================================================== figures
POS = dict(lw=2)


def axis_style(ax, xlabel):
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)
    for sp in ("top", "right", "left"):
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel(xlabel, color=INK)


# Reduction scale used in the HbA1c figures: positive = lower HbA1c in intervention
# schools relative to control, which is how the paper's text reads "0.02% reduction
# (95% CI, 0.03%-0.14%)". The unadjusted I-C difference is sign-flipped onto it.
RED_LABEL = ("Reduction in HbA1c vs control, percentage points\n"
             "(positive = lower in intervention schools, as the paper's text reads it)")

forest_rows = [
    ("Reported in the paper\n(Abstract, Results, Table 3)", EST, LO, HI, ORANGE, "o",
     f"0.02, CI 0.03 to 0.14, P = .005\nestimate lies outside its own interval"),
    ("Recomputed: interval from the\nreported estimate 0.02 and SE 0.01", EST,
     ci_from_est_t[0], ci_from_est_t[1], BLUE, "o",
     f"CI {ci_from_est_t[0]:.3f} to {ci_from_est_t[1]:.3f}, P = {fp(p_est_t)}"),
    ("Recomputed: centre and SE implied\nby the reported CI", mid, LO, HI, BLUE, "D",
     f"{mid:.3f} (SE {se_ci_t:.3f}), P = {fp(p_ci_t)}"),
    ("Recomputed: unadjusted difference\nin group changes, -0.03 vs +0.06", -raw,
     -ci_raw[1], -ci_raw[0], BLUE, "s",
     f"{-raw:.2f}, CI {-ci_raw[1]:.3f} to {-ci_raw[0]:.3f}\n(ignores clustering)"),
]


def draw_forest(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    n = len(forest_rows)
    for j, (lab, e, lo, hi, col, mk, note) in enumerate(forest_rows):
        y = n - 1 - j
        ax.plot([lo, hi], [y, y], color=col, solid_capstyle="butt", zorder=3, **POS)
        for xx in (lo, hi):
            ax.plot([xx, xx], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
        ax.plot(e, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
        ax.text(0.205, y, note, va="center", ha="left", fontsize=9.5, color=INK2)
    # highlight the gap on the reported row
    y0 = n - 1
    ax.text(EST, y0 + 0.30, "0.02 < lower limit 0.03", ha="left", va="bottom",
            fontsize=9.5, color=INK)
    ax.axvline(0, color=INK2, lw=0.8, zorder=1)
    ax.text(0.001, -0.75, "no effect", fontsize=9, color=INK2, ha="left")
    ax.set_yticks(range(n))
    ax.set_yticklabels([r[0] for r in forest_rows][::-1], fontsize=10)
    for t, r in zip(ax.get_yticklabels(), forest_rows[::-1]):
        t.set_color(INK)
    ax.set_xlim(-0.02, 0.36)
    ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20])
    ax.set_ylim(-0.9, n - 0.35)
    axis_style(ax, RED_LABEL)
    ax.xaxis.set_label_coords(0.29, -0.13)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=ORANGE, marker="o", lw=2, ms=8, label="Reported by the paper"),
         Line2D([], [], color=BLUE, marker="o", lw=2, ms=8,
                label="Recomputed from the paper's own numbers")]
    fig.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 0.985), ncol=2,
               frameon=False, fontsize=9.5)
    fig.text(0.0, 1.04, f"HbA1c effect, Table 3 (n = {N_A1C[0]} vs {N_A1C[1]} children, "
             f"10 schools); 95% intervals, t({DF})", fontsize=10.5, color=INK, ha="left")
    save(fig, path)


draw_forest(os.path.join(HERE, "fig1"))
draw_forest(os.path.join(CAND, "cand_A_forest"))


# ---- Candidate B: one number line, reported row only, annotated
def draw_numberline(path):
    fig, ax = plt.subplots(figsize=(7.2, 2.3))
    ax.plot([LO, HI], [0, 0], color=ORANGE, lw=2, solid_capstyle="butt")
    for xx in (LO, HI):
        ax.plot([xx, xx], [-0.12, 0.12], color=ORANGE, lw=2)
    ax.plot(EST, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5)
    ax.plot(mid, 0, "D", color=BLUE, ms=9, mec=BG, mew=1.5)
    ax.text(EST, -0.32, "reported estimate\n0.02", ha="center", va="top", color=INK)
    ax.text(LO, 0.25, "reported lower\nlimit 0.03", ha="center", va="bottom", color=INK)
    ax.text(HI, 0.25, "reported upper\nlimit 0.14", ha="center", va="bottom", color=INK)
    ax.text(mid, -0.32, f"recomputed centre\nof that CI, {mid:.3f}", ha="center", va="top",
            color=INK)
    ax.axvline(0, color=INK2, lw=0.8)
    ax.text(0.002, -0.9, "no effect", color=INK2, fontsize=9)
    ax.set_ylim(-1.0, 0.9)
    ax.set_xlim(-0.01, 0.16)
    ax.set_yticks([])
    axis_style(ax, "Reduction in HbA1c vs control, percentage points")
    ax.set_title("A 95% CI must contain its estimate: 0.02 lies outside [0.03, 0.14]\n"
                 "orange = reported by the paper, blue = recomputed", loc="left",
                 fontsize=10.5)
    save(fig, path)


draw_numberline(os.path.join(CAND, "cand_B_numberline"))


# ---- Candidate C: calculation-flow diagram
def box(ax, x, y, w, h, text, edge, fs=9.8, bold=False):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.04",
                                fc=BG, ec=edge, lw=2))
    ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, color=INK,
            fontweight="bold" if bold else "normal")


def arrow(ax, x0, y0, x1, y1, label=None, off=(0, 0.06)):
    ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
                arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
    if label:
        ax.text((x0 + x1) / 2 + off[0], (y0 + y1) / 2 + off[1], label, ha="center",
                fontsize=9, color=INK2)


def draw_flow(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ax.set_xlim(-0.1, 10.1)
    ax.set_ylim(0.3, 5.3)
    ax.axis("off")
    W, H, fs = 3.0, 1.45, 9.6
    box(ax, 0.0, 3.4, W, H, "Table 3, change\nintervention \u22120.03\ncontrol +0.06", ORANGE, fs=fs)
    box(ax, 3.5, 3.4, W, H, "Unadjusted difference\n\u22120.03 \u2212 0.06 = \u22120.09\n"
        "(reduction of 0.09)", BLUE, fs=fs)
    box(ax, 7.0, 3.4, W, H, f"SE {se_raw:.3f}\n(from the 2 group SEs)\nCI {-ci_raw[1]:.3f} to "
        f"{-ci_raw[0]:.3f}\n(as a reduction)", BLUE, fs=fs)
    box(ax, 0.0, 0.5, W, H, "Table 3, difference\n0.02 (SE 0.01)\nCI 0.03 to 0.14", ORANGE, fs=fs)
    box(ax, 3.5, 0.5, W, H, f"Centre of that CI\n(0.03 + 0.14) / 2\n= {mid:.3f}", BLUE, fs=fs)
    box(ax, 7.0, 0.5, W, H, f"SE implied by CI {se_ci_t:.3f}\nP implied by CI {fp(p_ci_t)}\n"
        f"(reported P .005)", BLUE, fs=fs)
    ym_top, ym_bot = 3.4 + H / 2, 0.5 + H / 2
    for yy in (ym_top, ym_bot):
        arrow(ax, 3.05, yy, 3.45, yy)
        arrow(ax, 6.55, yy, 6.95, yy)
    for xx in (1.5, 5.0):
        ax.annotate("", xy=(xx, 0.5 + H + 0.08), xytext=(xx, 3.4 - 0.08),
                    arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.2, ls="--"))
    yc = (0.5 + H + 3.4) / 2
    ax.text(5.1, yc, f"agree: about 0.09 / {mid:.3f}", fontsize=9.5, color=INK, va="center")
    ax.text(1.6, yc, "estimate 0.02 is outside\nits own CI [0.03, 0.14]", fontsize=9.5,
            color=INK, va="center")
    ax.text(0.0, 5.12, "HbA1c, Table 3. Orange boxes: reported by the paper.  Blue boxes: "
            "recomputed from those numbers.", fontsize=9.5, color=INK2)
    save(fig, path)


draw_flow(os.path.join(CAND, "cand_C_flow"))


# ---- Candidate D: consistency graph of the four reported quantities
def draw_consistency(path):
    fig, ax = plt.subplots(figsize=(7.2, 4.2))
    ax.set_xlim(0, 10.4)
    ax.set_ylim(0.1, 5.85)
    ax.axis("off")
    W, H = 2.2, 1.1
    nodes = {"est": (0.1, 4.6, "Estimate\n0.02"), "se": (0.1, 0.3, "SE\n0.01"),
             "ci": (8.1, 4.6, "95% CI\n0.03 to 0.14"), "p": (8.1, 0.3, "P value\n.005")}
    for k, (x, y, t) in nodes.items():
        box(ax, x, y, W, H, t, ORANGE, fs=10.5, bold=True)

    def c(k):
        x, y, _ = nodes[k]
        return x + W / 2, y + H / 2

    def edge(a, b, txt, ok, t=0.5, above=False):
        (x0, y0), (x1, y1) = c(a), c(b)
        ax.plot([x0, x1], [y0, y1], color=BLUE if ok else INK2, lw=2,
                ls="-" if ok else (0, (4, 3)), zorder=0)
        # a label placed above a horizontal link leaves the dashes visible
        dy, va = (0.22, "bottom") if above else (0, "center")
        ax.text(x0 + t * (x1 - x0), y0 + t * (y1 - y0) + dy, txt, ha="center", va=va,
                fontsize=9.3, color=INK, zorder=5,
                bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=BLUE if ok else INK2, lw=1))

    edge("est", "ci", f"INCONSISTENT: estimate lies outside the CI\n"
         f"(CI centre is {mid:.3f})", False, above=True)
    edge("est", "p", f"INCONSISTENT:\n0.02 / 0.01 gives\nP = {fp(p_est_t)}, not .005", False,
         t=0.3)
    edge("se", "ci", f"INCONSISTENT:\nCI width implies\nSE {se_ci_t:.3f}, not 0.01", False,
         t=0.7)
    edge("ci", "p", f"consistent:\nCI implies\nP = {fp(p_ci_t)}", True)
    edge("est", "se", f"INCONSISTENT:\ntogether give\nCI {ci_from_est_t[0]:.3f}\n"
         f"to {ci_from_est_t[1]:.3f},\nnot 0.03 to 0.14", False)
    fig.text(0.0, 1.0, "HbA1c, Table 3. Orange boxes: values reported by the paper. Each link is "
             "a check recomputed\nfrom them (t reference, df = 14, as the other rows of "
             "Table 3 imply).\nSolid blue link = consistent; dashed gray link = inconsistent.",
             fontsize=9.5, color=INK2, va="bottom")
    save(fig, path)


draw_consistency(os.path.join(CAND, "cand_D_consistency"))


# ---- Candidate E (also fig2): position of each estimate within its own CI, all of Table 3
def draw_position(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    names = list(ROWS)[::-1]
    ax.axvspan(0, 1, color="#efeeea", zorder=0, lw=0)
    for x in (0, 0.5, 1):
        ax.axvline(x, color=INK2, lw=0.8 if x != 0.5 else 0.6, ls="-" if x != 0.5 else ":",
                   zorder=1)
    for y, k in enumerate(names):
        i, c, e, s, lo, hi, p = ROWS[k]
        pos = (e - lo) / (hi - lo)
        out = not (lo <= e <= hi)
        col = ORANGE
        ax.plot(pos, y, "o", color=col, ms=9, mec=BG, mew=1.5, zorder=3)
        if out or k == "Triglycerides":
            ax.text(pos + (-0.04 if out else 0.04), y, f"{e} in [{lo}, {hi}]",
                    ha="right" if out else "left", va="center", fontsize=9.3, color=INK)
    ax.set_yticks(range(len(names)))
    ax.set_yticklabels(names, color=INK)
    ax.set_xlim(-0.75, 1.35)
    ax.set_xticks([0, 0.5, 1])
    ax.set_xticklabels(["lower\nlimit", "centre", "upper\nlimit"])
    axis_style(ax, "Where the reported estimate sits within its own reported 95% CI")
    ax.grid(False)
    ax.set_title("Table 3, all rows (orange = positions computed from reported values)",
                 loc="left", fontsize=10.5)
    save(fig, path)


draw_position(os.path.join(HERE, "fig2"))
draw_position(os.path.join(CAND, "cand_E_table_overview"))

hdr("Figures written")
for f in sorted(os.listdir(HERE)) + ["candidates/" + g for g in sorted(os.listdir(CAND))]:
    if f.endswith((".png", ".svg")):
        print("  ", f)

Diabetes-only hyperglycemia primary outcome has denominators larger than the trial's possible diabetic population

JAMA Network Open (2023) · doi:10.1001/jamanetworkopen.2023.32517 · PMC10517374

Summary

This pilot randomized trial (ACTFAST-3) defines one of its two primary outcomes, avoidance of postoperative hyperglycemia, for patients with diabetes only, and states that "patients without diabetes were excluded from analysis". Yet the reported denominators (8676 intervention, 8815 control) exceed the most cases with diabetes that the paper's own Table 1 allows (5713 and 5887, even counting every case of unknown status as diabetic), and they equal exactly the analyzed cases with any glucose value in the Figure 1 flowchart. The published result (85.8% vs 85.8%; RR 1.00, 95% CI 0.99–1.01) therefore describes a population in which at least a third of cases, and probably about two-thirds, did not have diabetes. Its narrow confidence interval, which the Discussion describes as "tight confidence bounds", depends on that larger denominator, and simple calculations suggest an interval for the stated population would be roughly 2 to 3.5 times as wide.

What the paper reports

Methods, "Study Population":

For glycemic outcomes, patients without diabetes were excluded from analysis.

Methods, "Outcomes":

The primary outcomes were avoidance of hypothermia, defined as the proportion of patients who had a final recorded intraoperative core temperature greater than 36 °C, and avoidance of hyperglycemia, defined as the proportion of patients with diabetes who had a blood glucose level of 180 mg/dL or lower on arrival to the postanesthesia recovery area.

Abstract, "Main Outcomes and Measures":

(defined as the proportion of patients with diabetes who had a blood glucose level ≤180 mg/dL on arrival to the postanesthesia recovery area)

Abstract, "Results". The Results text (primary outcomes paragraph) repeats the same counts, percentages and RR:

There was no treatment effect on avoidance of hyperglycemia (7445 of 8676 patients [85.8%] in the intervention group vs 7559 of 8815 [85.8%] in the control group; rate ratio [RR], 1.00; 95% CI, 0.99-1.01)

Table 2, primary outcome row (columns: intervention, n = 12 980 | control, n = 13 274 | coefficient (95% CI) | P value):

No postoperative hyperglycemia | 7445/8676 (85.8) | 7559/8815 (85.8) | 1.00 (0.99 to 1.01) | >.99

Table 1, comorbidity row. Table 1 lists the control column first (control, n = 13 274 | intervention, n = 12 980):

Diabetes | 2909/10 296 (28.3) | 2824/10 091 (28.0)

Figure 1, the study flowchart. The flowchart is an image, so this text was read from the publisher's image file (jamanetwopen-e2332517-g001.jpg) rather than from a text conversion:

12980 Analyzed (1581 without temperature data, 4304 without glucose data)

13274 Analyzed (1652 without temperature data, 4459 without glucose data)

Figure 1 caption:

The analysis included all patients randomized during the study with outcomes present.

The findings and conclusions that rest on this outcome. Key Points, "Findings":

an intraoperative telemedicine decision support intervention did not significantly reduce postoperative hypothermia or hyperglycemia

Abstract, "Conclusions and Relevance":

anesthesia care quality measures did not differ between groups, with high confidence in the findings

Discussion, second paragraph:

There are several potential reasons for the lack of an intention-to-treat effect with tight confidence bounds on these quality of care outcomes.

Discussion, third paragraph:

Second, the overall adherence to each measure at baseline was high, making statistically significant improvements difficult to achieve.

What the paper's own numbers show

Method. Table 1, Table 2 and Figure 1 count cases in the same unit and with the same arm totals (12 980 and 13 274). For each arm I took the cases with diabetes from Table 1. I counted cases with unknown diabetes status as the arm total minus the Table 1 denominator. The sum of the two is an upper bound on the number of cases that could have diabetes. I compared that bound with the Table 2 hyperglycemia denominator, and compared the denominator with Figure 1's analyzed cases minus cases "without glucose data". I repeated the check for the two sensitivity analyses in Supplement 2 (eTables 4–7). Those tables print diabetes only as n (%), so there the bound uses the smallest known-status total consistent with the rounded percentage. For contrast I looked at the secondary outcome that is explicitly limited to patients with diabetes. Finally I checked the Table 2 arithmetic with crude, unclustered rate ratios. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Cases with diabetes, intervention Table 1 2824/10 091 (28.0) 28.0%; 12 980 − 10 091 = 2889 with unknown status
Cases with diabetes, control Table 1 2909/10 296 (28.3) 28.3%; 13 274 − 10 296 = 2978 with unknown status
Most cases that could have diabetes, intervention Table 1 — 2824 + 2889 = 5713
Most cases that could have diabetes, control Table 1 — 2909 + 2978 = 5887
Hyperglycemia denominator, intervention Table 2; Abstract; Results 8676 2963 more than the 5713 maximum; 3.07 × the documented diabetic cases
Hyperglycemia denominator, control Table 2; Abstract; Results 8815 2928 more than the 5887 maximum; 3.03 × the documented diabetic cases
Analyzed cases with glucose data, intervention Figure 1 12 980 analyzed, 4304 without glucose data 12 980 − 4304 = 8676, identical to the denominator
Analyzed cases with glucose data, control Figure 1 13 274 analyzed, 4459 without glucose data 13 274 − 4459 = 8815, identical to the denominator
Denominator as a share of all analyzed cases Table 2 vs Figure 1 — 66.8% and 66.4%, against a diabetes prevalence of 28.0% and 28.3%
Share of the denominator that cannot have diabetes derived — at least 34.2% (intervention) and 33.2% (control); 67.5% and 67.0% if only documented diabetes is counted
Sensitivity analysis, excluding repeat surgeries within 30 days eTables 4–5 denominators 8069 / 8169; diabetes 2614 (27.6%) / 2667 (27.6%) of 12 179 / 12 457 most possible diabetic cases ≤ 5339 / 5478; denominators exceed this by 2730 / 2691
Sensitivity analysis, ignoring prior assignment eTables 6–7 denominators 8693 / 8798; diabetes 2844 (28.2%) / 2889 (28.0%) of 12 965 / 13 289 most possible diabetic cases ≤ 5741 / 5878; denominators exceed this by 2952 / 2920
Diabetes-restricted secondary outcome (intraoperative glucose measurement), for contrast Table 2 denominators 1962 / 1996 69.5% and 68.6% of the documented diabetic cases, so this outcome does fit inside the diabetic population
No postoperative hyperglycemia, % and RR Table 2 85.8 vs 85.8; RR 1.00 (0.99 to 1.01) 85.81% vs 85.75%; crude RR 1.001 (unclustered 95% CI 0.989–1.013), consistent with the printed values for the printed denominators

The Table 2 arithmetic is internally consistent. The problem is the population. The denominators equal every analyzed case with a glucose value, not the cases with diabetes. The same pattern appears in all three published versions of the analysis (the main analysis and both sensitivity analyses; Figure 2 below), so it is not a single transcription slip.

The comparison also turned up some smaller, separate inconsistencies. None of them changes a conclusion:

Quantity Location Reported Recomputed
Arm sizes Abstract vs Methods, Table 1, Table 2, Figure 1 Abstract: "control group; n = 12 980" and "intervention group; n = 13 274" Every other location gives intervention 12 980 and control 13 274, so the abstract's labels are swapped
Hypothermia denominators Table 2 vs Figure 1 11 447 / 11 672 12 980 − 1581 = 11 399 and 13 274 − 1652 = 11 622, which differ by 48 and 50
Intraoperative glucose measurement, intervention % Table 2 1346/1962 (68.7) 68.60%, which rounds to 68.6
Control hypothermia % Abstract "[66.7.%]" typographical (66.7%)

Figure

Hyperglycemia denominators compared with the number of cases that could have diabetes, by arm

Figure 1. Counts for each arm on one axis of surgical cases, drawn from zero. The orange bar is the reported denominator of the hyperglycemia primary outcome (Table 2), which the paper defines for patients with diabetes only. The blue bars come from the paper's other numbers: the cases with documented diabetes (Table 1); the most cases that could have diabetes if every case with unknown status had it (Table 1); and the analyzed cases with any glucose value (Figure 1: analyzed minus "without glucose data"). The gray bar is all analyzed cases. In both arms the reported denominator exceeds the largest possible diabetic population, by 2963 and 2928 cases, and equals the number of cases with glucose data exactly.

The same excess in the main and both sensitivity analyses

Figure 2. The same comparison for the main analysis (Tables 1–2) and the two sensitivity analyses in Supplement 2 (eTables 4–5, excluding surgeries within 30 days of an index operation; eTables 6–7, ignoring prior treatment assignment). Hollow blue circles are cases with documented diabetes as printed. Blue diamonds are the most cases that could have diabetes; for the sensitivity analyses this is an upper bound derived from the rounded percentage. Orange circles are the reported hyperglycemia denominators. The number on each row is the reported denominator minus that maximum. In all six arm-analyses the denominator lies about 2700–3000 cases above the largest possible diabetic population.

Why it matters

Avoidance of hyperglycemia is one of the trial's two primary outcomes. Its null result is reported in the Key Points, the Abstract, the Results, Table 2 and eTables 5 and 7. The paper's stated population for it is patients with diabetes, and the published protocol that the paper cites for its sample size (reference 22, F1000Research 2018;7:623) planned it that way: its power calculation states that "we assumed that the prevalence of diabetes mellitus among Barnes-Jewish Hospital surgical patients is about 20%", and its sample-size table plans glucose clusters about a quarter the size of the temperature clusters (59 vs 239 cases, a ratio of 0.247). The reported denominators are instead about two-thirds of all analyzed cases (0.668 and 0.664).

Three consequences follow from the published numbers alone.

  1. The estimate does not describe the stated population. At least 34% of each denominator cannot have diabetes, and about 67% did not if only documented diabetes is counted. Glucose above 180 mg/dL on arrival in recovery is presumably much less common in patients without diabetes. Including them raises the avoidance rate (85.8%) and, if their rates are similar between arms, pulls any between-arm ratio toward 1. The diabetes-only rates and RR cannot be recovered from the published tables.
  2. The "tight confidence bounds" depend on the larger denominator. With the reported denominators, the crude unclustered 95% CI for the RR is 0.989–1.013, which reproduces the published 0.99–1.01. Two illustrations show the scale of the change; neither is an estimate of the true result. (a) With denominators the size of the documented diabetic population (2824 and 2909) and no difference between arms, the same calculation gives 0.979–1.021 at an 85.8% avoidance rate and 0.959–1.043 at 60%, about 1.7 to 3.5 times as wide. (b) If the cases without diabetes had the same avoidance rate q in both arms and every documented diabetic case were in the denominator, the implied diabetes-only avoidance would be 77.1% vs 77.1% at q = 0.90 and 58.5% vs 58.9% at q = 0.99. The RR would stay between 0.994 and 1.000, with unclustered 95% CIs from 0.972–1.029 to 0.951–1.038, 2.3 to 3.6 times as wide. All of these come before the clustering and Bonferroni-corrected α that the paper applies to its CIs, which widen them further.
  3. The explanation offered for the null result is affected. The Discussion attributes the null results partly to high baseline adherence. For this outcome, the 85.8% baseline describes the mixed population and has not been shown for patients with diabetes; in the scenario above it would be closer to 59–77%.

The trial's other primary outcome, hypothermia, does not depend on this issue. Most secondary outcomes are null and, apart from those limited to diabetes, are not affected either. The scenario in point 2 suggests the null result for hyperglycemia may well survive a correct analysis, but with a substantially wider interval. As published, the hyperglycemia result cannot be read as a finding about patients with diabetes, and the "high confidence" statement in the Abstract rests partly on it.

Explanations considered

Status

Suggested correction

  1. Re-run the hyperglycemia primary outcome restricted to cases with diabetes, as the Methods state, and report the numerators, denominators, RR, Bonferroni-adjusted CI and Holm-adjusted P value. Update the Abstract, Key Points, Results text, Table 2, eTables 5 and 7, and Figure 2 if it uses the same population. Also report how many cases with diabetes had no glucose value.
  2. Alternatively, if the intended population was all cases with a postoperative glucose value, as in the registry wording, revise the outcome definition in the Abstract and Methods. Remove or qualify the sentence "For glycemic outcomes, patients without diabetes were excluded from analysis", report the change from the protocol's diabetic framing, and give the diabetes-only result as a secondary analysis.
  3. In either case, revisit the "high confidence" and "tight confidence bounds" statements as they apply to hyperglycemia.
  4. Minor: correct the swapped arm sizes in the Abstract; explain why the hypothermia denominators (11 447 / 11 672) differ from Figure 1 (11 399 / 11 622); change 68.7 to 68.6 for 1346/1962 if it is a crude proportion; and fix "[66.7.%]".
Recomputation script (python recompute.py.txt)
"""Recompute the hyperglycemia primary-outcome denominators of PMC10517374
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2023.32517).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC and from Supplement 2:
  Figure 1 (flowchart image): analyzed cases and cases without temperature / glucose data
  Table 1: cases with diabetes (No./total No. with known status)
  Table 2: primary and secondary outcome counts
  eTables 4-7 (Supplement 2): the two sensitivity analyses
Unit: the paper counts surgical cases (Table 1 and Table 2 use the same n per arm).
"""
import math
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
ARMS = ("intervention", "control")
N = {"intervention": 12980, "control": 13274}                 # Figure 1, Table 1, Table 2
NO_GLUCOSE = {"intervention": 4304, "control": 4459}          # Figure 1 "without glucose data"
NO_TEMP = {"intervention": 1581, "control": 1652}             # Figure 1 "without temperature data"
DIABETES = {"intervention": (2824, 10091), "control": (2909, 10296)}   # Table 1 "Diabetes"
HYPERGLY = {"intervention": (7445, 8676), "control": (7559, 8815)}     # Table 2 primary
HYPOTHERM = {"intervention": (7602, 11447), "control": (7783, 11672)}  # Table 2 primary
GLU_MEAS = {"intervention": (1346, 1962), "control": (1274, 1996)}     # Table 2 secondary
GLU_MEAS_PRINTED_PCT = {"intervention": 68.7, "control": 63.8}

# Sensitivity analyses (Supplement 2). Diabetes is printed as n (%) without a denominator.
SENS = {
    "Excluding repeat surgeries (eTables 4-5)": {
        "N": {"intervention": 12179, "control": 12457},
        "dm": {"intervention": (2614, 27.6), "control": (2667, 27.6)},
        "hyper": {"intervention": (6923, 8069), "control": (6989, 8169)},
    },
    "Ignoring prior assignment (eTables 6-7)": {
        "N": {"intervention": 12965, "control": 13289},
        "dm": {"intervention": (2844, 28.2), "control": (2889, 28.0)},
        "hyper": {"intervention": (7469, 8693), "control": (7535, 8798)},
    },
}

# Published protocol (ref 22, F1000Res 2018;7:623), sample-size table: cluster sizes
PROTO_CLUSTER = {"temperature": 239, "glucose": 59}
PROTO_DM_PREV = 0.20


def pct(a, b):
    return 100.0 * a / b


def rr_ci(a1, n1, a0, n0, z=1.959964):
    rr = (a1 / n1) / (a0 / n0)
    se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
    return rr, math.exp(math.log(rr) - z * se), math.exp(math.log(rr) + z * se)


R = {}  # results reused by the figures

print("=== 1. Hyperglycemia denominator vs the population it is defined for ===")
for g in ARMS:
    d, known = DIABETES[g]
    unknown = N[g] - known
    max_dm = d + unknown
    den = HYPERGLY[g][1]
    with_glu = N[g] - NO_GLUCOSE[g]
    R[g] = dict(d=d, unknown=unknown, max_dm=max_dm, den=den, with_glu=with_glu)
    print(f"\n[{g}]  n = {N[g]}")
    print(f"  Table 1 diabetes: {d}/{known} = {pct(d, known):.1f}%; status unknown: {N[g]} - {known} = {unknown}")
    print(f"  most cases that could have diabetes (all unknown counted as diabetes): {d} + {unknown} = {max_dm}")
    print(f"  Table 2 hyperglycemia denominator: {den}")
    print(f"  denominator - maximum possible diabetic cases = {den - max_dm}")
    print(f"  denominator / documented diabetic cases = {den / d:.2f}")
    print(f"  Figure 1: {N[g]} - {NO_GLUCOSE[g]} without glucose data = {with_glu}  -> equals denominator: {with_glu == den}")
    print(f"  denominator as share of all analyzed cases: {pct(den, N[g]):.1f}%")
    print(f"  share of denominator that cannot have diabetes (lower bound): ({den} - {max_dm}) / {den} = {pct(den - max_dm, den):.1f}%")
    print(f"  share of denominator without documented diabetes if all {d} diabetic cases had glucose data: "
          f"({den} - {d}) / {den} = {pct(den - d, den):.1f}%")

print("\n=== 2. Same check in the two sensitivity analyses (Supplement 2) ===")
SENS_R = {}
for name, v in SENS.items():
    for g in ARMS:
        d, p = v["dm"][g]
        n = v["N"][g]
        den = v["hyper"][g][1]
        # Diabetes % is rounded to 0.1; the smallest known-status total consistent with it
        known_min = math.ceil(d / ((p + 0.05) / 100))
        max_dm = d + (n - known_min)
        SENS_R[(name, g)] = dict(d=d, max_dm=max_dm, den=den)
        print(f"{name} [{g}]: n={n}; diabetes {d} ({p}%) -> known-status total >= {known_min}; "
              f"max possible diabetic cases <= {max_dm}; hyperglycemia denominator {den}; "
              f"denominator - documented diabetes = {den - d}; denominator - max possible = {den - max_dm}")

print("\n=== 3. A diabetes-restricted glucose outcome for comparison (Table 2, secondary) ===")
for g in ARMS:
    a, n = GLU_MEAS[g]
    d = DIABETES[g][0]
    print(f"{g}: intraoperative glucose measurement denominator {n} = {pct(n, d):.1f}% of {d} diabetic cases "
          f"(fits inside the diabetic population: {n <= d})")

print("\n=== 4. Published protocol sample size (ref 22) ===")
print(f"glucose cluster size / temperature cluster size = {PROTO_CLUSTER['glucose']}/{PROTO_CLUSTER['temperature']} "
      f"= {PROTO_CLUSTER['glucose'] / PROTO_CLUSTER['temperature']:.3f} (protocol assumed diabetes prevalence {PROTO_DM_PREV:.0%})")
print(f"reported denominators / analyzed cases: {HYPERGLY['intervention'][1] / N['intervention']:.3f}, "
      f"{HYPERGLY['control'][1] / N['control']:.3f}")

print("\n=== 5. Table 2 arithmetic (crude, unclustered; the paper's CIs are cluster-robust and Bonferroni-adjusted) ===")
for name, dct in (("No postoperative hyperglycemia", HYPERGLY), ("No postoperative hypothermia", HYPOTHERM),
                  ("Intraoperative glucose measurement", GLU_MEAS)):
    a1, n1 = dct["intervention"]
    a0, n0 = dct["control"]
    rr, lo, hi = rr_ci(a1, n1, a0, n0)
    print(f"{name}: {pct(a1, n1):.2f}% vs {pct(a0, n0):.2f}%; crude RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f})")
for g in ARMS:
    a, n = GLU_MEAS[g]
    print(f"glucose measurement {g}: {a}/{n} = {pct(a, n):.3f}% -> one decimal {pct(a, n):.1f} (printed {GLU_MEAS_PRINTED_PCT[g]})")

print("\n=== 6. Precision at diabetes-sized denominators (illustration only) ===")
a1, n1 = HYPERGLY["intervention"]
a0, n0 = HYPERGLY["control"]
rr, lo, hi = rr_ci(a1, n1, a0, n0)
print(f"reported denominators {n1}/{n0}: unclustered 95% CI half-width on log scale {(math.log(hi) - math.log(lo)) / 2:.4f}")
for p in (0.858, 0.70, 0.60):
    m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
    se = math.sqrt((1 - p) / (p * m1) + (1 - p) / (p * m0))
    print(f"  {m1}/{m0} cases at avoidance rate {p:.3f} and RR 1: unclustered 95% CI {math.exp(-1.96 * se):.3f}-{math.exp(1.96 * se):.3f}")

print("\n=== 6b. Illustrative diabetes-only scenario (assumption, not an estimate) ===")
print("Assume every documented diabetic case (Table 1) is in the denominator and the cases without")
print("diabetes avoid hyperglycemia at the same rate q in both arms; back out the diabetic-only rates.")
_rr0, _lo0, _hi0 = rr_ci(*HYPERGLY["intervention"], *HYPERGLY["control"])
REF_W = _hi0 - _lo0
for q in (0.90, 0.95, 0.97, 0.99):
    a = {}
    for g in ARMS:
        num, den = HYPERGLY[g]
        d = DIABETES[g][0]
        a[g] = num - q * (den - d)
    m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
    rr, lo, hi = rr_ci(a["intervention"], m1, a["control"], m0)
    print(f"  q = {q:.2f}: diabetic avoidance {pct(a['intervention'], m1):.1f}% vs {pct(a['control'], m0):.1f}%; "
          f"RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f}); CI width {hi - lo:.3f}, "
          f"{(hi - lo) / REF_W:.1f} x the unclustered width at the reported denominators")
print(f"  reference: unclustered CI width at reported denominators = {REF_W:.3f} ({_lo0:.3f}-{_hi0:.3f})")

print("\n=== 7. Other, smaller inconsistencies ===")
for g in ARMS:
    exp_t = N[g] - NO_TEMP[g]
    print(f"hypothermia {g}: Figure 1 gives {N[g]} - {NO_TEMP[g]} = {exp_t}; Table 2 denominator {HYPOTHERM[g][1]}; "
          f"difference {HYPOTHERM[g][1] - exp_t}")
print("Abstract: 'control group; n = 12 980' and 'intervention group; n = 13 274'; Methods, Table 1, Table 2 and "
      "Figure 1 give intervention 12 980 and control 13 274 (labels swapped in the abstract).")
print("Abstract prints '[66.7.%]' for the control hypothermia percentage.")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, CONTEXT = "#eb6834", "#2a78d6", "#e4e3df", "#d6d5d0"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


def fmt(n):
    return f"{n:,}"


# Figure 1: counts per arm, horizontal bars on one count axis
rows = []  # (label, value, color, group)
for g in ARMS:
    r = R[g]
    rows += [
        (f"All analyzed cases (Figure 1)", N[g], CONTEXT, g),
        (f"Cases with glucose data: {fmt(N[g])} − {fmt(NO_GLUCOSE[g])} (Figure 1)", r["with_glu"], BLUE, g),
        ("Hyperglycemia denominator (Table 2)", r["den"], ORANGE, g),
        (f"Most cases that could have diabetes:\n{fmt(r['d'])} + {fmt(r['unknown'])} status unknown (Table 1)", r["max_dm"], BLUE, g),
        ("Cases with documented diabetes (Table 1)", r["d"], BLUE, g),
    ]

fig, ax = plt.subplots(figsize=(7.2, 7.0))
BAR_H = 0.52
y, ys = 0.0, []
heads = {}
for i, (lab, val, col, g) in enumerate(rows):
    if i % 5 == 0:
        heads[g] = y
        y += 0.8
    ys.append(y)
    y += 1.0
    if i % 5 == 4:
        y += 0.5
ys = [-v for v in ys]
for (lab, val, col, g), yy in zip(rows, ys):
    ax.barh(yy, val, height=BAR_H, color=col, edgecolor="none", zorder=2)
    ax.text(val + 150, yy, fmt(val), va="center", ha="left", fontsize=10, color=TXT, zorder=3)
for g, yy in heads.items():
    ax.text(-200, -yy, f"{g.capitalize()} arm (n = {fmt(N[g])})", fontsize=11, fontweight="bold",
            ha="right", va="center", color=TXT)
for i, g in enumerate(ARMS):
    r = R[g]
    y_den, y_max = ys[5 * i + 2], ys[5 * i + 3]
    y_arrow = (y_den + y_max) / 2
    ax.plot([r["max_dm"], r["max_dm"]], [y_max - BAR_H / 2, y_den + BAR_H / 2], color=TXT, lw=1.2,
            ls=(0, (3, 2)), zorder=4)
    ax.annotate("", xy=(r["den"], y_arrow), xytext=(r["max_dm"], y_arrow),
                arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.4, shrinkA=0, shrinkB=0), zorder=5)
    ax.text(r["den"] + 150, y_arrow, f"{fmt(r['den'] - r['max_dm'])} more than\nthe maximum",
            ha="left", va="center", fontsize=10, color=TXT, zorder=5, linespacing=1.0)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 15800)
ax.set_xticks(range(0, 12001, 4000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.patches import Patch  # noqa: E402
fig.legend(handles=[Patch(color=ORANGE, label="Reported denominator of the diabetes-only primary outcome (Table 2)"),
                    Patch(color=BLUE, label="Implied by the paper's other numbers (Table 1, Figure 1)"),
                    Patch(color=CONTEXT, label="All analyzed cases (context)")],
           loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.11, 1, 1))
save(fig, "fig1")

# Figure 2: the same gap in all three published analyses
items = [("Main analysis (Tables 1-2)", g, R[g]["d"], R[g]["max_dm"], R[g]["den"]) for g in ARMS]
for name in SENS:
    for g in ARMS:
        s = SENS_R[(name, g)]
        items.append((name, g, s["d"], s["max_dm"], s["den"]))
fig, ax = plt.subplots(figsize=(7.2, 5.0))
yy = []
y = 0
last = None
for name, g, d, mx, den in items:
    if last is not None and name != last:
        y += 0.6
    yy.append(-y)
    last = name
    y += 1
for (name, g, d, mx, den), yv in zip(items, yy):
    ax.plot([d, den], [yv, yv], color=CONTEXT, lw=2, zorder=1)
    ax.plot([d], [yv], "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
    ax.plot([mx], [yv], "D", ms=8, color=BLUE, zorder=3)
    ax.plot([den], [yv], "o", ms=9, color=ORANGE, zorder=3)
    ax.text(den + 200, yv, f"+{fmt(den - mx)}", va="center", fontsize=10, color=TXT)
ax.set_yticks(yy)
ax.set_yticklabels([f"{n.split(' (')[0]}, {g}" for n, g, *_ in items], fontsize=10)
ax.set_xlim(0, 10600)
ax.set_xticks(range(0, 10001, 2000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.lines import Line2D  # noqa: E402
fig.legend(handles=[
    Line2D([], [], marker="o", ls="none", ms=9, mfc=BG, mec=BLUE, mew=2, label="Cases with documented diabetes (Table 1, eTables 4, 6)"),
    Line2D([], [], marker="D", ls="none", ms=8, color=BLUE, label="Most cases that could have diabetes (recomputed)"),
    Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE, label="Reported hyperglycemia denominator (Table 2, eTables 5, 7)"),
    Line2D([], [], ls="none", label="+n: reported denominator minus the maximum possible"),
], loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.2, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Subgroup effects called significantly different although their 95% CIs overlap, contrary to the paper's criterion

JAMA Network Open (2024) · doi:10.1001/jamanetworkopen.2023.56619 · PMC10891466

Summary

The paper asks whether food insecurity modifies the association between SNAP receipt and nonadherence to antihypertensive medication. Its Methods say two subgroup estimates count as significantly different only if their 95% CIs do not overlap, and its Results then call the food-insecure estimate (−13.6 percentage points [pp]; 95% CI −25.0 to −2.3) and the food-secure estimate (5.0 pp; 95% CI −15.8 to 25.7) "statistically significantly different". Those intervals overlap from −15.8 to −2.3, and a Wald test on the same intervals gives a difference of 18.6 pp (95% CI −5.1 to 42.3; p = 0.12). The paper answers yes to its own effect-modification question, but neither its stated criterion nor a conventional test supports that answer. The food-insecure estimate on its own is not affected.

What the paper reports

The research question (Abstract, Key Points, Question):

"Is receipt of Supplemental Nutrition Assistance Program (SNAP) benefits associated with nonadherence to antihypertensive medications and, if so, does this association vary by food insecurity status?"

The criterion for effect modification (Methods, Statistical Analysis, second paragraph):

"The potential effect modification of food insecurity was tested by evaluating for statistical differences in the magnitude and direction of the PATEs for the food-secure and food-insecure subgroups; PATEs were considered statistically significantly different if their respective 95% CIs did not overlap."

The two subgroup estimates (Results, "Association Between SNAP Receipt and Nonadherence to Antihypertensive Medications"):

"Similarly, the rate of nonadherence to antihypertensive medication was similar (385 [56.7% weighted] vs 2671 [55.2% weighted]; PATE, 5.0 [95% CI, -15.8 to 25.7]) between SNAP recipients and nonrecipients among the 5354 patients who did not report experiencing food insecurity."

"In contrast, among the subgroup of 1338 antihypertensive medication users who experienced food insecurity, the rate of nonadherence to antihypertensive medications was 8.17 percentage points lower among SNAP recipients compared with nonrecipients (302 [56.0% weighted] vs 491 [60.0% weighted]; PATE, -13.6 [95% CI, -25.0 to -2.3])."

The conclusion drawn from them (Results, same paragraph, the sentence that follows):

"The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association between SNAP and nonadherence to antihypertensive medication therapy."

Table 2 gives the same values: food insecure, 302 (56.0) vs 491 (60.0), PATE −13.6 (−25.0 to −2.3); food secure, 385 (56.7) vs 2671 (55.2), PATE 5.0 (−15.8 to 25.7); overall, 687 (56.4) vs 3162 (55.8), PATE 1.7 (−21.2 to 24.6).

The finding is carried into the Discussion (first paragraph):

"Among a US sample of patients with hypertension identified from a linked, nationally representative dataset, we observed that SNAP had a differential association with nonadherence to antihypertensive medications between food-secure and food-insecure subgroups."

and into the Strengths (Discussion, Strengths and Limitations):

"Second, this study presented data suggesting that the potential consequences of SNAP receipt on adherence to antihypertensive medications are greatest among the food-insecure population. Thus, we have identified the food-insecure hypertension population as a target for food assistance intervention to address adherence to antihypertensive medications, especially when food assistance resources are limited."

(Quotes are from the PubMed Central archive, which prints minus signs as hyphens; the publisher's page prints the same values with true minus signs.)

What the paper's own numbers show

Method. All inputs come from Table 2 and the Results text. (1) The paper's criterion is applied as written: do the two reported 95% CIs overlap? (2) Each standard error (SE) is recovered from its CI as (upper − lower)/(2 × 1.96). Both CIs are symmetric about their point estimates, so this is valid. (3) The two subgroups are disjoint sets of participants, and the Methods describe a stratified analysis with subgroup-specific propensity scores. The difference is therefore tested with the usual Wald test for two independent estimates, z = (5.0 − (−13.6)) / √(SE₁² + SE₂²). (4) To check whether a different confidence level would rescue the claim, the confidence level at which the two intervals just stop overlapping is also computed. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed
Food-insecure PATE (95% CI) Table 2; Results; Abstract −13.6 (−25.0 to −2.3) Symmetric (half-widths 11.4 / 11.3); SE 5.79 pp; single-group p = 0.019
Food-secure PATE (95% CI) Table 2; Results 5.0 (−15.8 to 25.7) Symmetric (half-widths 20.8 / 20.7); SE 10.59 pp
Do the two 95% CIs overlap? (paper's criterion) Methods vs Results "statistically significantly different" Yes, they overlap, from −15.8 to −2.3 (13.5 pp). The criterion is not met
Food-insecure point estimate inside the food-secure 95% CI? Table 2 — Yes (−13.6 lies within −15.8 to 25.7)
Difference, food secure − food insecure derived from Table 2 "statistically significantly different" 18.6 pp, SE 12.07, 95% CI −5.1 to 42.3 (includes 0)
Wald test of the difference derived from Table 2 no test statistic given z = 1.54; two-sided p = 0.123 (one-sided p = 0.062)
Confidence level at which the two CIs stop overlapping derived — Only below 74.4%. The 90%, 85.1%, 83.4%, 80% and 75% intervals all overlap
Confidence level at which non-overlap would match p < 0.05 for these SEs derived — 85.1%. The intervals overlap at that level too
Correlation between the subgroup estimates needed for p = 0.05 derived — 0.45
Nonadherent counts Table 2; Results 687; 3162; 3849 302 + 385 = 687; 491 + 2671 = 3162; 687 + 3162 = 3849 (consistent)
Subgroup sizes Results 5354 food secure; 1338 food insecure 6692 − 1338 = 5354; 540 + 798 = 1338 (consistent)
Food-insecure gap described in the text Results "8.17 percentage points lower" Weighted crude gap 56.0 − 60.0 = −4.0 pp; unweighted 55.9% − 61.5% = −5.6 pp; adjusted PATE −13.6 pp. 8.17 matches none of them (separate, minor)
Model behind the PATEs Methods and Abstract vs Table 2 footnote b Methods and Abstract: probit; footnote b: "from a logit model" Internal inconsistency in the description (minor; does not change the CIs)

The counts and sample sizes are mutually consistent, so the problem is not in the reported data. It lies only in the inference drawn from the two intervals.

Figure

Reported subgroup estimates with overlapping 95% CIs, and the recomputed difference between them

Figure 1. All values are on one percentage-point axis (SNAP recipients minus nonrecipients; negative means less nonadherence with SNAP). Orange: the two subgroup PATEs and 95% CIs as reported in Table 2. The shaded band marks where the two intervals overlap (−15.8 to −2.3, 13.5 pp); under the paper's stated rule, any overlap means the estimates are not significantly different. Blue: the difference between subgroups (food secure − food insecure), recomputed from the same table, 18.6 pp (95% CI −5.1 to 42.3; z = 1.54; p = 0.12). The thin vertical line labelled "null (0)" marks no effect for the subgroup rows and no difference for the bottom row; the interval for the difference includes it.

Upper bound of the food-insecure CI and lower bound of the food-secure CI as functions of the confidence level

Figure 2. Would a different confidence level change the conclusion? Blue curves: the food-insecure upper bound (solid) and the food-secure lower bound (dashed), recomputed from the SEs implied by Table 2 for confidence levels from 50% to 99%. Orange points: the reported 95% bounds (−2.3 and −15.8). The intervals overlap (shaded) at every level above 74.4%. That includes 85.1%, the level at which non-overlap of these two particular intervals would correspond to a two-sided Wald p < 0.05 (dotted line).

Why it matters

Effect modification by food insecurity is the paper's stated question. It is the second half of the Key Points question, it is the Objective ("To evaluate whether receipt of SNAP benefits can modify the consequences of food insecurity on nonadherence to antihypertensive medications."), and it is the purpose of the prespecified stratified analysis. The claim that the association differs by food-insecurity status underpins:

On the paper's own numbers, the data are compatible with no difference between the subgroups: the difference has a 95% CI of −5.1 to 42.3 pp. A result that is significant in one subgroup and not in the other is not evidence that the subgroups differ, and the paper's non-overlap rule, conservative as it is, was the guard it set against that inference. What survives is the food-insecure estimate on its own (−13.6 pp, CI excluding 0, p ≈ 0.02 before any adjustment for multiple subgroups). The literal wording "associated ... among the food-insecure subgroup but not among their food-secure counterparts" therefore remains accurate as a description of the two subgroup estimates. The statement that food insecurity modified the association, the "differential association", and the targeting recommendations built on it are not supported. The hedged overall conclusion ("may be less likely ... especially if") is weakened rather than reversed.

Explanations considered

One further observation does not change the conclusion. The overall-sample interval (1.7, −21.2 to 24.6; half-width 22.9) is about twice as wide as the food-insecure interval (half-width 11.4), although it covers five times as many participants. Without data or code this cannot be examined further; the analysis above uses the subgroup intervals exactly as published.

Status

As of 2026-10-23:

Suggested correction

  1. Replace "The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association …" with a statement that matches the stated criterion, for example: "The 95% CIs for the food-secure and food-insecure subgroups overlapped; by our prespecified criterion, the PATEs were therefore not statistically significantly different, and we did not find statistical evidence that food insecurity modified the association."
  2. Preferably also report a direct estimate of the difference between subgroups (an interaction model, or a test of the difference between the two PATEs), with its CI and p value. From the published intervals this is approximately 18.6 pp (95% CI −5.1 to 42.3; p ≈ 0.12).
  3. Revise the Key Points (Findings, Meaning), the Abstract Results and Conclusions, the opening of the Discussion, the Strengths paragraph and the Conclusions so that they report the food-insecure estimate without presenting a difference between subgroups as established.
  4. Correct "8.17 percentage points lower" to agree with Table 2 (adjusted PATE −13.6; weighted crude difference −4.0), and make footnote b of Table 2 ("from a logit model") agree with the probit model described in the Methods and the Abstract, or vice versa.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC10891466 write-up and regenerate its figures.

Paper: "Supplemental Nutrition Assistance Program and Adherence to
Antihypertensive Medications", JAMA Network Open 2024,
doi:10.1001/jamanetworkopen.2023.56619 (PMC10891466).

All inputs are copied from Table 2 and the Results text of the paper.
Run: python recompute.py.txt   (needs numpy, scipy, matplotlib)
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
# Table 2: PATE, percentage-point difference (95% CI)
ROWS = {
    "overall":       (1.7, -21.2, 24.6),
    "food_insecure": (-13.6, -25.0, -2.3),
    "food_secure":   (5.0, -15.8, 25.7),
}
# Table 2: nonadherent, No. (weighted %), SNAP vs no SNAP
COUNTS = {
    "overall":       ((687, 56.4), (3162, 55.8)),
    "food_insecure": ((302, 56.0), (491, 60.0)),
    "food_secure":   ((385, 56.7), (2671, 55.2)),
}
N_TOTAL, N_FI, N_FS = 6692, 1338, 5354          # Results text
N_FI_SNAP, N_FI_NOSNAP = 540, 798                # Results text
NONADH_TOTAL = 3849                              # Results text
TEXT_FI_GAP = 8.17                               # Results text, "8.17 percentage points lower"

Z975 = norm.ppf(0.975)


def section(title):
    print("\n" + "=" * 72 + "\n" + title + "\n" + "=" * 72)


# ------------------------------------------- 1. SEs implied by the CIs
section("1. Standard errors implied by the reported 95% CIs")
SE = {}
for k, (est, lo, hi) in ROWS.items():
    SE[k] = (hi - lo) / (2 * Z975)
    print(f"{k:14s} PATE {est:6.1f}  CI [{lo:6.1f}, {hi:5.1f}]  "
          f"lower half-width {est - lo:5.1f}  upper half-width {hi - est:5.1f}  "
          f"midpoint {(lo + hi) / 2:6.2f}  SE {SE[k]:6.2f}  "
          f"single-group two-sided p {2 * norm.sf(abs(est) / SE[k]):.3f}")
print("-> Each CI is symmetric about its estimate (to rounding), so no minus sign is lost.")

fi, fs = ROWS["food_insecure"], ROWS["food_secure"]

# ------------------------------------------- 2. The paper's own criterion
section("2. The paper's own criterion: do the two 95% CIs overlap?")
ov_lo, ov_hi = max(fi[1], fs[1]), min(fi[2], fs[2])
print(f"Overlap interval: [{ov_lo}, {ov_hi}]  overlap = {ov_lo < ov_hi}  width = {ov_hi - ov_lo:.1f} pp")
print(f"Food-insecure estimate {fi[0]} inside food-secure CI [{fs[1]}, {fs[2]}]: {fs[1] <= fi[0] <= fs[2]}")
print(f"Food-secure estimate {fs[0]} inside food-insecure CI [{fi[1]}, {fi[2]}]: {fi[1] <= fs[0] <= fi[2]}")
print(f"Food-secure CI covers 0: {fs[1] <= 0 <= fs[2]};  food-insecure CI covers 0: {fi[1] <= 0 <= fi[2]}")

# ------------------------------------------- 3. Wald test of the difference
section("3. Wald test of the difference between the subgroup PATEs (independent strata)")
d = fs[0] - fi[0]
se_d = np.hypot(SE["food_insecure"], SE["food_secure"])
z = d / se_d
p2 = 2 * norm.sf(abs(z))
p1 = norm.sf(abs(z))
d_lo, d_hi = d - Z975 * se_d, d + Z975 * se_d
print(f"Difference (food secure - food insecure) = {d:.1f} pp")
print(f"SE = sqrt({SE['food_insecure']:.2f}^2 + {SE['food_secure']:.2f}^2) = {se_d:.2f} pp")
print(f"z = {z:.2f}; two-sided p = {p2:.3f}; one-sided p = {p1:.3f}")
print(f"95% CI for the difference: {d_lo:.1f} to {d_hi:.1f} pp (includes 0: {d_lo <= 0 <= d_hi})")

# correlation between the two estimates that would be needed for p = 0.05
se_needed = d / Z975
rho = (SE["food_insecure"] ** 2 + SE["food_secure"] ** 2 - se_needed ** 2) / (
    2 * SE["food_insecure"] * SE["food_secure"])
print(f"For two-sided p = 0.05 the SE of the difference would have to be {se_needed:.2f}, "
      f"i.e. a correlation of {rho:.2f} between the two subgroup estimates "
      f"(they come from disjoint groups of participants, fitted separately).")

# ------------------------------------------- 4. Narrower CI levels
section("4. At which confidence level would the intervals stop overlapping?")
for lvl in (0.95, 0.90, 0.851, 0.834, 0.80, 0.75):
    zz = norm.ppf(0.5 + lvl / 2)
    fi_hi = fi[0] + zz * SE["food_insecure"]
    fs_lo = fs[0] - zz * SE["food_secure"]
    print(f"{lvl * 100:5.1f}% CIs: food-insecure upper {fi_hi:6.2f}, food-secure lower {fs_lo:6.2f} "
          f"-> overlap = {fs_lo < fi_hi}")
z_touch = d / (SE["food_insecure"] + SE["food_secure"])
lvl_touch = 2 * norm.cdf(z_touch) - 1
print(f"The intervals just touch at the {100 * lvl_touch:.1f}% level (z = {z_touch:.2f}).")
# level at which 'non-overlap' would coincide with a two-sided p = 0.05 Wald test, for these SEs
z_eq = Z975 * se_d / (SE["food_insecure"] + SE["food_secure"])
lvl_eq = 2 * norm.cdf(z_eq) - 1
print(f"For these two SEs, non-overlap of {100 * lvl_eq:.1f}% CIs is equivalent to two-sided p < 0.05; "
      f"the intervals still overlap at that level ({100 * lvl_touch:.1f}% < {100 * lvl_eq:.1f}%).")

# ------------------------------------------- 5. Other numbers in the paragraph
section("5. Other numbers in the same Results paragraph")
s_snap = COUNTS["food_insecure"][0][0] + COUNTS["food_secure"][0][0]
s_nos = COUNTS["food_insecure"][1][0] + COUNTS["food_secure"][1][0]
print(f"SNAP nonadherent 302 + 385 = {s_snap} (reported {COUNTS['overall'][0][0]})")
print(f"No-SNAP nonadherent 491 + 2671 = {s_nos} (reported {COUNTS['overall'][1][0]})")
print(f"Total nonadherent 687 + 3162 = {687 + 3162} (reported {NONADH_TOTAL})")
print(f"Food-secure n = 6692 - 1338 = {N_TOTAL - N_FI} (reported {N_FS})")
print(f"Food-insecure n = 540 + 798 = {N_FI_SNAP + N_FI_NOSNAP} (reported {N_FI})")
raw_fi = COUNTS["food_insecure"][0][1] - COUNTS["food_insecure"][1][1]
raw_fs = COUNTS["food_secure"][0][1] - COUNTS["food_secure"][1][1]
print(f"Food-insecure weighted raw difference 56.0 - 60.0 = {raw_fi:.1f} pp; "
      f"adjusted PATE {fi[0]} pp; text says {TEXT_FI_GAP} pp lower "
      f"(matches neither: {abs(raw_fi) != TEXT_FI_GAP and abs(fi[0]) != TEXT_FI_GAP})")
print(f"Food-secure weighted raw difference 56.7 - 55.2 = {raw_fs:.1f} pp; adjusted PATE {fs[0]} pp")
uw_snap = 100 * COUNTS["food_insecure"][0][0] / N_FI_SNAP
uw_nos = 100 * COUNTS["food_insecure"][1][0] / N_FI_NOSNAP
print(f"Food-insecure unweighted: 302/540 = {uw_snap:.1f}%, 491/798 = {uw_nos:.1f}%, "
      f"difference {uw_snap - uw_nos:.1f} pp (also not 8.17)")

# ------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans",
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG,
})


def fmt(v):
    return f"{v:.1f}".replace("-", "\u2212")


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
    plt.close(fig)


# Figure 1: the two reported subgroup CIs and the recomputed difference, one axis
fig, ax = plt.subplots(figsize=(7.2, 3.9))
rows = [
    ("Food insecure\n(Table 2, reported)", fi[0], fi[1], fi[2], ORANGE, "o"),
    ("Food secure\n(Table 2, reported)", fs[0], fs[1], fs[2], ORANGE, "o"),
    ("Difference, secure − insecure\n(recomputed from Table 2)", d, d_lo, d_hi, BLUE, "D"),
]
ys = [2.0, 1.0, -0.3]
ax.axvspan(ov_lo, ov_hi, ymin=0.38, ymax=0.97, color="#f1d9cf", alpha=0.6, lw=0, zorder=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for (lab, est, lo, hi, col, mk), y in zip(rows, ys):
    ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
    ax.plot([lo, lo], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
    ax.plot([hi, hi], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
    ax.plot(est, y, marker=mk, ms=9, color=col, zorder=4)
    ax.text(hi + 1.2, y, f"{fmt(est)} ({fmt(lo)} to {fmt(hi)})", va="center", fontsize=10, color=col,
            bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.text(4.0, 2.6,
        f"95% CIs overlap from {fmt(ov_lo)} to {fmt(ov_hi)} ({ov_hi - ov_lo:.1f} pp):\nthe paper's criterion for 'different' is not met",
        ha="center", va="center", fontsize=10, color=INK, bbox=dict(boxstyle="square,pad=0.2", fc=BG, ec="none"))
ax.text(d, -0.75, f"Wald z = {z:.2f}, two-sided p = {p2:.2f}; CI includes 0",
        ha="center", va="center", fontsize=10, color=BLUE, bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.axhline(0.35, color=GRID, lw=1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(-1.0, 2.9)
ax.set_xlim(-30, 64)
ax.set_xlabel("Percentage-point difference in nonadherence, SNAP vs no SNAP (PATE)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.6, 0.45, "null (0)", fontsize=10, color=INK2, va="bottom")
save(fig, "fig1")

# Figure 2: interval bounds as a function of confidence level
lv = np.linspace(0.50, 0.99, 400)
zz = norm.ppf(0.5 + lv / 2)
fi_up = fi[0] + zz * SE["food_insecure"]
fs_lo = fs[0] - zz * SE["food_secure"]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.plot(lv * 100, fi_up, color=BLUE, lw=2, label="Food insecure: upper CI bound (recomputed)")
ax.plot(lv * 100, fs_lo, color=BLUE, lw=2, ls="--", label="Food secure: lower CI bound (recomputed)")
ax.plot([95], [fi[2]], "o", ms=9, color=ORANGE, zorder=5)
ax.plot([95], [fs[1]], "o", ms=9, color=ORANGE, zorder=5, label="Reported 95% CI bounds (Table 2)")
ax.annotate(fmt(fi[2]), (95, fi[2]), xytext=(-34, 4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(fmt(fs[1]), (95, fs[1]), xytext=(-40, -4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.fill_between(lv * 100, fs_lo, fi_up, where=fs_lo < fi_up, color="#f1d9cf", alpha=0.6, lw=0)
ax.axvline(lvl_touch * 100, color=INK2, lw=0.8)
ax.text(lvl_touch * 100 - 0.8, 4.0, f"intervals separate only at\nlevels below {lvl_touch * 100:.1f}%",
        ha="right", va="center", fontsize=10, color=INK)
ax.axvline(lvl_eq * 100, color=INK2, lw=0.8, ls=":")
ax.text(lvl_eq * 100 + 0.5, 4.0, f"{lvl_eq * 100:.1f}%: non-overlap here\n\u21d4 Wald p < 0.05", ha="left", va="center",
        fontsize=10, color=INK2)
ax.text(91.0, -9.5, "overlap", ha="center", fontsize=10, color=INK)
ax.set_xlim(50, 99)
ax.set_ylim(-27, 8)
ax.set_xlabel("Confidence level of both intervals (%)")
ax.set_ylabel("Percentage points")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.legend(loc="lower left", frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written to", HERE)

Printed stroke confidence interval excludes its own estimate; Table 2 shows the effect is non-significant

JAMA Network Open (2025) · doi:10.1001/jamanetworkopen.2025.39862 · PMC12569718

Summary

This UK Biobank cohort study used longitudinal targeted maximum likelihood estimation to estimate how 5-year risks of hospitalization for cardiovascular disease would change under hypothetical PM2.5 interventions at thresholds of 12, 10 and 9 µg/m³. The Abstract and the Results text give the stroke risk difference at the 9 µg/m³ threshold (the US EPA annual standard) as −2.26 per mille with a 95% CI of −8.97 to −20.64. That interval is impossible: its lower bound is above its upper bound, and the point estimate lies outside it. Table 2, Supplement 1 eTable 2, and a recomputation from the Table 2 risks all give an upper bound of about +20.6, so the interval includes zero and the stroke effect at 9 µg/m³ is not statistically significant. The Key Points nonetheless say that the 12 and 9 µg/m³ thresholds "would significantly reduce" stroke, heart failure and myocardial infarction. By Table 2, that holds for myocardial infarction, but not for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. Two smaller sign discrepancies between the text and Table 2 point in the same direction.

What the paper reports

The PMC-archived text uses hyphen-minus signs, and the quotes below reproduce it. The publisher's version of record prints the same characters as true minus signs (U+2212).

Key Points, Findings:

implementing a hypothetical intervention for reducing PM2.5 exposure by 5% if it is above the thresholds of 12 and 9 µg/m3 would significantly reduce the 5-year risk of hospitalization for stroke, heart failure, and myocardial infarction, but not for arrhythmia.

Abstract, Results (the same sentence fragment also appears in the Results text, in the paragraph that describes Table 2):

the hypothetical PM2.5 intervention of reducing PM2.5 exposure by 5% if it is above the threshold of 9 µg/m3, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)

Results text, Table 2 paragraph:

For the threshold intervention of 9 µg/m3 compared with no intervention, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)

Table 2, stroke rows (columns: hypothetical intervention | 5-y risk (95% CI), per mille | risk difference (95% CI), per mille):

No intervention | 12.32 (11.92 to 12.72) | [Reference] ≤12 | 10.78 (10.07 to 11.67) | -1.54 (-2.21 to -0.73) ≤10 | 6.98 (3.76 to 19.60) | -5.34 (-8.59 to 7.24) ≤9 | 10.06 (3.37 to 32.97) | -2.26 (-8.97 to 20.64)

Supplement 1, eTable 2, stroke, main analysis (columns: no-intervention risk | ≤9 µg/m³ risk | risk difference):

Main analysis 12.32 (11.92, 12.72) 10.06 (3.37, 32.97) -2.26 (-8.97 to 20.64)

The two further discrepancies. Abstract, Results, stroke at 12 µg/m³:

the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to 0.73 per mille). However, the reduction in risk for arrhythmia was not statistically significant

Results text, stroke at 12 µg/m³, which agrees with Table 2:

the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to -0.73 per mille)

Results text, heart failure at 10 µg/m³:

for heart failure, -2.15 per mille (95% CI, -3.55 to -0.35 per mille)

Table 2, heart failure row ≤10:

≤10 | 4.35 (2.95 to 6.89) | -2.15 (-3.55 to 0.35)

Heart failure at 12 µg/m³, which agrees between the Results text and Table 2:

for heart failure, -0.51 per mille (95% CI, -1.07 to 0.35 per mille)

The significance criterion, from Methods, Statistical Analysis:

Two-sided P < .05 was considered statistically significant.

The statements that build on these results. Discussion, first paragraph:

Our findings indicate a decreased 5-year risk of hospitalization with a primary diagnosis of stroke, myocardial infarction, or heart failure for the hypothetical intervention of reducing PM2.5 exposure by 5% if it is above the annual average standards of 12 µg/m3, 10 µg/m3, or 9 µg/m3 compared with no intervention on ambient PM2.5. However, the 5-year risk difference was not statistically significant for the hospitalization with a primary diagnosis of arrhythmia.

Discussion, comparison with earlier studies:

While our study found decreased 5-year risks of stroke, myocardial infarction, and heart failure for the hypothetical PM2.5 intervention, the risk difference was not statistically significant for the outcome of arrhythmia.

Conclusions:

Our results indicate that a more stringent ambient PM2.5 standard in the United Kingdom may lower the risk of hospitalization for stroke, myocardial infarction, and heart failure.

What the paper's own numbers show

Method. Table 2 gives, for each of four outcomes, the 5-year risk with no intervention (the reference) and under each of the three threshold interventions, each with a 95% CI, plus the risk difference (RD) with its own 95% CI. I made two checks from Table 2 alone. (a) The RD point estimate should equal the intervention risk minus the reference risk. (b) The RD CI should be close to the intervention-risk CI minus the reference point estimate. Check (b) is an approximation: it ignores the uncertainty in the reference risk and its correlation with the intervention risk. It works here because the reference risks are estimated far more precisely (for stroke, 11.92 to 12.72) than the intervention risks (for stroke at 9 µg/m³, 3.37 to 32.97). I measured the approximation's accuracy against all 24 Table 2 RD bounds and used that accuracy as the yardstick for the text-versus-table discrepancies. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Accuracy of check (a), all 12 RD point estimates Table 2 — every RD equals risk(intervention) − risk(no intervention) within 0.010 per mille (rounding)
Accuracy of check (b), all 24 RD CI bounds Table 2 — every bound within 0.22 per mille of risk CI − reference risk (largest gap: arrhythmia ≤10 upper, 1.28 vs 1.06)
Stroke RD, ≤9 µg/m³ Abstract; Results text −2.26 (−8.97 to −20.64) lower bound > upper bound; −2.26 is outside the interval, and stays outside if the bounds are reordered (−20.64 to −8.97)
Stroke RD, ≤9 µg/m³ Table 2; eTable 2 −2.26 (−8.97 to 20.64) 10.06 − 12.32 = −2.26; (3.37 − 12.32, 32.97 − 12.32) = (−8.95 to +20.65). Includes 0
Stroke RD upper bound, ≤9 µg/m³ — — text vs recomputed: 41.29 apart; Table 2 vs recomputed: 0.01 apart. Using either end of the reference CI instead of 12.32 gives 20.25 to 21.05, so the bound is positive in every case
Stroke RD, ≤9 µg/m³, three robustness checks eTable 2 −3.39 (−9.25 to 18.47); −2.26 (−9.30 to 20.49); −4.02 (−9.26 to 14.91) (−9.17 to 18.39); (−9.27 to 20.54); (−9.20 to 15.05). All include 0
Stroke RD, ≤12 µg/m³ Abstract −1.54 (−2.21 to 0.73) Table 2 and Results text: −2.21 to −0.73; recomputed (−2.25 to −0.65). The abstract's upper bound is 1.38 from the recomputed value, against 0.08 for Table 2
Heart failure RD, ≤10 µg/m³ Results text −2.15 (−3.55 to −0.35) Table 2: −3.55 to 0.35; recomputed (−3.55 to +0.39). The text's upper bound is 0.74 from the recomputed value, against 0.04 for Table 2. Includes 0
Heart failure RD, ≤12 µg/m³ Results text; Table 2 −0.51 (−1.07 to 0.35) recomputed (−1.13 to +0.41). Consistent, and includes 0
Stroke RD, ≤10 µg/m³ Results text; Table 2 −5.34 (−8.59 to 7.24) recomputed (−8.56 to +7.28). Consistent, and includes 0
Myocardial infarction RD, ≤12 and ≤9 µg/m³ Results text; Table 2 −1.41 (−1.98 to −0.67); −8.64 (−9.16 to −6.38) recomputed (−2.03 to −0.62); (−9.12 to −6.44). Consistent, and both exclude 0
Heart failure RD, ≤9 µg/m³ Abstract; Results text; Table 2 −3.20 (−4.16 to −1.25) recomputed (−4.10 to −1.30). Consistent, and excludes 0
All other text values (abstract and Results text) — — identical to Table 2

The paper does not report P values for the main risk differences. When a test and its 95% CI are derived consistently, a CI that includes zero corresponds to a two-sided P ≥ .05, which is the paper's own significance threshold. By Table 2, the significance of each risk difference is:

Outcome ≤12 µg/m³ ≤10 µg/m³ ≤9 µg/m³
Stroke significant not significant not significant
Myocardial infarction significant significant significant
Heart failure not significant not significant (the Results text's −0.35 would make it appear significant) significant
Arrhythmia not significant not significant not significant

The Key Points claim six significant reductions (stroke, heart failure and myocardial infarction, each at 12 and 9 µg/m³). Table 2 supports four of them. The two it does not support are stroke at 9 µg/m³, where only the impossible text interval excludes zero, and heart failure at 12 µg/m³, where the text and the table agree that the interval includes zero.

One further, minor observation. In Figure 3 of the paper (a forest plot of the 9 µg/m³ main analysis and robustness checks), the stroke main-analysis interval is drawn crossing the zero line, ending near 0.02 on the axis. That agrees with Table 2. The axis is labeled "per mille", but its tick values (−0.02 to 0.03) appear to be on the proportion scale, where 0.02 corresponds to 20 per mille.

Figure

Stroke risk difference at the 9 µg/m³ threshold: the interval printed in the text, the interval in Table 2 and eTable 2, and the interval recomputed from the Table 2 risks

Figure 1. The 5-year stroke risk difference for the 9 µg/m³ intervention versus no intervention, in per mille, on one linear axis. Orange marks are values printed in the paper; the hollow orange marker is the text value that disagrees with Table 2. Blue is the interval recomputed as the Table 2 risk CI under the intervention (3.37 to 32.97) minus the no-intervention risk (12.32). The thin vertical line is zero difference. The interval printed in the Abstract and Results text (−8.97 to −20.64) lies entirely to the left of its own point estimate. Table 2, eTable 2 and the recomputation agree on an interval of about −9 to +20.6, which includes zero. Plus signs in the value column are added for clarity; the paper prints positive bounds without a sign.

The two smaller sign discrepancies, and the heart failure result at 12 µg/m³

Figure 2. Three risk differences on one linear axis (per mille), using the same encoding as Figure 1. For stroke at 12 µg/m³, the Abstract prints an upper bound of +0.73. The Results text and Table 2 print −0.73, and the recomputed bound is −0.65. For heart failure at 10 µg/m³, the Results text prints −0.35, while Table 2 prints +0.35 and the recomputed bound is +0.39. In both cases the recomputation sides with Table 2, within the approximation's measured accuracy of 0.22 per mille. Heart failure at 12 µg/m³ is printed consistently, and its interval includes zero. Even so, the Key Points describe it as a significant reduction.

Why it matters

The 9 µg/m³ threshold is the one the paper ties to the US EPA primary annual standard, and the Abstract reports the 9 µg/m³ results first. The impossible stroke interval appears twice, in the Abstract and in the Results text. In both places all of the printed interval lies below zero, which makes the stroke reduction look statistically significant. The paper's own table says otherwise. Under the correct interval (−8.97 to +20.64 per mille) the data are compatible with anything from about a 9 per mille decrease to a 21 per mille increase in 5-year stroke hospitalization risk. Stroke is also not significant at 10 µg/m³, and it is not significant in any of the three robustness checks at 9 µg/m³. So the paper's data support a significant stroke reduction only at 12 µg/m³.

The Key Points sentence ("thresholds of 12 and 9 µg/m3 would significantly reduce ... stroke, heart failure, and myocardial infarction") is what most readers take from the article. Read naturally, it is not supported by Table 2 for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. The Discussion's first paragraph lists decreased risk of stroke, myocardial infarction and heart failure at all three thresholds and contrasts them with arrhythmia, which "was not statistically significant". Read that way, the paragraph implies significance for stroke at 10 and 9 µg/m³ and for heart failure at 12 and 10 µg/m³, none of which Table 2 supports. The point estimates are all negative, so "decreased" is accurate as a description of direction only.

The scope of the problem is limited. Myocardial infarction is significant at every threshold, and heart failure at 9 µg/m³. The general conclusion that stricter PM2.5 standards may lower cardiovascular hospitalization is still supported by those results. What changes is the outcome-specific claim, which for stroke matters at the threshold most relevant to current policy.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. In the Abstract (Results) and the Results text, change the stroke risk difference at 9 µg/m³ to "−2.26 per mille (95% CI, −8.97 to 20.64 per mille)", to match Table 2 and eTable 2.
  2. In the Abstract, change the stroke interval at 12 µg/m³ to "−2.21 to −0.73 per mille". In the Results text, change the heart failure interval at 10 µg/m³ to "−3.55 to 0.35 per mille".
  3. Revise the Key Points to match Table 2. For example: the 12 and 9 µg/m³ thresholds significantly reduced the risk for myocardial infarction. The reduction for stroke was significant at 12 µg/m³ but not at 10 or 9 µg/m³. The reduction for heart failure was significant at 9 µg/m³ but not at 12 or 10 µg/m³. No threshold significantly reduced the risk for arrhythmia.
  4. Revise the first Discussion paragraph and the comparison paragraph so that they do not imply significant stroke reductions at 10 and 9 µg/m³ or significant heart failure reductions at 12 and 10 µg/m³. Qualify the Conclusions sentence on stroke accordingly.
  5. Minor: check the Figure 3 axis, whose tick values appear to be proportions although the label says per mille.
Recomputation script (python recompute.py.txt)
"""Recompute the risk-difference confidence intervals of PMC12569718
(JAMA Netw Open 2025, doi:10.1001/jamanetworkopen.2025.39862).

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC (Abstract, Results text, Table 2)
and from Supplement 1, eTable 2. Units: per mille (per 1000 participants), 5-year risk.

Method. Table 2 reports, for each outcome, the 5-year risk under no intervention
(the reference) and under each threshold intervention, each with a 95% CI, plus the
risk difference (RD) with its own 95% CI. Two checks follow from Table 2 alone:
  (a) RD point estimate = risk(intervention) - risk(no intervention);
  (b) RD 95% CI ~= risk(intervention) CI - risk(no intervention) point estimate.
(b) is an approximation: it ignores the uncertainty in the reference risk and its
correlation with the intervention risk. It works here because the reference risk is
estimated far more precisely than the intervention risks. Its accuracy is measured
below against the 24 Table 2 RD bounds, and that measured accuracy is the yardstick
for the discrepancies between the text and the table.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from matplotlib.transforms import blended_transform_factory  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
OUTCOMES = ("stroke", "MI", "HF", "arrhythmia")
NAMES = {"stroke": "Stroke", "MI": "Myocardial infarction", "HF": "Heart failure",
         "arrhythmia": "Arrhythmia"}
THRESHOLDS = (12, 10, 9)

# Table 2, "No intervention" rows: 5-y risk (95% CI), per mille
REF = {"stroke": (12.32, 11.92, 12.72), "MI": (11.47, 11.08, 11.84),
       "HF": (6.50, 6.23, 6.76), "arrhythmia": (18.86, 18.36, 19.35)}

# Table 2, threshold rows: (5-y risk (95% CI)), (risk difference (95% CI)), per mille
T2 = {
    "stroke": {12: ((10.78, 10.07, 11.67), (-1.54, -2.21, -0.73)),
               10: ((6.98, 3.76, 19.60), (-5.34, -8.59, 7.24)),
               9: ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64))},
    "MI": {12: ((10.07, 9.44, 10.85), (-1.41, -1.98, -0.67)),
           10: ((4.15, 3.47, 5.54), (-7.32, -8.05, -5.86)),
           9: ((2.83, 2.35, 5.03), (-8.64, -9.16, -6.38))},
    "HF": {12: ((5.99, 5.37, 6.91), (-0.51, -1.07, 0.35)),
           10: ((4.35, 2.95, 6.89), (-2.15, -3.55, 0.35)),
           9: ((3.29, 2.40, 5.20), (-3.20, -4.16, -1.25))},
    "arrhythmia": {12: ((16.80, 14.07, 21.79), (-2.06, -4.79, 3.12)),
                   10: ((9.81, 6.28, 19.92), (-9.05, -12.59, 1.28)),
                   9: ((14.70, 6.14, 31.74), (-4.16, -12.70, 12.93))},
}

# Abstract, Results section: RD (95% CI) as printed
ABSTRACT = {("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
            ("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93),
            ("stroke", 12): (-1.54, -2.21, 0.73), ("arrhythmia", 12): (-2.06, -4.79, 3.12)}

# Results text, paragraph describing Table 2: RD (95% CI) as printed
RESULTS = {("stroke", 12): (-1.54, -2.21, -0.73), ("MI", 12): (-1.41, -1.98, -0.67),
           ("HF", 12): (-0.51, -1.07, 0.35), ("arrhythmia", 12): (-2.06, -4.79, 3.12),
           ("stroke", 10): (-5.34, -8.59, 7.24), ("MI", 10): (-7.32, -8.05, -5.86),
           ("HF", 10): (-2.15, -3.55, -0.35), ("arrhythmia", 10): (-9.05, -12.59, 1.28),
           ("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
           ("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93)}

# Supplement 1, eTable 2, stroke at <=9 ug/m3: (5-y risk (95% CI)), (RD (95% CI))
# The no-intervention risk is 12.32 (11.92, 12.72) in every row.
ETABLE2_STROKE = {
    "Main analysis": ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64)),
    "Further adjustment for NO2 and NOx (2010)": ((8.93, 3.15, 30.71), (-3.39, -9.25, 18.47)),
    "Further adjustment for family history of CVD": ((10.06, 3.05, 32.86), (-2.26, -9.30, 20.49)),
    "Further adjustment for BMI": ((8.30, 3.12, 27.37), (-4.02, -9.26, 14.91)),
}

# Key Points, Findings: thresholds 12 and 9 "would significantly reduce" these outcomes
KEY_POINTS_CLAIM = [(o, th) for o in ("stroke", "HF", "MI") for th in (12, 9)]


def approx_ci(o, risk):
    """RD 95% CI approximated as the intervention-risk CI minus the reference point."""
    r0 = REF[o][0]
    return risk[1] - r0, risk[2] - r0


def sig(lo, hi):
    return "significant" if (hi < 0 or lo > 0) else "NOT significant (CI includes 0)"


def f(x):
    return f"{x:+.2f}".replace("+-", "-")


# ---------------------------------------------------------------- 1. Table 2 internal check
print("1) Table 2 internal check")
print("   RD = risk(int) - risk(no int); RD CI ~= risk(int) CI - risk(no int)")
APPROX = {}
max_pt, max_bd = 0.0, 0.0
for o in OUTCOMES:
    for th in THRESHOLDS:
        risk, (d, dl, dh) = T2[o][th]
        pt = risk[0] - REF[o][0]
        al, ah = approx_ci(o, risk)
        APPROX[(o, th)] = (pt, al, ah)
        max_pt = max(max_pt, abs(pt - d))
        max_bd = max(max_bd, abs(al - dl), abs(ah - dh))
        print(f"   {o:10s} <={th:2d}: RD reported {d:6.2f}, recomputed {pt:6.2f} | "
              f"CI reported ({dl:6.2f}, {dh:6.2f}), recomputed ({al:6.2f}, {ah:6.2f}) | "
              f"Table 2: {sig(dl, dh)}")
print(f"   Largest |reported - recomputed| RD point estimate over 12 cells: {max_pt:.3f}")
print(f"   Largest |reported - recomputed| RD CI bound over 24 bounds:      {max_bd:.3f}")

# ---------------------------------------------------------------- 2. text vs Table 2


def check_text(label, d):
    print(f"\n2) {label} vs Table 2")
    out = []
    for (o, th), (e, lo, hi) in d.items():
        te, tl, thi = T2[o][th][1]
        issues = []
        if lo > hi:
            issues.append("lower bound > upper bound")
        if not (min(lo, hi) <= e <= max(lo, hi)):
            issues.append("estimate outside its own CI")
        if (e, lo, hi) != (te, tl, thi):
            diffs = [n for n, a, b in (("estimate", e, te), ("lower", lo, tl), ("upper", hi, thi))
                     if a != b]
            issues.append(f"differs from Table 2 ({te}, {tl} to {thi}) in: {', '.join(diffs)}")
        _, al, ah = APPROX[(o, th)]
        print(f"   {o:10s} <={th:2d}: printed {e} ({lo} to {hi}) | Table 2 {te} ({tl} to {thi}) | "
              f"recomputed ({al:.2f} to {ah:.2f}) | {'; '.join(issues) or 'OK'}")
        if issues:
            out.append((o, th))
    print(f"   cells with a problem: {out}")
    return out


check_text("Abstract (Results)", ABSTRACT)
check_text("Results text", RESULTS)

print("\n   Size of each text-vs-table discrepancy, against the approximation's accuracy:")
for label, (o, th), which, printed in (
        ("Abstract & Results, stroke <=9 upper", ("stroke", 9), 2, -20.64),
        ("Abstract, stroke <=12 upper", ("stroke", 12), 2, 0.73),
        ("Results, HF <=10 upper", ("HF", 10), 2, -0.35)):
    tab = T2[o][th][1][which]
    rec = APPROX[(o, th)][which]
    print(f"   {label:38s}: printed {printed:7.2f}, Table 2 {tab:6.2f}, recomputed {rec:6.2f} | "
          f"|printed - recomputed| = {abs(printed - rec):5.2f} vs |Table 2 - recomputed| = "
          f"{abs(tab - rec):4.2f}")

# The printed stroke <=9 interval cannot be made valid by reordering its bounds
e, lo, hi = ABSTRACT[("stroke", 9)]
print(f"\n   Stroke <=9 printed interval reordered: ({min(lo, hi)} to {max(lo, hi)}); "
      f"contains estimate {e}? {min(lo, hi) <= e <= max(lo, hi)}")
r = T2["stroke"][9][0]
print(f"   Stroke <=9 RD upper bound using either end of the reference CI instead of 12.32: "
      f"{r[2] - REF['stroke'][2]:.2f} to {r[2] - REF['stroke'][1]:.2f} (all > 0)")

# ---------------------------------------------------------------- 3. Key Points vs Table 2
print("\n3) Key Points: thresholds 12 and 9 'would significantly reduce' stroke, HF and MI")
for o, th in KEY_POINTS_CLAIM:
    d, lo, hi = T2[o][th][1]
    _, al, ah = APPROX[(o, th)]
    print(f"   {o:6s} <={th:2d}: Table 2 {d} ({lo} to {hi}) -> {sig(lo, hi)}; "
          f"recomputed CI ({al:.2f} to {ah:.2f}) -> {sig(al, ah)}")

print("\n   Significance grid from Table 2 (S = CI excludes 0, ns = CI includes 0):")
print("   " + " " * 22 + "  ".join(f"<={th:2d}" for th in THRESHOLDS))
for o in OUTCOMES:
    cells = ["  S " if (T2[o][th][1][2] < 0 or T2[o][th][1][1] > 0) else " ns " for th in THRESHOLDS]
    print(f"   {NAMES[o]:22s}" + "  ".join(cells))

# ---------------------------------------------------------------- 4. Supplement eTable 2
print("\n4) Supplement 1, eTable 2, stroke at <=9 (main analysis and robustness checks)")
for k, (risk, (d, dl, dh)) in ETABLE2_STROKE.items():
    pt = risk[0] - REF["stroke"][0]
    al, ah = risk[1] - REF["stroke"][0], risk[2] - REF["stroke"][0]
    print(f"   {k:45s}: RD {d} ({dl} to {dh}), recomputed {pt:.2f} ({al:.2f} to {ah:.2f}) "
          f"-> {sig(dl, dh)}")

# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)
    print(f"   wrote {stem}.svg and {stem}.png")


def interval(ax, y, est, lo, hi, color, marker="o", filled=True):
    a, b = min(lo, hi), max(lo, hi)
    ax.plot([a, b], [y, y], color=color, lw=2, solid_capstyle="butt", zorder=3)
    for x in (a, b):
        ax.plot([x, x], [y - 0.12, y + 0.12], color=color, lw=2, zorder=3)
    ax.plot([est], [y], marker=marker, ms=9, color=color, mfc=color if filled else BG,
            mew=2, zorder=4, ls="none")


def style(ax, xlo, xhi, xlabel):
    ax.set_xlim(xlo, xhi)
    ax.axvline(0, color=SEC, lw=0.8, zorder=1)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel(xlabel, fontsize=10.5, color=TXT)


print("\n5) Figures")


def value_col(ax, y, ee, ll, hh):
    ax.text(1.03, y, f"{f(ee)} ({f(ll)} to {f(hh)})".replace("-", "\u2212"),
            transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=TXT)


LEG_REPORTED_BAD = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mfc=BG, mew=2,
                          label="Reported in text, differs from Table 2")
LEG_REPORTED = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in table")
LEG_RECOMP = Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
                    label="Recomputed from Table 2 risks")

# ---- fig1: stroke at <=9 ug/m3
fig, ax = plt.subplots(figsize=(7.2, 3.9))
e, lo, hi = ABSTRACT[("stroke", 9)]
t_e, t_lo, t_hi = T2["stroke"][9][1]
s_e, s_lo, s_hi = ETABLE2_STROKE["Main analysis"][1]
r_e, r_lo, r_hi = APPROX[("stroke", 9)]
rows = [
    ("Abstract and Results text", e, lo, hi, ORANGE, False),
    ("Table 2", t_e, t_lo, t_hi, ORANGE, True),
    ("Supplement 1, eTable 2", s_e, s_lo, s_hi, ORANGE, True),
    ("Recomputed: Table 2 risk CI\nminus no-intervention risk", r_e, r_lo, r_hi, BLUE, True),
]
ys = [3, 2, 1, 0]
for (lab, ee, ll, hh, c, filled), y in zip(rows, ys):
    interval(ax, y, ee, ll, hh, c, filled=filled)
    value_col(ax, y, ee, ll, hh)
ax.text(1.03, 3.6, "RD (95% CI)", transform=ax.get_yaxis_transform(),
        va="center", ha="left", fontsize=10, color=SEC)
ax.annotate("estimate lies outside its printed interval", xy=(e - 0.3, 3.12),
            xytext=(-23.5, 3.62), fontsize=10, color=SEC, ha="left", va="center", zorder=5,
            bbox=dict(fc=BG, ec="none", pad=1.5),
            arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, 3.95)
style(ax, -24, 24, "5-year risk difference for stroke, \u22649 \u00b5g/m\u00b3\nvs no intervention (per mille)")
ax.text(0.5, -0.55, "no difference", fontsize=10, color=SEC, va="bottom")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
           bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
           handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.80, bottom=0.18)
save(fig, "fig1")

# ---- fig2: the two smaller sign discrepancies and the HF <=12 cell
fig, ax = plt.subplots(figsize=(7.2, 4.9))
groups = [
    ("Stroke, \u226412 \u00b5g/m\u00b3", [("Abstract", ABSTRACT[("stroke", 12)], ORANGE, False),
                          ("Results text, Table 2", T2["stroke"][12][1], ORANGE, True),
                          ("Recomputed", APPROX[("stroke", 12)], BLUE, True)]),
    ("Heart failure, \u226410 \u00b5g/m\u00b3", [("Results text", RESULTS[("HF", 10)], ORANGE, False),
                                 ("Table 2", T2["HF"][10][1], ORANGE, True),
                                 ("Recomputed", APPROX[("HF", 10)], BLUE, True)]),
    ("Heart failure, \u226412 \u00b5g/m\u00b3", [("Results text, Table 2", T2["HF"][12][1], ORANGE, True),
                                 ("Recomputed", APPROX[("HF", 12)], BLUE, True)]),
]
y = 0
yt, yl, heads = [], [], []
for gname, items in groups:
    heads.append((gname, y))
    y -= 0.75
    for lab, (ee, ll, hh), c, filled in items:
        interval(ax, y, ee, ll, hh, c, filled=filled)
        value_col(ax, y, ee, ll, hh)
        yt.append(y)
        yl.append(lab)
        y -= 0.75
    y -= 0.35
for gname, yy in heads:
    ax.text(0.015, yy, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
            ha="left", va="center",
            fontsize=10.5, fontweight="bold", color=TXT)
ax.text(1.03, 0, "RD (95% CI)", transform=ax.get_yaxis_transform(), va="center", ha="left",
        fontsize=10, color=SEC)
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.6, 0.45)
style(ax, -4, 1, "5-year risk difference vs no intervention (per mille)")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
           bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
           handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.86, bottom=0.11)
save(fig, "fig2")

Registry share of Asian or Pacific Islander patients is 2.98%, not 5.9%, reversing reported underrepresentation

JAMA Network Open (2022) · doi:10.1001/jamanetworkopen.2022.39884 · PMC9634497

Summary

The study compares the racial and ethnic makeup of 8309 participants in US phase 1 metastatic-cancer drug trials with incident cases in the Cancer in North America (CiNA) registry. For the whole 2000-2018 period, the Results report that Asian or Pacific Islander (API) participants were underrepresented: 4.5% of trial participants against 5.9% of registry cases, a difference of −1.5 percentage points (P < .001). The registry count and denominator printed in the same sentence, 145 568 of 4 891 486, give 2.98%, not 5.9%. That count is the exact sum of the paper's two period counts, whose shares (2.7% and 3.4%) are printed correctly. Measured against 2.98%, trial participation is higher than the registry share under either trial count the paper gives: +1.49 points (P ≈ 2 × 10⁻¹⁵) with the Table's 371 of 8309, or +0.30 points (P = .12, not significant) with the sentence's 272 of 8308. The reported overall API underrepresentation, which the Conclusions repeat, therefore does not follow from the paper's own data. Separately, the API trial counts in the Table do not add up (121 + 151 = 272, but the "All" row is 371), and this bears on the API time trend.

What the paper reports

Results, third paragraph, the overall (2000-2018) comparison. The PMC-archived text has a hyphen before 1.5; the publisher's HTML and the Europe PMC XML have a true minus sign (U+2212):

an underrepresentation of Asian or Pacific Islander participants (trials, 272 of 8308 [4.5%]; CiNA, 145 568 of 4 891 486 [5.9%]; difference, -1.5 percentage points; P < .001)

The same paragraph, period-specific comparison (repeated in the Abstract):

Asian or Pacific Islander patients (2000-2011: trials, 121 of 3245 [3.7%]; CiNA, 75 033 of 2 800 711 [2.7%]; difference, 1.1 percentage points; 2012-2018: trials, 151 of 5063 [3.0%]; CiNA 70 535 of 2 090 775 [3.4%]; difference, -0.75 percentage points)

Results, second paragraph, the overall trial count. The Abstract gives "371 Asian or Pacific Islander participants (4.5%)":

Of the 8309 total participants, 23 (0.3%) were American Indian or Alaska Native, 371 (4.5%) were Asian or Pacific Islander

The Table, "Phase 1 trials", Asian or Pacific Islander column:

All (N = 8309) | 371 (4.5) 2000-2011 (n = 3245) | 121 (3.7) 2012-2018 (n = 5063) | 151 (3.0)

The Table, "Industry trials", same column:

2012-2018 (n = 4628) | 226 (4.9)

Methods, "Statistical Analysis", the test, the pooling and the sign convention:

Proportions accounting for the sample size of each trial were calculated for each racial and ethnic group by combining the total number of participants for each racial and ethnic group and dividing by the total number of trial participants

differences with 95% CIs were estimated using a 2-sample test for equality of proportions with continuity correction

with positive percentage differences indicating overrepresentation and negative percentage differences indicating underrepresentation

Results, third paragraph, the ratios shown in Figure 3:

especially Asian and Pacific Islander patients (ratio of trial to CiNA participants, 1.44 in 2000-2011 to 0.87 in 2012-2018)

Key Points, "Findings":

found worsening underrepresentation of American Indian or Alaska Native, Asian or Pacific Islander, Black, and Hispanic or Latinx participants over time

Conclusions:

These include underenrollment of Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in clinical trials overall and underenrollment of American Indian or Alaska Native patients, Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in trials conducted at academic institutions

What the paper's own numbers show

Method. I took every count printed in the Abstract, Results, Table and Supplement eTable, recomputed each share, and checked whether overall counts equal the sums of the period counts. I then reran each comparison with the test the Methods name: a two-sample test for equality of proportions with continuity correction (R's prop.test, whose statistic is the Yates-corrected χ² on the 2 × 2 table), with its continuity-corrected 95% CI. As a check that this is the paper's test, it reproduces the reported White 2000-2011 result (+0.76 points, P = .235; reported 0.8, P = .23) and the overall White (+2.01; reported 2.0) and Black (−5.64; reported −5.6) differences. It also matches every significance marker in the paper's Figure 2: P < .001 for API 2000-2011, P = .048 for American Indian or Alaska Native 2012-2018 (marked P < .05), and no marker for API 2012-2018 (P = .13). Every number below is printed by recompute.py.txt.

Additivity. The overall CiNA counts equal the sums of the period counts, for API (75 033 + 70 535 = 145 568), for the denominator (2 800 711 + 2 090 775 = 4 891 486), and for White and Black. The overall API registry share is therefore the pooled share of the two periods, so it must lie between 2.68% and 3.37%. It cannot be 5.9%.

Quantity Location Reported Recomputed from the paper's numbers
CiNA API share, 2000-2011 Abstract; Results ¶3 75 033 / 2 800 711 [2.7%] 2.68% (consistent)
CiNA API share, 2012-2018 Abstract; Results ¶3 70 535 / 2 090 775 [3.4%] 3.37% (consistent)
CiNA API count, 2000-2018 Results ¶3 145 568 75 033 + 70 535 = 145 568 (exact)
CiNA API share, 2000-2018 Results ¶3 145 568 / 4 891 486 [5.9%] 2.98%. 5.9% of 4 891 486 would be 288 598 cases (1.98 × the printed count)
CiNA groups summed, with API at 5.9% derived — White + Black + AIAN + API = 5 000 409, which exceeds the 4 891 486 total by 108 923
Trial API share, 2000-2018 (sentence) Results ¶3 272 / 8308 [4.5%] 3.27%. The bracketed 4.5% belongs to 371/8309
Trial API share, 2000-2018 (Table) Table "All"; Abstract; Results ¶2 371 / 8309 (4.5) 4.47% (consistent)
Overall API difference, trials − CiNA, using 371/8309 Results ¶3 −1.5 pp, P < .001 +1.49 pp (95% CI +1.04 to +1.94), P = 1.9 × 10⁻¹⁵
Overall API difference, using 272/8308 Results ¶3 −1.5 pp, P < .001 +0.30 pp (95% CI −0.09 to +0.69), P = .12
Overall API difference from the printed shares Results ¶3 −1.5 pp 4.5 − 5.9 = −1.4; 4.47 − 5.9 = −1.43. Neither gives −1.5
Overall API trial/CiNA ratio not reported (underrepresentation, ratio < 1) 1.50 (371/8309) or 1.10 (272/8308)
API difference, 2000-2011 Abstract; Results ¶3 +1.1 pp +1.05 pp (95% CI +0.38 to +1.72), P = .0003 (rounding-level)
API difference, 2012-2018 Abstract; Results ¶3 −0.75 pp, no P value −0.39 pp (95% CI −0.87 to +0.09), P = .13; printed shares give 3.0 − 3.4 = −0.4
API trial/CiNA ratio, 2000-2011 → 2012-2018 Results ¶3; Figure 3 1.44 → 0.87 1.39 → 0.88
White overall difference (method check) Abstract; Results ¶3 +2.0 pp, P < .001 +2.01 pp, P = 5.8 × 10⁻⁷ (consistent)
Black overall difference (method check) Results ¶3 −5.6 pp, P < .001 −5.64 pp, P = 7.9 × 10⁻⁵⁷ (consistent)

Against the registry share its own counts imply, the overall API difference has the reported size (1.49 points with the Table's count) but the opposite sign.

A second, separate inconsistency: the API trial counts. In the Table every other column adds up across the period rows (White 2780 + 4374 = 7154; Black 244 + 270 = 514; and so on). The API column does not.

Quantity Location Reported Recomputed
API, 2000-2011 + 2012-2018 vs "All" Table, phase 1 trials 121 + 151; All = 371 121 + 151 = 272, which is the count used in the Results sentence; 99 short of 371
API, industry-funded subset vs all trials, 2012-2018 Table industry 226 (4.9); all trials 151 (3.0) the subset exceeds the total by 75. The industry rows themselves add up (106 + 226 = 332)
"All" row minus the 2000-2011 row Table — 371 − 121 = 250 (4.94% of 5063), which is ≥ the industry count of 226
Supplement eTable (EGFR-positive trials added back) minus Table, overall vs by period eTable; Table — additive for White (406 = 406), Black (17 = 17), AIAN (1 = 1) and Hispanic (4 = 4). For API, 870 − 371 = 499 overall but 598 by period. A 2012-2018 Table count of 250 would make API additive too
API difference, 2012-2018, if the count were 250 derived — +1.56 pp (95% CI +0.96 to +2.17), P = 9.7 × 10⁻¹⁰; ratio 1.46

So the printed 151, the printed 371, or both cannot be right. The paper's other numbers (the industry subset and the eTable) are consistent with 371 overall and 250 in 2012-2018, but the paper alone does not establish which figure is wrong.

Smaller points, none of which changes a conclusion. The CiNA White share for 2012-2018, 1 735 077 / 2 090 775 = 82.99%, is printed as 82.9% (difference recomputes to +3.40 against the printed 3.5). The Table footnote gives 5076 participants in trials reporting ethnicity, but its period denominators sum to 1792 + 3295 = 5087. The overall Black comparison uses 8308 as the denominator against 8309 elsewhere. The text ratios for Black (0.64 → 0.42) and Hispanic or Latinx (1.55 → 0.98) recompute to 0.65 → 0.44 and 1.49 → 0.98. In the paper's Figure 3 the plotted API point estimates sit near 1.39 and 0.88, and the text values 1.44 and 0.87 lie close to the ends of the plotted intervals (read from the image, so approximate).

Figure

Asian or Pacific Islander shares: values printed in the paper vs values recomputed from the counts printed beside them

Figure 1. Asian or Pacific Islander shares on one percentage axis starting at zero. Orange rings are the values printed in the paper. Blue dots are the values recomputed from the numerator and denominator printed with them (listed under each row label). For each period the two agree (2.7 vs 2.68; 3.4 vs 3.37). For 2000-2018 the paper prints 5.9%, but its own count, 145 568 of 4 891 486, gives 2.98%. Because that count and denominator are the sums of the period counts, the pooled share has to fall in the shaded band between the two period shares. Both trial figures the paper gives for 2000-2018, 3.27% (272/8308, Results sentence, printed there as 4.5%) and 4.47% (371/8309, Table and Abstract), lie at or above 2.98%, not below it.

Asian or Pacific Islander difference, trial share minus CiNA share: printed vs recomputed, with 95% CIs

Figure 2. Trial share minus CiNA share, in percentage points, with 95% CIs from the test the paper names (two-sample test for equality of proportions with continuity correction). The thin vertical line marks no difference; by the paper's convention, negative values mean underrepresentation. Orange rings are printed values (the paper gives no CIs for these comparisons). For 2000-2018 the paper prints −1.5 (P < .001). Recomputed against the registry share its counts imply, the difference is +1.49 with the Table's count and +0.30 (P = .12) with the sentence's count. For 2000-2011 the printed and recomputed values agree. For 2012-2018 the printed −0.75 recomputes to −0.39 (P = .13). The hollow blue marker shows the 2012-2018 difference if the all-trial count were 371 − 121 = 250, the value the Table's "All" row and industry subset imply; it is a derived scenario, not a value printed in the paper.

Why it matters

Explanations considered

None of these explains the printed 5.9%. The material published with the paper does not show how it arose.

Status

Sources: doi.org/10.1001/jamanetworkopen.2022.39884; publisher full text; PMC9634497; Europe PMC full-text XML; Europe PMC supplementary files; Crossref record.

Suggested correction

  1. Results, overall API comparison. Recompute the registry share as 145 568 / 4 891 486 = 3.0% and settle the trial count (item 2). The sentence would then report an overrepresentation, "trials, 371 of 8309 [4.5%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 1.5 percentage points; P < .001", or, if 272 of 8308 is correct, "trials, 272 of 8308 [3.3%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 0.3 percentage points; P = .12", with no significant difference.
  2. Table, API column. Reconcile the "All" row (371) with the period rows (121 + 151 = 272), and the all-trial 2012-2018 count (151) with the industry-funded 2012-2018 count (226). Then update the 2012-2018 API comparison in the Abstract and Results (currently −0.75 points, which recomputes to −0.39 with 151), Figure 2, Figure 3 and the ratios in the text.
  3. Conclusions and Key Points. Revise the statement of API underenrollment "in clinical trials overall", and the API part of "worsening underrepresentation ... over time", in line with items 1 and 2. Check the statements on API and Hispanic or Latinx underenrollment overall and at academic centers against the Table's counts, which are above the CiNA shares.
  4. Minor. Change 82.9% to 83.0% for 1 735 077 / 2 090 775, reconcile the Hispanic or Latinx denominator (5076 vs 1792 + 3295 = 5087), use 8309 consistently, and state whether the ratios quoted in the text are point estimates.
Recomputation script (python recompute.py.txt)
"""Recompute the Asian or Pacific Islander (API) registry comparison of PMC9634497
(JAMA Netw Open 2022;5(11):e2239884, doi:10.1001/jamanetworkopen.2022.39884).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC:
  Results, paragraphs 2-3 (trial and CiNA counts, by period and overall)
  Table (trial counts: all trials, industry-funded, academic centers)
  Supplement eTable (the same Table with the EGFR+ NSCLC trials added back)
Test: the paper states "a 2-sample test for equality of proportions with continuity
correction" (R prop.test). prop.test's statistic equals the Yates-corrected chi-square on
the 2x2 table, and its CI is the Wald interval widened by the continuity correction.
"""
import math
import os

import numpy as np
from scipy.stats import chi2_contingency, norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
# CiNA registry counts, Results paragraph 3 (period) and paragraph 2 (overall)
CINA_N = {"2000-2011": 2800711, "2012-2018": 2090775, "overall": 4891486}
CINA = {  # group: (2000-2011, 2012-2018, overall as printed or None if not printed)
    "White": (2378019, 1735077, 4113096),
    "Black": (322701, 255625, 578326),
    "API": (75033, 70535, 145568),
    "AIAN": (10905, 9484, None),
    "Hispanic": (169297, 156118, None),
}
CINA_API_PCT_PRINTED_OVERALL = 5.9

# Trial counts, Table ("All" rows) and Results text
TRIAL_N = {"2000-2011": 3245, "2012-2018": 5063, "overall": 8309}
TRIAL = {  # group: (2000-2011, 2012-2018, Table "All")
    "White": (2780, 4374, 7154),
    "Black": (244, 270, 514),
    "API": (121, 151, 371),
    "AIAN": (10, 13, 23),
    "Hispanic": (161, 240, 401),
}
HISP_TRIAL_N = {"2000-2011": 1792, "2012-2018": 3295, "overall": 5076}  # Table footnote
API_TEXT = (272, 8308)        # Results paragraph 2, overall API comparison
BLACK_TEXT = (514, 8308)      # Results paragraph 2, overall Black comparison

# Table subsets (API column): (2000-2011, 2012-2018, All), and subset sizes
INDUSTRY_API = (106, 226, 332)
INDUSTRY_N = (2737, 4628, 7365)
ACADEMIC_API = (68, 153, 221)
ACADEMIC_N = (1806, 4145, 5951)
ACADEMIC_HISP = (62, 214, 276)
ACADEMIC_HISP_N = (841, 2682, None)
ACADEMIC_AIAN_ALL = 16

# Supplement eTable, "All Phase I" rows: (2001-2011, 2012-2018, All)
ETABLE = {
    "White": (2906, 4654, 7560),
    "Black": (253, 278, 531),
    "API": (126, 744, 870),
    "AIAN": (10, 14, 24),
    "Hispanic": (164, 241, 405),
}

# Reported values (text) for the comparisons
REPORTED = {
    "API overall": dict(trial=4.5, cina=5.9, diff=-1.5, p="< .001"),
    "API 2000-2011": dict(trial=3.7, cina=2.7, diff=1.1, p="not given"),
    "API 2012-2018": dict(trial=3.0, cina=3.4, diff=-0.75, p="not given"),
    "White overall": dict(trial=86.1, cina=84.1, diff=2.0, p="< .001"),
    "Black overall": dict(trial=6.2, cina=11.8, diff=-5.6, p="< .001"),
    "White 2000-2011": dict(trial=85.7, cina=84.9, diff=0.8, p=".23"),
    "White 2012-2018": dict(trial=86.4, cina=82.9, diff=3.5, p="< .001"),
    "Black 2000-2011": dict(trial=7.5, cina=11.5, diff=-4.0, p="not given"),
    "Black 2012-2018": dict(trial=5.3, cina=12.2, diff=-6.9, p="not given"),
    "AIAN 2000-2011": dict(trial=0.3, cina=0.4, diff=-0.08, p="not given"),
    "AIAN 2012-2018": dict(trial=0.3, cina=0.5, diff=-0.20, p="not given"),
    "Hispanic 2000-2011": dict(trial=9.0, cina=6.0, diff=3.0, p="not given"),
    "Hispanic 2012-2018": dict(trial=7.3, cina=7.5, diff=-0.2, p="not given"),
}
REPORTED_RATIOS = {  # Results paragraph 3: (2000-2011, 2012-2018)
    "API": (1.44, 0.87), "Black": (0.64, 0.42), "Hispanic": (1.55, 0.98)}


# ---------------------------------------------------------------- helpers
def pct(a, b):
    return 100.0 * a / b


def prop_test(x1, n1, x2, n2, conf=0.95):
    """R prop.test(c(x1,x2), c(n1,n2)) with continuity correction.
    Returns (p1 %, p2 %, diff pp, ci_lo pp, ci_hi pp, P)."""
    p1, p2 = x1 / n1, x2 / n2
    delta = p1 - p2
    yates = min(0.5, abs(delta) / (1 / n1 + 1 / n2))
    z = norm.ppf(0.5 + conf / 2)
    width = z * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2) + yates * (1 / n1 + 1 / n2)
    table = np.array([[x1, n1 - x1], [x2, n2 - x2]])
    _, p, _, _ = chi2_contingency(table, correction=True)
    return 100 * p1, 100 * p2, 100 * delta, 100 * (delta - width), 100 * (delta + width), p


def fmt_p(p):
    return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"


def line(label, res, reported=None):
    p1, p2, d, lo, hi, p = res
    s = (f"  {label:46s} trial {p1:6.2f}%  CiNA {p2:6.2f}%  diff {d:+6.2f} pp "
         f"(95% CI {lo:+.2f} to {hi:+.2f})  P = {fmt_p(p)}")
    if reported:
        s += (f"\n  {'':46s} reported: {reported['trial']}% vs {reported['cina']}%, "
              f"diff {reported['diff']:+} pp, P {reported['p']}")
    print(s)


results = {}

# ---------------------------------------------------------------- 1. sums and the CiNA share
print("1. Do the overall counts equal the sum of the period counts?")
print(f"  CiNA N: {CINA_N['2000-2011']} + {CINA_N['2012-2018']} = "
      f"{CINA_N['2000-2011'] + CINA_N['2012-2018']} (printed overall {CINA_N['overall']})")
for g, (a, b, o) in CINA.items():
    print(f"  CiNA {g:8s}: {a} + {b} = {a + b}" + (f" (printed overall {o})" if o else " (overall not printed)"))
print(f"  Trial N: {TRIAL_N['2000-2011']} + {TRIAL_N['2012-2018']} = "
      f"{TRIAL_N['2000-2011'] + TRIAL_N['2012-2018']} (Table 'All' N = {TRIAL_N['overall']})")
for g, (a, b, o) in TRIAL.items():
    flag = "" if a + b == o else "   <-- does not add up"
    print(f"  Trial {g:8s}: {a} + {b} = {a + b} (Table 'All' = {o}){flag}")
print(f"  Hispanic trial N (footnote): {HISP_TRIAL_N['2000-2011']} + {HISP_TRIAL_N['2012-2018']} = "
      f"{HISP_TRIAL_N['2000-2011'] + HISP_TRIAL_N['2012-2018']} (footnote overall {HISP_TRIAL_N['overall']})")

print("\n2. CiNA share of Asian or Pacific Islander patients")
api_c = CINA["API"]
cina_api_p00 = pct(api_c[0], CINA_N["2000-2011"])
cina_api_p12 = pct(api_c[1], CINA_N["2012-2018"])
cina_api_all = pct(api_c[2], CINA_N["overall"])
print(f"  2000-2011: {api_c[0]}/{CINA_N['2000-2011']} = {cina_api_p00:.2f}% (printed 2.7%)")
print(f"  2012-2018: {api_c[1]}/{CINA_N['2012-2018']} = {cina_api_p12:.2f}% (printed 3.4%)")
print(f"  2000-2018: {api_c[2]}/{CINA_N['overall']} = {cina_api_all:.2f}% (printed {CINA_API_PCT_PRINTED_OVERALL}%)")
print(f"  A pooled share must lie between the period shares: {cina_api_p00:.2f}% to {cina_api_p12:.2f}%")
implied = round(CINA_API_PCT_PRINTED_OVERALL / 100 * CINA_N["overall"])
print(f"  Count implied by 5.9% of {CINA_N['overall']}: {implied} "
      f"({implied / api_c[2]:.2f} x the printed 145 568)")
w, b = CINA["White"][2], CINA["Black"][2]
aian = CINA["AIAN"][0] + CINA["AIAN"][1]
print(f"  White + Black + AIAN + API (printed counts): {w} + {b} + {aian} + {api_c[2]} = "
      f"{w + b + aian + api_c[2]} of {CINA_N['overall']}")
print(f"  ... with API at 5.9%: {w + b + aian + implied} of {CINA_N['overall']} "
      f"(exceeds the total by {w + b + aian + implied - CINA_N['overall']})")
print(f"  White + Black + API(5.9%) alone: {w + b + implied} "
      f"({'exceeds' if w + b + implied > CINA_N['overall'] else 'within'} the total)")

print("\n3. Trial share of Asian or Pacific Islander participants")
print(f"  Results text: {API_TEXT[0]}/{API_TEXT[1]} = {pct(*API_TEXT):.2f}% (printed 4.5%)")
print(f"  Table/Abstract: {TRIAL['API'][2]}/{TRIAL_N['overall']} = {pct(TRIAL['API'][2], TRIAL_N['overall']):.2f}% (printed 4.5%)")
print(f"  {API_TEXT[0]} = {TRIAL['API'][0]} + {TRIAL['API'][1]} (sum of the Table's period rows): "
      f"{API_TEXT[0] == TRIAL['API'][0] + TRIAL['API'][1]}")

# ---------------------------------------------------------------- 4. the tests
print("\n4. Two-sample tests with continuity correction (method check, then API)")
for key, (x1, n1, x2, n2) in {
    "White overall": (7154, 8309, 4113096, 4891486),
    "Black overall": (BLACK_TEXT[0], BLACK_TEXT[1], 578326, 4891486),
    "White 2000-2011": (2780, 3245, 2378019, 2800711),
    "White 2012-2018": (4374, 5063, 1735077, 2090775),
    "Black 2000-2011": (244, 3245, 322701, 2800711),
    "Black 2012-2018": (270, 5063, 255625, 2090775),
    "AIAN 2000-2011": (10, 3245, 10905, 2800711),
    "AIAN 2012-2018": (13, 5063, 9484, 2090775),
    "Hispanic 2000-2011": (161, 1792, 169297, 2800711),
    "Hispanic 2012-2018": (240, 3295, 156118, 2090775),
}.items():
    results[key] = prop_test(x1, n1, x2, n2)
    line(key, results[key], REPORTED[key])

print()
results["API overall, 371/8309"] = prop_test(371, 8309, 145568, 4891486)
results["API overall, 272/8308"] = prop_test(272, 8308, 145568, 4891486)
results["API 2000-2011"] = prop_test(121, 3245, 75033, 2800711)
results["API 2012-2018"] = prop_test(151, 5063, 70535, 2090775)
line("API overall, Table count 371/8309", results["API overall, 371/8309"], REPORTED["API overall"])
line("API overall, text count 272/8308", results["API overall, 272/8308"], REPORTED["API overall"])
line("API 2000-2011", results["API 2000-2011"], REPORTED["API 2000-2011"])
line("API 2012-2018", results["API 2012-2018"], REPORTED["API 2012-2018"])
d371 = results["API overall, 371/8309"][2]
print(f"  Reported -1.5 vs recomputed {d371:+.2f} with 371/8309: same size, opposite sign")
print(f"  Printed percentages: 4.5 - 5.9 = {4.5 - 5.9:+.1f};  4.47 - 5.9 = "
      f"{pct(371, 8309) - 5.9:+.2f};  4.47 - 2.98 = {pct(371, 8309) - cina_api_all:+.2f}")
print(f"  Printed percentages for 2012-2018: 3.0 - 3.4 = {3.0 - 3.4:+.1f} (printed -0.75)")

# ---------------------------------------------------------------- 5. ratios
print("\n5. Ratio of trial share to CiNA share (Results paragraph 3, Figure 3)")
ratio = {}
for g, (n_t, n_c) in {"API": (TRIAL_N, CINA_N), "Black": (TRIAL_N, CINA_N),
                      "Hispanic": (HISP_TRIAL_N, CINA_N)}.items():
    r00 = (TRIAL[g][0] / n_t["2000-2011"]) / (CINA[g][0] / n_c["2000-2011"])
    r12 = (TRIAL[g][1] / n_t["2012-2018"]) / (CINA[g][1] / n_c["2012-2018"])
    ratio[g] = (r00, r12)
    print(f"  {g:8s}: {r00:.3f} -> {r12:.3f}   (text: {REPORTED_RATIOS[g][0]} -> {REPORTED_RATIOS[g][1]})")
r_all_371 = (371 / 8309) / (145568 / 4891486)
r_all_272 = (272 / 8308) / (145568 / 4891486)
print(f"  API overall: {r_all_371:.3f} (371/8309), {r_all_272:.3f} (272/8308)")

# ---------------------------------------------------------------- 6. Table consistency
print("\n6. Table: subsets vs all trials (API column)")
print(f"  2012-2018: industry {INDUSTRY_API[1]}, academic {ACADEMIC_API[1]}, all trials {TRIAL['API'][1]}"
      f"  -> industry subset exceeds all trials: {INDUSTRY_API[1] > TRIAL['API'][1]}")
print(f"  Industry rows add up: {INDUSTRY_API[0]} + {INDUSTRY_API[1]} = {INDUSTRY_API[0] + INDUSTRY_API[1]} "
      f"(All {INDUSTRY_API[2]}); academic: {ACADEMIC_API[0]} + {ACADEMIC_API[1]} = "
      f"{ACADEMIC_API[0] + ACADEMIC_API[1]} (All {ACADEMIC_API[2]})")
print(f"  Industry 2012-2018 API share: {INDUSTRY_API[1]}/{INDUSTRY_N[1]} = {pct(INDUSTRY_API[1], INDUSTRY_N[1]):.2f}% (printed 4.9)")
print(f"  All-trials 2012-2018 API share: {TRIAL['API'][1]}/{TRIAL_N['2012-2018']} = "
      f"{pct(TRIAL['API'][1], TRIAL_N['2012-2018']):.2f}% (printed 3.0)")
implied_12 = TRIAL["API"][2] - TRIAL["API"][0]
print(f"  All-row minus 2000-2011 row: {TRIAL['API'][2]} - {TRIAL['API'][0]} = {implied_12} "
      f"({pct(implied_12, TRIAL_N['2012-2018']):.2f}% of {TRIAL_N['2012-2018']}); >= industry {INDUSTRY_API[1]}: "
      f"{implied_12 >= INDUSTRY_API[1]}")
print("  Supplement eTable minus Table, overall vs sum of periods (EGFR+ trials added back):")
for g, (e0, e1, eall) in ETABLE.items():
    t0, t1, tall = TRIAL[g]
    d_all = eall - tall
    d_per = (e0 - t0) + (e1 - t1)
    print(f"    {g:8s}: eTable periods {e0}+{e1}={e0 + e1} (All {eall}); added overall {d_all}, "
          f"added by period {d_per}{'' if d_all == d_per else '   <-- differs'}")
x = ETABLE["API"][2] - TRIAL["API"][2] - (ETABLE["API"][0] - TRIAL["API"][0])
x = ETABLE["API"][1] - x
print(f"  2012-2018 API count that makes the API column additive like the others: {x}")
results["API 2012-2018, count 250"] = prop_test(implied_12, 5063, 70535, 2090775)
line("API 2012-2018 if count were 371-121=250", results["API 2012-2018, count 250"])
print(f"  ratio with 250: {(implied_12 / 5063) / (70535 / 2090775):.3f}")

# ---------------------------------------------------------------- 7. Conclusions sentence
print("\n7. Groups named in the Conclusions as under-enrolled 'overall' and 'at academic institutions'")
hisp_c = CINA["Hispanic"][0] + CINA["Hispanic"][1]
aian_c = CINA["AIAN"][0] + CINA["AIAN"][1]
results["Hispanic overall"] = prop_test(401, 5076, hisp_c, CINA_N["overall"])
results["AIAN overall"] = prop_test(23, 8309, aian_c, CINA_N["overall"])
line("Hispanic overall (401/5076 vs CiNA period sum)", results["Hispanic overall"])
line("AIAN overall (23/8309 vs CiNA period sum)", results["AIAN overall"])
line("Black overall", results["Black overall"])
line("API overall, Table count 371/8309", results["API overall, 371/8309"])
results["Academic API overall"] = prop_test(ACADEMIC_API[2], ACADEMIC_N[2], 145568, CINA_N["overall"])
results["Academic API 2000-2011"] = prop_test(ACADEMIC_API[0], ACADEMIC_N[0], 75033, CINA_N["2000-2011"])
results["Academic API 2012-2018"] = prop_test(ACADEMIC_API[1], ACADEMIC_N[1], 70535, CINA_N["2012-2018"])
ah_all_n = ACADEMIC_HISP_N[0] + ACADEMIC_HISP_N[1]
results["Academic Hispanic overall"] = prop_test(ACADEMIC_HISP[2], ah_all_n, hisp_c, CINA_N["overall"])
results["Academic Hispanic 2000-2011"] = prop_test(ACADEMIC_HISP[0], ACADEMIC_HISP_N[0], 169297, CINA_N["2000-2011"])
results["Academic Hispanic 2012-2018"] = prop_test(ACADEMIC_HISP[1], ACADEMIC_HISP_N[1], 156118, CINA_N["2012-2018"])
results["Academic AIAN overall"] = prop_test(ACADEMIC_AIAN_ALL, ACADEMIC_N[2], aian_c, CINA_N["overall"])
for k in ["Academic API overall", "Academic API 2000-2011", "Academic API 2012-2018",
          "Academic Hispanic overall", "Academic Hispanic 2000-2011", "Academic Hispanic 2012-2018",
          "Academic AIAN overall"]:
    line(k, results[k])
print(f"  (academic Hispanic denominator {ACADEMIC_HISP_N[0]} + {ACADEMIC_HISP_N[1]} = {ah_all_n}; "
      f"276/{ah_all_n} = {pct(276, ah_all_n):.2f}%, printed 7.8)")

# ---------------------------------------------------------------- figures
INK, INK2 = "#0b0b0b", "#52514e"
BG, ORANGE, BLUE, GRID = "#fcfcfb", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "lines.linewidth": 2,
})


def style(ax):
    ax.grid(axis="x", color=GRID, linewidth=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: shares, reported vs recomputed
rows1 = [  # label, reported %, recomputed %, recomputed label
    ("CiNA registry, 2000–2011\n75 033 / 2 800 711", 2.7, cina_api_p00),
    ("CiNA registry, 2012–2018\n70 535 / 2 090 775", 3.4, cina_api_p12),
    ("CiNA registry, 2000–2018\n145 568 / 4 891 486", 5.9, cina_api_all),
    ("Trials, 2000–2018 (Results text)\n272 / 8308", 4.5, pct(*API_TEXT)),
    ("Trials, 2000–2018 (Table, Abstract)\n371 / 8309", 4.5, pct(371, 8309)),
]
fig, ax = plt.subplots(figsize=(7.2, 4.7))
y = np.arange(len(rows1))[::-1].astype(float)
y[3:] -= 0.35  # gap between registry and trial rows
ax.axvspan(cina_api_p00, cina_api_p12, ymin=0.47, ymax=1.0, color=BLUE, alpha=0.10, linewidth=0)
ax.text((cina_api_p00 + cina_api_p12) / 2, y[0] + 0.55,
        "a pooled 2000–2018 share\nmust fall in this band", ha="center", va="bottom",
        fontsize=10, color=INK2)
for (lab, rep, rec), yy in zip(rows1, y):
    ax.plot([min(rep, rec), max(rep, rec)], [yy, yy], color=GRID, linewidth=2, zorder=1)
    ax.scatter([rep], [yy], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=3)
    ax.scatter([rec], [yy], s=70, color=BLUE, zorder=4)
ax.text(5.9 + 0.15, y[2], "printed 5.9%", va="center", color=ORANGE, fontsize=10.5)
ax.text(cina_api_all - 0.15, y[2] - 0.32, f"{cina_api_all:.2f}%", ha="right", va="center",
        color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[3], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(*API_TEXT) - 0.15, y[3] - 0.32, f"{pct(*API_TEXT):.2f}%", ha="right", va="center",
        color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[4], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(371, 8309) - 0.15, y[4] - 0.32, f"{pct(371, 8309):.2f}%", ha="right", va="center",
        color=BLUE, fontsize=10.5)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows1], fontsize=10)
ax.set_xlim(0, 7.6)
ax.set_ylim(y[-1] - 0.7, y[0] + 1.35)
ax.set_xlabel("Asian or Pacific Islander share, %")
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
                label="printed in the paper")
h2 = ax.scatter([], [], s=70, color=BLUE, label="recomputed from the paper's own counts")
fig.legend(handles=[h1, h2], loc="lower center", frameon=False, fontsize=10, ncol=2,
           bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.08, 1, 1))
save(fig, "fig1")

# Figure 2: differences trial - CiNA with 95% CIs
rows2 = [  # label, reported diff or None, result key or None, hollow
    ("2000–2018, as printed", REPORTED["API overall"]["diff"], None),
    ("2000–2018, Table count 371 / 8309", None, "API overall, 371/8309"),
    ("2000–2018, text count 272 / 8308", None, "API overall, 272/8308"),
    ("2000–2011, 121 / 3245", REPORTED["API 2000-2011"]["diff"], "API 2000-2011"),
    ("2012–2018, 151 / 5063", REPORTED["API 2012-2018"]["diff"], "API 2012-2018"),
    ("2012–2018, if 371 − 121 = 250", None, "API 2012-2018, count 250"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
y = np.array([5.6, 4.6, 3.6, 2.2, 1.0, 0.0])
ax.axvline(0, color=INK2, linewidth=0.9, zorder=1)
for (lab, rep, key), yy in zip(rows2, y):
    off = 0.14 if (rep is not None and key is not None) else 0.0
    if key is not None:
        _, _, d, lo, hi, p = results[key]
        hollow = key.endswith("250")
        ax.plot([lo, hi], [yy - off, yy - off], color=BLUE, linewidth=2, zorder=2)
        ax.scatter([d], [yy - off], s=70, zorder=3, linewidth=2,
                   facecolor=BG if hollow else BLUE, edgecolor=BLUE)
        ptxt = "P < .001" if p < 0.001 else f"P = {p:.2f}".replace("0.", ".")
        ax.text(hi + 0.08, yy - off, f"{d:+.2f}  ({ptxt})".replace("-", "\u2212"), va="center",
                color=BLUE, fontsize=10)
    if rep is not None:
        ax.scatter([rep], [yy + off], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=4)
        ptxt = "  (P < .001)" if key is None else ""
        txt = f"{rep:+g}{ptxt}".replace("-", "\u2212")
        if key is None:
            ax.text(rep, yy + 0.24, txt.replace("  ", " "), va="bottom", ha="center",
                    color=ORANGE, fontsize=10)
        else:
            ax.text(rep + (0.14 if rep > 0 else -0.14), yy + off, txt, va="center",
                    ha="left" if rep > 0 else "right", color=ORANGE, fontsize=10)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_xlim(-2.6, 3.9)
ax.set_ylim(-0.6, 6.4)
ax.set_xlabel("Trial share minus CiNA share, percentage points\n"
              "← under-representation          over-representation →", fontsize=10.5)
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
                label="printed in the paper")
h2 = ax.errorbar([], [], xerr=[], fmt="o", color=BLUE, markersize=8, linewidth=2,
                 label="recomputed, 95% CI (prop.test)")
h3 = ax.scatter([], [], s=70, facecolor=BG, edgecolor=BLUE, linewidth=2,
                label="recomputed for a count implied by the Table")
fig.legend(handles=[h1, h2, h3], loc="lower center", frameon=False, fontsize=10, ncol=2,
           bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.12, 1, 1))
save(fig, "fig2")
print("\nFigures written to", HERE)

Right anterior insula odds ratio of 0.96 lies outside its own reported 95% CI (0.01–0.75)

JAMA Psychiatry (2015) · doi:10.1001/jamapsychiatry.2014.1604 · PMC5103694

Summary

The paper's main predictive result, given in the abstract and in the Results, is that right anterior insula (AI) volume predicted a diagnosis of major depressive disorder (MDD) after imaging, with "odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03". The point estimate 0.96 lies above the upper bound of its own 95% CI, which is impossible for any standard interval. The CI and the P value agree with each other: on the log-odds scale the CI is centred on an OR of about 0.087 (0.061–0.106 when the printed bounds are allowed to vary within their rounding), and that implies a Wald P of 0.026, which matches the reported .03. An OR of 0.96 with the same standard error would give P ≈ 0.97. The direction and statistical significance of the finding therefore stand, but the printed effect size understates the association the paper's own CI and P value imply, roughly a 91% rather than a 4% lower odds of later MDD per unit of whole-brain-adjusted volume.

What the paper reports

The abstract:

"When covarying for children’s lifetime history of MDD episodes, their experience of pathological guilt during preschool, as well as their sex and age at the time of imaging, schoolchildren’s right-side AI volume was a significant predictor of being diagnosed as having an MDD episode after imaging (odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03)." (Abstract, Results)

The same model in the Results, together with the other estimates printed in the same paragraph:

"Children with an MDD diagnosis up to the time of imaging were approximately 11 times as likely as same-age peers without a prior diagnosis of MDD to have an MDD diagnosis after their imaging (odds ratio [OR], 11.38; 95% CI, 2.88–44.94; P < .01). However, even when including the robust effect of MDD up to the time of imaging, as well as children’s age, sex, and PO guilt symptoms (OR, 0.86; 95% CI, 0.28–2.65; P = .79), children with larger right-side AI volumes were significantly less likely to be diagnosed as having full MDD after their imaging (diagnosed on average 14 months after the imaging date [OR, 0.96; 95% CI, 0.01–0.75; P = .03]) (Figure 3). Left-side AI volume was not significantly associated with children’s MDD diagnosis after the time of imaging (OR, 2.70; 95% CI, 0.24–29.88; P = .42)." (Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1)

"Results indicated that smaller right AI volume remained a significant predictor (OR, 0.12; 95% CI, 0.01–0.99; P = .04) of DSM-5 MDD diagnosis after imaging when covarying for MDD from baseline up until the time of imaging, age, sex, PO guilt, and children’s stressful or traumatic life events." (Results, same paragraph, follow-up model with life-event covariates)

The model and the predictor scale:

"Binary logistic regression analysis was used to test whether schoolchildren’s right or left AI volume predicted the likelihood of being diagnosed as having MDD any time after their imaging." (Methods, Statistical Analysis, "AI Volume as a Predictor of Full MDD After Imaging")

"AI volumes were adjusted by the total segmented whole-brain volume (structure divided by whole-brain volume, times 1000) before all analyses." (Methods, "Magnetic Resonance Imaging Acquisition and AI Volume Analysis", para 2)

"Fourth was MDD diagnosed after the time of imaging (yes or no) (MDD after imaging variable [n = 24]). This variable is used as the dependent variable in the final analysis." (Methods, Measures, "DSM Psychiatric Diagnoses")

What the paper's own numbers show

Method. A logistic-regression Wald interval is symmetric on the log-odds scale. From a printed OR, lower bound L and upper bound U, the estimate implied by the interval is the geometric midpoint exp((ln L + ln U)/2), the standard error is SE = (ln U − ln L)/(2 × 1.96), and the two-sided P value is 2Φ(−|ln OR|/SE). This was applied to all five estimates in the Results paragraph. For the right AI estimate the printed bounds were also varied over every value that rounds to them (0.005–0.015 and 0.745–0.755). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed
Right AI OR, main model: is the OR inside its CI? Abstract; Results para 1 OR 0.96; 95% CI 0.01–0.75 No: 0.96 > 0.75
Right AI, OR implied by the printed CI derived from Abstract / Results 0.96 0.087 (0.061–0.106 allowing for rounding of the bounds)
Right AI, SE(log OR) implied by the CI derived — 1.10 (1.00–1.28 allowing for rounding)
Right AI, P implied by the CI Abstract; Results para 1 P = .03 0.026 (z = −2.22; 0.024–0.029 allowing for rounding)
Right AI, P that OR 0.96 would have at that SE derived P = .03 0.97 (0.967–0.975)
Right AI, 95% CI that OR 0.96 would need for P = .03 derived 0.01–0.75 0.925–0.996 (SE 0.019)
Prior MDD Results para 1 OR 11.38; 2.88–44.94; P < .01 midpoint 11.377; P = 0.001 (consistent)
PO guilt Results para 1 OR 0.86; 0.28–2.65; P = .79 midpoint 0.861; P = 0.795 (consistent)
Left AI volume Results para 1 OR 2.70; 0.24–29.88; P = .42 midpoint 2.678; P = 0.423 (consistent)
Right AI, + life-event covariates Results para 1 OR 0.12; 0.01–0.99; P = .04 midpoint 0.099 (0.070–0.122 allowing for rounding); P = 0.048–0.050 (OR inside its CI; P close)

In the four other estimates the printed OR sits at the log-midpoint of its CI to within rounding: for prior MDD, for example, ln(OR) − ln(L) = 1.374 and ln(U) − ln(OR) = 1.373. Only the right AI main-model estimate departs from this pattern: ln(0.96) − ln(0.01) = 4.564, while ln(0.75) − ln(0.96) = −0.247. The CI implies an SE of 1.10, close to the left AI estimate's 1.23 and the follow-up model's 1.17, so the interval's width is in line with the other volume estimates from the same 24 events. It looks asymmetric only because the printed OR lies outside it.

Figure

Reported odds ratios and CIs from the Results paragraph compared with values recomputed from the printed intervals, on one log axis

Figure 1. Each row is one estimate from the Results paragraph on MDD after imaging, on a single log-scale odds-ratio axis, with the thin vertical line at OR = 1. Orange circles and lines are the ORs and 95% CIs as printed in the paper. Open blue diamonds mark the geometric midpoint of each printed CI; for the bottom four estimates they coincide with, or sit close to, the printed OR. The top row is the right AI main-model estimate as printed: the OR (0.96) is to the right of its own interval (0.01–0.75). The second row shows the estimate implied by that interval, 0.087 (blue diamond; the printed CI is shown faded); the short thick blue bar spans 0.061–0.106, the values obtained when the printed bounds vary within their rounding. The third row shows the 95% CI (0.925–0.996, an open blue square with a short line hidden mostly behind it) that an OR of 0.96 would need in order to have P = .03. All recomputed values come from recompute.py.txt.

Wald P value as a function of the right AI odds ratio at the standard error implied by the printed CI

Figure 2. The blue curve gives the two-sided Wald P value for any right AI odds ratio, at the standard error (SE of log OR = 1.10) implied by the printed CI of 0.01–0.75. The reported P = .03 falls on the curve at OR ≈ 0.087 (filled blue diamond, P ≈ 0.026). At the printed OR of 0.96 the curve gives P ≈ 0.97 (open blue diamond). The orange circle marks the reported pair (OR 0.96, P = .03), which lies far off the curve: the printed CI and P are consistent with each other, and the printed OR is not consistent with either.

Why it matters

The right AI result is the third aim of the study (the Objectives ask "whether AI volumes predict subsequent likelihood of having an episode of MDD") and the only predictive result reported in the abstract. The Discussion builds on it: it says that "right-side AI volume was a significant and robust predictor of DSM-5 MDD diagnosis after imaging", that "these study findings provide evidence for the first structural brain biomarker to date of risk for recurrent depression in childhood", and that reduced AI volume "served as a statistically significant biomarker of later depression recurrence".

The direction and significance of the association are not affected. The printed CI excludes 1, the printed P is .03, the two agree, and the follow-up model with life-event covariates (OR 0.12; 0.01–0.99; P = .04) points the same way. What is wrong is the effect size, and the difference is large. Read at face value, 0.96 describes a 4% reduction in the odds of later MDD per unit of whole-brain-adjusted AI volume (structure ÷ whole-brain volume × 1000), and an estimate that close to 1 could not reach P = .03 with an interval this wide. The interval and P value instead imply an OR near 0.09, a 91% reduction per unit, though one that is very imprecisely estimated: the printed CI spans a 75-fold range and the outcome has 24 events. A reader who uses the printed OR would conclude that the predictive effect is negligible, or would find the result self-contradictory. The correct point estimate cannot be read from the paper; only the authors' regression output can supply it. Studies that pool or cite this estimate would carry the error forward.

Explanations considered

Typesetting or archiving artifact. Ruled out. The same "0.96; 95% CI, 0.01–0.75; P = .03" appears in the abstract and in the Results of the PMC author manuscript and of the version of record on jamanetwork.com. The number sits in running prose, so a lost minus sign, a misread mid-dot or a shifted table cell cannot explain it.

Different scale or units for the predictor. Ruled out. Rescaling AI volume (per unit vs per SD, or a different multiplier on the whole-brain adjustment) changes the OR and both CI bounds together. It cannot place the estimate outside its own interval.

A different model behind the OR than behind the CI. Not supported. The Methods describe a single binary logistic regression with age, sex, prior MDD, PO symptoms and AI volume, and the abstract and the Results attach the same OR, CI and P to the same covariate set.

Rounding. Ruled out. Allowing each printed bound to take any value that rounds to it moves the implied OR only between 0.061 and 0.106.

Profile-likelihood or other asymmetric interval. Ruled out as an explanation for an estimate outside the interval. Such intervals can sit off-centre around the estimate, but they always contain it. The other four estimates in the paragraph are also almost exactly log-symmetric around their printed ORs, which indicates Wald-type intervals.

One-sided test. Not relevant. A one-sided P value would change the P value but not whether the estimate lies inside its own CI.

A dropped digit (e.g. 0.096 printed as 0.96). Possible but unverified. 0.096 lies within the rounding range of the CI's midpoint (0.061–0.106) and gives P = 0.033 at the implied SE. The paper offers no way to confirm the exact value.

The claim that the CI is "implausibly wide". Not an independent error. The implied SE of 1.10 on the log scale is similar to the left AI estimate's (1.23) and the follow-up model's (1.17). This width is plausible for a continuous predictor with 24 events.

Side observations, not part of this finding. In the follow-up model the printed OR (0.12) lies inside its CI and close to the midpoint (0.099); the CI implies P ≈ 0.049 against a printed .04, a small difference that could reflect a different test statistic. Separately, the Table's group-comparison column prints several "OR" values with negative CI bounds (for example "OR, -0.71; 95% CI, -1.10 to 0.37" for female sex), which looks like a labelling or scale issue in that column. Neither affects the result above.

Status

As of 2026-10-23:

Sources: PMC5103694 (PubMed Central archive, read with src/show.py); https://jamanetwork.com/journals/jamapsychiatry/fullarticle/1935483; https://api.crossref.org/works?filter=updates:10.1001/jamapsychiatry.2014.1604; Europe PMC REST search for DOI:10.1001/jamapsychiatry.2014.1604.

Suggested correction

Replace the right-side AI volume odds ratio in the abstract and in the Results paragraph "Do AI Volumes Predict a Subsequent Depression Diagnosis?" with the value from the fitted logistic regression, checking that it lies within the reported 95% CI (0.01–0.75) and fits P = .03. The paper's own numbers place it near 0.09 (0.061–0.106). If the CI or P value was the term misreported instead, correct that term and state the corrected triplet in both places. The authors could also state the unit of AI volume for which the OR is expressed (per unit of structure ÷ whole-brain volume × 1000) so readers can interpret the magnitude. In the same correction they might review the "OR" entries with negative confidence bounds in the Table.

Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC5103694 write-up and regenerate fig1/fig2.

Run:  python recompute.py.txt
Inputs are the odds ratios, 95% CIs and P values printed in the paper
(Abstract; Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1).
Method: for a Wald interval on the log-odds scale, log(OR) is the midpoint of
log(lower) and log(upper), SE = (log(upper) - log(lower)) / (2 * 1.959964),
and the two-sided P = 2 * (1 - Phi(|log OR| / SE)).
"""
import os
import itertools
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.isf(0.025)  # 1.959964

# label, reported OR, lower, upper, reported P (as printed)
ROWS = [
    ("Prior MDD (main model)",              11.38, 2.88, 44.94, "<.01"),
    ("PO guilt (main model)",                0.86, 0.28,  2.65, ".79"),
    ("Right AI volume (main model)",         0.96, 0.01,  0.75, ".03"),
    ("Left AI volume",                       2.70, 0.24, 29.88, ".42"),
    ("Right AI volume (+ life events)",      0.12, 0.01,  0.99, ".04"),
]

def from_ci(lo, hi):
    a, b = np.log(lo), np.log(hi)
    se = (b - a) / (2 * Z)
    mid = (a + b) / 2
    return np.exp(mid), se, 2 * norm.sf(abs(mid) / se)

def wald_p(or_, se):
    return 2 * norm.sf(abs(np.log(or_)) / se)

def half_unit(x):
    """Interval of values that round to x at the printed number of decimals (2 dp)."""
    return x - 0.005, x + 0.005

print("== 1. Each reported OR against the CI it is printed with ==")
print(f"{'quantity':34s} {'OR':>6s} {'95% CI':>13s} {'OR in CI':>8s} {'geo-mid':>8s} "
      f"{'SE(logOR)':>9s} {'P(mid)':>7s} {'P(rep OR)':>9s} {'rep P':>5s}")
res = {}
for lab, o, l, u, p in ROWS:
    mid, se, pm = from_ci(l, u)
    pr = wald_p(o, se)
    res[lab] = dict(o=o, l=l, u=u, mid=mid, se=se, pm=pm, pr=pr, p=p)
    print(f"{lab:34s} {o:6.2f} {l:5.2f}-{u:<7.2f} {str(l <= o <= u):>8s} {mid:8.3f} "
          f"{se:9.3f} {pm:7.3f} {pr:9.3f} {p:>5s}")
    print(f"{'':34s} log(OR) - log(lower) = {np.log(o)-np.log(l):6.3f}; "
          f"log(upper) - log(OR) = {np.log(u)-np.log(o):6.3f}")

print("\n== 2. Right AI (main model): allow for rounding of the printed CI bounds ==")
# 0.01 could be anything in [0.005, 0.015); 0.75 in [0.745, 0.755)
lo_r, hi_r = half_unit(0.01), half_unit(0.75)
mids, ps, ses, p096 = [], [], [], []
for a, b in itertools.product(np.linspace(*lo_r, 201), np.linspace(*hi_r, 21)):
    m, s, pm = from_ci(a, b)
    mids.append(m); ps.append(pm); ses.append(s); p096.append(wald_p(0.96, s))
print(f"implied OR (geometric midpoint): {min(mids):.3f} to {max(mids):.3f}")
print(f"implied SE(log OR):              {min(ses):.3f} to {max(ses):.3f}")
print(f"implied Wald P:                  {min(ps):.3f} to {max(ps):.3f}  (reported P = .03)")
print(f"P for OR = 0.96 at those SEs:    {min(p096):.3f} to {max(p096):.3f}")

print("\n== 3. CI that OR = 0.96 would need for a two-sided P = .03 ==")
z03 = norm.isf(0.03 / 2)
se_req = abs(np.log(0.96)) / z03
ci_req = np.exp(np.log(0.96) + np.array([-Z, Z]) * se_req)
print(f"z = {z03:.3f}; SE(log OR) = {se_req:.4f}; 95% CI = {ci_req[0]:.3f} to {ci_req[1]:.3f}")
# P range 0.025-0.035 (values that round to .03)
for pp in (0.025, 0.035):
    s = abs(np.log(0.96)) / norm.isf(pp / 2)
    c = np.exp(np.log(0.96) + np.array([-Z, Z]) * s)
    print(f"  if P = {pp:.3f}: 95% CI = {c[0]:.3f} to {c[1]:.3f}")

r = res["Right AI volume (main model)"]
print("\n== 4. Summary values used in the write-up ==")
print(f"Right AI main: geometric midpoint {r['mid']:.3f}, SE {r['se']:.2f}, z {np.log(r['mid'])/r['se']:.2f}, "
      f"P {r['pm']:.3f}; P if OR were 0.96: {r['pr']:.2f}")
print(f"Implied odds reduction per unit volume: {100*(1-r['mid']):.0f}% (vs {100*(1-0.96):.0f}% for OR 0.96)")
print(f"Width of reported CI, upper/lower ratio: {0.75/0.01:.0f}-fold")
lr = res["Left AI volume"]
print(f"SE comparison: right AI {r['se']:.2f}, left AI {lr['se']:.2f}, right AI + life events "
      f"{res['Right AI volume (+ life events)']['se']:.2f}")
print(f"Candidate 0.096 within rounding range of implied OR: {min(mids) <= 0.096 <= max(mids)}; "
      f"P at OR 0.096, SE {r['se']:.3f}: {wald_p(0.096, r['se']):.3f}")
# life-events model rounding
mids2, ps2 = [], []
for a, b in itertools.product(np.linspace(*half_unit(0.01), 201), np.linspace(*half_unit(0.99), 21)):
    m, s, pm = from_ci(a, b); mids2.append(m); ps2.append(pm)
print(f"Right AI + life events: implied OR {min(mids2):.3f}-{max(mids2):.3f}, implied P "
      f"{min(ps2):.3f}-{max(ps2):.3f} (reported OR 0.12, P = .04)")

# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
                     "svg.fonttype": "none", "font.family": "DejaVu Sans"})

def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
    plt.close(fig)

# Figure 1: dot-and-interval plot, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
rows = [  # (y label, kind)
    ("Right AI volume, main model\n(as printed)", "rep_main"),
    ("Right AI volume, main model\n(OR implied by its printed CI)", "mid_main"),
    ("Right AI volume, main model\n(CI needed if OR = 0.96, P = .03)", "req"),
    ("Right AI volume + life events", "other:Right AI volume (+ life events)"),
    ("Left AI volume", "other:Left AI volume"),
    ("PO guilt", "other:PO guilt (main model)"),
    ("Prior MDD", "other:Prior MDD (main model)"),
]
n = len(rows)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.0, n - 0.35, " OR = 1 (no association)", color=SEC, fontsize=9.5, va="center", ha="left")
for i, (lab, kind) in enumerate(rows):
    y = n - 1 - i
    if kind == "rep_main":
        ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, zorder=2)
        ax.plot(0.96, y, "o", ms=9, color=ORA, zorder=3)
        ax.annotate("OR 0.96 printed outside\nits own CI 0.01–0.75", (0.96, y), xytext=(2.2, y),
                    color=ORA, fontsize=9.5, va="center",
                    arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
    elif kind == "mid_main":
        ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, alpha=0.35, zorder=2)
        ax.plot([min(mids), max(mids)], [y, y], color=BLU, lw=5, solid_capstyle="butt", zorder=3)
        ax.plot(r["mid"], y, "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
        ax.annotate(f"≈{r['mid']:.3f} (range {min(mids):.3f}–{max(mids):.3f}\nwith bound rounding); P ≈ {r['pm']:.3f}",
                    (1.12, y), color=BLU, fontsize=9.5, va="center")
    elif kind == "req":
        ax.plot(ci_req, [y, y], color=BLU, lw=2, zorder=2)
        ax.plot(0.96, y, "s", ms=9, color=BLU, mfc=BG, mew=2, zorder=3)
        ax.annotate(f"{ci_req[0]:.3f}–{ci_req[1]:.3f}", (ci_req[1], y), xytext=(2.2, y),
                    color=BLU, fontsize=9.5, va="center",
                    arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
    else:
        d = res[kind.split(":", 1)[1]]
        ax.plot([d["l"], d["u"]], [y, y], color=ORA, lw=2, zorder=2)
        ax.plot(d["o"], y, "o", ms=9, color=ORA, zorder=3)
        ax.plot(d["mid"], y, "D", ms=9, color=BLU, mfc="none", mew=1.6, zorder=4)
ax.set_yticks(range(n))
ax.set_yticklabels([lab for lab, _ in rows][::-1], fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.004, 120)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xticks([0.01, 0.1, 1, 10, 100])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
ax.set_xlabel("Odds ratio for MDD diagnosis after imaging (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORA, marker="o", ms=8, lw=2, label="Reported in the paper (OR and 95% CI)"),
     Line2D([], [], color=BLU, marker="D", ms=8, mfc=BG, mew=2, lw=0,
            label="Recomputed: geometric midpoint of the printed CI (thick bar: rounding range)"),
     Line2D([], [], color=BLU, marker="s", ms=8, mfc=BG, mew=2, lw=2,
            label="Recomputed: 95% CI implied by OR 0.96 with P = .03")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.42, -0.13), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig1")

# Figure 2: Wald P as a function of the OR, at the SE implied by the printed CI
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
grid = np.exp(np.linspace(np.log(0.003), np.log(10), 600))
ax.plot(grid, wald_p(grid, r["se"]), color=BLU, lw=2, zorder=2,
        label=f"Recomputed: Wald P for any OR, SE(log OR) = {r['se']:.2f} from the printed CI")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.plot(0.96, 0.03, "o", ms=10, color=ORA, zorder=4, label="Reported pair: OR 0.96, P = .03")
ax.plot(r["mid"], r["pm"], "D", ms=9, color=BLU, zorder=4)
ax.plot(0.96, r["pr"], "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
ax.annotate(f"OR ≈ {r['mid']:.3f} gives P ≈ {r['pm']:.3f}", (r["mid"], r["pm"]), xytext=(0.006, 0.22),
            color=BLU, fontsize=9.5, arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate(f"OR 0.96 gives P ≈ {r['pr']:.2f}", (0.96, r["pr"]), xytext=(0.5, 0.90),
            color=BLU, fontsize=9.5, ha="right", va="center",
            arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate("printed:\nOR 0.96, P = .03", (0.96, 0.03), xytext=(1.3, 0.10), color=ORA, fontsize=9.5,
            arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
ax.set_xscale("log")
ax.set_xlim(0.003, 10)
ax.set_ylim(0, 1.05)
ax.set_xticks([0.01, 0.1, 1, 10])
ax.set_xticklabels(["0.01", "0.1", "1", "10"])
ax.set_xlabel("Odds ratio for right AI volume (log scale)")
ax.set_ylabel("Two-sided P value")
ax.grid(color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written to", HERE)

Abstract cites non-significant interaction odds ratios as evidence that correct fluid significantly reduced mortality

Lancet Child Adolesc Health (2018) · doi:10.1016/S2352-4642(18)30130-5 · PMC6004535

Summary

The abstract states that correct fluid prescription "significantly reduced the risk of early mortality (within 2 days) in all subgroups". It supports this with five adjusted odds ratios (AORs): 1·23, 0·95, 1·70, 1·50 and 0·86. These five values, with their CIs, are the "Multiplicative interaction" column of Table 2, which the table footnote defines as OR11/(OR10 × OR01), and they can be recomputed from the table's other cells. They measure how much the fluid association differs between children with and without a clinical sign. They are not the effect of correct fluid in either group. All five 95% CIs include 1 (approximate p = 0.50, 0.87, 0.07, 0.22, 0.58), and three of the five point estimates are above 1. So the statistics the abstract cites do not show a significant reduction in any subgroup. The same table, read for the correct-versus-wrong contrast within each stratum, does point to lower odds of early death with correct fluid in every subgroup. The headline direction may therefore hold, but the estimates, their label and the significance claim in the abstract's key fluid result are wrong as printed. Table 2 also contains two cell errors in its additive-interaction column.

What the paper reports

Abstract, Findings:

Correct fluid prescription significantly reduced the risk of early mortality (within 2 days) in all subgroups: abnormal respiratory signs (AOR 1·23, 0·68–2·24), abnormal circulatory signs (0·95, 0·53–1·73), pallor (1·70, 0·95–3·02), dehydration signs only (1·50, 0·79–2·88), and abnormal neurological signs (0·86, 0·51–1·48).

Abstract, Interpretation:

the prescription of recommended rehydration guidelines reduces risk of death

Research in context, Added value of this study:

Our findings also show that comorbidities are common and that correct fluid prescription is associated with reduced risk of death in these patients.

Methods, Data analysis (third paragraph, on effect modification):

We calculated the relative excess odds due to interaction, the attributable proportion, and the multiplicative interaction odds ratio (OR).

Results (effect-modification paragraph), where the same numbers appear under their correct name:

In fact, positive multiplicative interactions (OR>1) for association of incorrect fluid management with death in children with dehydration signs (OR 1·50, 95% CI 0·79–2·88), respiratory signs (OR 1·23, 0·68–2·24), and pallor (OR 1·70, 0·95–3·02) are evidence of interaction (figure 3).

Table 2, "Interactions of fluid management with risk factors for in-hospital early deaths" (columns: Multiplicative interaction | Fluid wrong and symptom present (OR10) | Fluid right and symptom present (OR11) | Fluid right and symptom absent (OR01) | RERI | Attributable proportion):

Pallor | 1·70 (0·95 to 3·02) | 2·16 (1·66 to 2·82) | 1·07 (0·69 to 1·67) | 0·29 (0·21 to 0·41) | -0·38 (-1·07 to 0·31) | -0·35 (-1·09 to 0·38)

Circulatory | 0·95 (0·53 to 1·73) | 1·92 (1·47 to 2·51) | 0·64 (0·40 to 1·03) | 0·35 (0·25 to 0·50) | -0·63 (-1·19 to -0·06) | -0·98 (-2·06 to 0·11)

Dehydration signs only | 1·50 (0·79 to 2·88) | 0·96 (0·70 to 1·33) | 0·38 (0·26 to 0·55) | 0·26 (0·15 to 0·45) | 0·25 (-0·18 to 0·48) | 0·40 (-0·53 to 1·33)

Neurological | 0·86 (0·51 to 1·48) | 3·57 (2·75 to 4·64) | 1·15 (0·77 to 1·72) | 0·37 (0·25 to 0·54) | -1·80 (-2·71 to -0·88) | -1·56 (-2·61 to -0·52)

Respiratory | 1·23 (0·68 to 2·24) | 3·18 (2·35 to 4·31) | 1·17 (0·80 to 1·73) | 0·30 (0·18 to 0·49) | -1·31 (-2·16 to -9·45) | -1·11 (-1·90 to -0·33)

Table 2 footnote (excerpts):

Interaction exists if RERI is not zero (RERI=OR11 – OR10 – OR01 + 1).

Multiplicative interaction=OR11/(OR10 ×   OR01).

The reference group was fluid wrong and symptom absent (OR00)=1·00.

Discussion, final paragraph:

Importantly, prescription of recommended rehydration guidelines reduces risk of death in this group of children.

What the paper's own numbers show

Method. I compared the five abstract values with each column of Table 2. Using the footnote formulas, I recomputed the multiplicative interaction OR, the RERI and the attributable proportion (AP) from the three cell ORs in each row (OR10, OR11, OR01; OR00 = 1). For each printed 95% CI, I took the standard error of log(OR) as (ln upper − ln lower)/(2 × 1.96) and derived an approximate two-sided Wald p value. Within the "symptom present" stratum, the effect of correct versus wrong fluid is OR11/OR10, because both cells share the reference OR00. Within the "symptom absent" stratum it is OR01. The paper does not report a CI for OR11/OR10. I approximated one by adding the two variances on the log scale and ignoring their covariance. Both estimates are contrasts against the same reference group, so their covariance is usually positive, and in that case the approximate interval is wider than the true one. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Respiratory "AOR" for correct fluid Abstract; Table 2, multiplicative interaction 1·23 (0·68–2·24) Identical to Table 2 interaction OR; OR11/(OR10·OR01) = 1.17/(3.18 × 0.30) = 1.23; CI includes 1; p ≈ 0.50
Circulatory "AOR" Abstract; Table 2 0·95 (0·53–1·73) Identical to interaction OR; 0.64/(1.92 × 0.35) = 0.95; CI includes 1; p ≈ 0.87
Pallor "AOR" Abstract; Table 2 1·70 (0·95–3·02) Identical to interaction OR; 1.07/(2.16 × 0.29) = 1.71; CI includes 1; p ≈ 0.07
Dehydration signs only "AOR" Abstract; Table 2 1·50 (0·79–2·88) Identical to interaction OR; 0.38/(0.96 × 0.26) = 1.52 (rounding of the inputs allows 1.47–1.58); CI includes 1; p ≈ 0.22
Neurological "AOR" Abstract; Table 2 0·86 (0·51–1·48) Identical to interaction OR; 1.15/(3.57 × 0.37) = 0.87 (rounding allows 0.85–0.89); CI includes 1; p ≈ 0.58
Cited CIs that exclude 1 Abstract "significantly reduced … in all subgroups" 0 of 5; 3 of 5 point estimates are above 1
Correct vs wrong fluid, symptom present (OR11/OR10) Derived from Table 2 not reported Respiratory 0.37 (≈0.23–0.60); circulatory 0.33 (≈0.19–0.57); pallor 0.50 (≈0.30–0.83); dehydration only 0.40 (≈0.24–0.65); neurological 0.32 (≈0.20–0.52). Approximate CIs, covariance ignored
Correct vs wrong fluid, symptom absent (OR01) Table 2 0·30, 0·35, 0·29, 0·26, 0·37 Same values; all five printed CIs exclude 1
Identity linking the two Table 2 footnote interaction = OR11/(OR10·OR01) (OR11/OR10) / OR01 equals the interaction OR in every row: the interaction OR is the ratio of the two within-stratum fluid effects
Dehydration signs only, RERI Table 2 0·25 (−0·18 to 0·48) OR11 − OR10 − OR01 + 1 = 0.38 − 0.96 − 0.26 + 1 = 0.16; the printed CI's midpoint is 0.15
Dehydration signs only, AP Table 2 0·40 RERI/OR11 = 0.16/0.38 = 0.42 with the recomputed RERI; 0.25/0.38 = 0.66 with the printed RERI
RERI and AP, other four rows Table 2 e.g. −0·38, −0·63, −1·80, −1·31 Recompute to −0.38, −0.63, −1.79, −1.31; the AP CI midpoints in all four rows, and the RERI CI midpoints for circulatory, pallor and neurological, match the point estimates. The respiratory RERI CI midpoint (−5.81) does not (see next row)
Respiratory RERI CI Table 2 −1·31 (−2·16 to −9·45) Upper limit (−9.45) is below both the point estimate and the lower limit, which is impossible; a symmetric interval around −1.31 would end near −0.46

So the abstract's numbers are the Table 2 interaction ORs. The cell ORs in each row reproduce them, so the column labels are internally consistent. None of the five is significant. The contrast the abstract's sentence describes (correct versus wrong fluid within a subgroup) is a different quantity. Table 2 points to it being below 1 in all five subgroups, in both strata, but the paper does not report it.

Figure

Abstract-cited "AORs" compared with the correct-versus-wrong fluid odds ratio implied by Table 2, by subgroup

Figure 1. One log-scale odds-ratio axis; the vertical line marks OR = 1. Orange circles and lines are the five values the abstract cites as AORs for correct fluid prescription, with their printed 95% CIs. They are identical to Table 2's multiplicative-interaction column. Each crosses 1, and the approximate p value is printed beside it. Blue squares are the odds ratio for correct versus wrong fluid among children with the sign present, recomputed from the same Table 2 row as OR11/OR10. Their intervals are approximate because the covariance of the two estimates is not reported and is ignored here. All five blue estimates lie below 1. The abstract's sentence describes the blue quantity but cites the orange numbers.

RERI cells of Table 2: printed values and CIs compared with the footnote formula

Figure 2. Table 2's RERI column on a linear axis; the vertical line marks 0. Orange: RERI and 95% CI as printed. Blue squares: OR11 − OR10 − OR01 + 1 from the same row. Gray ticks: midpoint of the printed CI. In three rows (circulatory, pallor, neurological) the three agree. For dehydration signs only, the printed point (0.25) differs from both the formula (0.16) and the CI midpoint (0.15). For respiratory signs, the printed point (−1.31) matches the formula, but the printed interval runs from −2.16 to −9.45, so the point lies outside it and the CI midpoint (−5.81) is far from it.

Why it matters

The effect of correct fluid prescription is one of the paper's two stated aims and is named in its title. The abstract's Findings sentence is the only place in the summary that quantifies it, and the Interpretation ("reduces risk of death") and the closing Discussion sentence are the abstract-level and concluding statements of it. The body gives one other quantitative statement: "The predicted probability of early in-hospital death was reduced by 6% when derived from model II with correct fluid prescription compared with when fluid prescription is wrong" (Results, effect-modification paragraph). That is an overall model prediction, not a subgroup estimate, and it is printed without a CI or test, so it does not support "significantly … in all subgroups". A reader of the abstract alone would take the five AORs as subgroup treatment effects. Three are above 1 and none excludes 1, so the reader would see a contradiction or would misread interaction estimates as effects of fluid prescription. Meta-analysts and guideline reviewers who extract the abstract figures would record five non-significant ORs, some pointing to harm, for a result the paper calls a significant reduction. The Results paragraph also calls three non-significant interaction ORs "evidence of interaction". The data do not support that claim on the multiplicative scale. On the additive scale, the RERI CIs for pallor and dehydration signs only include 0, and the respiratory RERI CI is unusable as printed. The paper's own cells point to a consistent protective association of correct fluid. The defect is therefore in which estimates are reported and how they are described, not necessarily in the direction of the conclusion. The magnitude and precision of the subgroup effects cannot be checked from what is published.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. In the abstract Findings, replace the five interaction ORs with estimates of the quantity the sentence describes. One option is the adjusted OR for correct versus wrong fluid within each subgroup (OR11/OR10, with a CI from the fitted model including the covariance term). Another is the overall adjusted OR for correct fluid. Alternatively, keep the interaction ORs but describe them accurately, for example: "There was no evidence that the association of correct fluid prescription with early mortality differed by clinical sign on the multiplicative scale (interaction ORs 0·86–1·70, all 95% CIs including 1)." Remove "significantly" and "in all subgroups" unless subgroup-specific estimates with CIs excluding 1 are reported.
  2. In the Results effect-modification paragraph, revise "are evidence of interaction" for the three interaction ORs whose CIs include 1.
  3. In Table 2, correct the dehydration-signs-only RERI. The footnote formula gives 0.16, consistent with the printed CI midpoint (0.15) and the printed AP (0.40, against 0.42). Correct the respiratory RERI CI, whose printed upper limit (−9·45) cannot be right; a symmetric interval would end near −0.46.
  4. Consider adding a column to Table 2 with the within-stratum correct-versus-wrong fluid OR and its CI for each sign, so readers can check the abstract's claim directly.
Recomputation script (python recompute.py.txt)
"""Recompute the fluid-prescription subgroup results of PMC6004535
(Lancet Child Adolesc Health 2018, doi:10.1016/S2352-4642(18)30130-5).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
  fig1.svg/png         the figure used in writeup.md (same design as candidate 1)
  fig2.svg/png         Table 2 additive-interaction (RERI) cells, reported vs recomputed
  candidate1..5.svg/png  five alternative designs for figure 1, for review

Inputs are copied from the paper as archived in PubMed Central:
  Abstract, Findings: the five "AOR" values cited for correct fluid prescription
  Table 2: multiplicative interaction OR, OR10, OR11, OR01, RERI and attributable
           proportion (AP), each with its 95% CI. Reference group OR00 = 1 (fluid wrong,
           symptom absent). Footnote: RERI = OR11 - OR10 - OR01 + 1; AP = RERI/OR11;
           multiplicative interaction = OR11/(OR10 x OR01).
"""
import itertools
import os

import numpy as np
from scipy.stats import norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (from the paper)
ROWS = ["Respiratory", "Circulatory", "Pallor", "Dehydration signs only", "Neurological"]  # abstract order
# Abstract, Findings: "(AOR 1.23, 0.68-2.24) ..." in this order
ABSTRACT = {
    "Respiratory": (1.23, 0.68, 2.24),
    "Circulatory": (0.95, 0.53, 1.73),
    "Pallor": (1.70, 0.95, 3.02),
    "Dehydration signs only": (1.50, 0.79, 2.88),
    "Neurological": (0.86, 0.51, 1.48),
}
# Table 2 (row label: columns as printed)
T2 = {
    "Pallor": dict(MI=(1.70, 0.95, 3.02), OR10=(2.16, 1.66, 2.82), OR11=(1.07, 0.69, 1.67),
                   OR01=(0.29, 0.21, 0.41), RERI=(-0.38, -1.07, 0.31), AP=(-0.35, -1.09, 0.38)),
    "Circulatory": dict(MI=(0.95, 0.53, 1.73), OR10=(1.92, 1.47, 2.51), OR11=(0.64, 0.40, 1.03),
                        OR01=(0.35, 0.25, 0.50), RERI=(-0.63, -1.19, -0.06), AP=(-0.98, -2.06, 0.11)),
    "Dehydration signs only": dict(MI=(1.50, 0.79, 2.88), OR10=(0.96, 0.70, 1.33), OR11=(0.38, 0.26, 0.55),
                                   OR01=(0.26, 0.15, 0.45), RERI=(0.25, -0.18, 0.48), AP=(0.40, -0.53, 1.33)),
    "Neurological": dict(MI=(0.86, 0.51, 1.48), OR10=(3.57, 2.75, 4.64), OR11=(1.15, 0.77, 1.72),
                         OR01=(0.37, 0.25, 0.54), RERI=(-1.80, -2.71, -0.88), AP=(-1.56, -2.61, -0.52)),
    "Respiratory": dict(MI=(1.23, 0.68, 2.24), OR10=(3.18, 2.35, 4.31), OR11=(1.17, 0.80, 1.73),
                        OR01=(0.30, 0.18, 0.49), RERI=(-1.31, -2.16, -9.45), AP=(-1.11, -1.90, -0.33)),
}
SHORT = {"Respiratory": "Respiratory", "Circulatory": "Circulatory", "Pallor": "Pallor",
         "Dehydration signs only": "Dehydration only", "Neurological": "Neurological"}


def se_log(c):
    """SE of log(OR) implied by a symmetric-on-log 95% CI."""
    return (np.log(c[2]) - np.log(c[1])) / (2 * Z)


def p_from_ci(c):
    return 2 * norm.sf(abs(np.log(c[0]) / se_log(c)))


def f2(x):
    return f"{x:.2f}"


# ---------------------------------------------------------------- 1. abstract = Table 2 interaction column
print("1) Abstract 'AOR' values vs Table 2 'Multiplicative interaction' column")
P_ABS = {}
for k in ROWS:
    a, t = ABSTRACT[k], T2[k]["MI"]
    P_ABS[k] = p_from_ci(a)
    print(f"   {k:23s} abstract {a}  Table2 MI {t}  identical={a == t}  "
          f"CI includes 1={a[1] <= 1 <= a[2]}  point>1={a[0] > 1}  approx two-sided p={P_ABS[k]:.2f}")
n_excl = sum(not (ABSTRACT[k][1] <= 1 <= ABSTRACT[k][2]) for k in ROWS)
n_above = sum(ABSTRACT[k][0] > 1 for k in ROWS)
print(f"   CIs excluding 1: {n_excl} of 5; point estimates above 1: {n_above} of 5")
print("   geometric midpoint of each CI (check that CIs are symmetric on the log scale):")
for k in ROWS:
    a = ABSTRACT[k]
    print(f"     {k:23s} sqrt(lo*hi)={np.sqrt(a[1] * a[2]):.3f}  vs point {a[0]}")

# ---------------------------------------------------------------- 2. recompute interaction measures
print("\n2) Interaction measures recomputed from Table 2 component ORs (footnote formulas)")
REC = {}
for k in ROWS:
    d = T2[k]
    o10, o11, o01 = d["OR10"][0], d["OR11"][0], d["OR01"][0]
    mi = o11 / (o10 * o01)
    # range allowed by 2-decimal rounding of the three inputs
    grid = [(o11 + e1) / ((o10 + e2) * (o01 + e3)) for e1, e2, e3 in itertools.product((-0.005, 0.005), repeat=3)]
    reri = o11 - o10 - o01 + 1
    ap = reri / o11
    REC[k] = dict(MI=mi, MI_lo=min(grid), MI_hi=max(grid), RERI=reri, AP=ap, AP_repRERI=d["RERI"][0] / o11)
    print(f"   {k:23s} MI reported {f2(d['MI'][0])} recomputed {mi:.3f} (rounding range {min(grid):.3f}-{max(grid):.3f})"
          f" | RERI reported {f2(d['RERI'][0])} recomputed {reri:.2f}"
          f" | AP reported {f2(d['AP'][0])}; RERI_recomputed/OR11={ap:.2f}; RERI_reported/OR11={REC[k]['AP_repRERI']:.2f}")

print("\n   CI midpoints (RERI and AP CIs are symmetric on the linear scale):")
for k in ROWS:
    d = T2[k]
    mr = (d["RERI"][1] + d["RERI"][2]) / 2
    ma = (d["AP"][1] + d["AP"][2]) / 2
    print(f"     {k:23s} RERI CI midpoint {mr:.3f} (reported point {f2(d['RERI'][0])}) | AP CI midpoint {ma:.3f} (reported {f2(d['AP'][0])})")
r = T2["Respiratory"]["RERI"]
sym_upper = 2 * r[0] - r[1]
print(f"   Respiratory RERI CI printed ({r[1]} to {r[2]}): upper < lower = {r[2] < r[1]}; "
      f"upper < point = {r[2] < r[0]}; a symmetric interval around {r[0]} would end near {sym_upper:.2f}")

# ---------------------------------------------------------------- 3. effect of correct vs wrong fluid within strata
print("\n3) Correct vs wrong fluid within each stratum (implied by Table 2; same model per row)")
print("   symptom present: OR11/OR10; approximate CI ignores the covariance of the two log-ORs")
print("   symptom absent : OR01 (contrast with OR00 = 1), CI as printed")
WITHIN = {}
for k in ROWS:
    d = T2[k]
    ratio = d["OR11"][0] / d["OR10"][0]
    s = np.sqrt(se_log(d["OR11"]) ** 2 + se_log(d["OR10"]) ** 2)
    lo, hi = ratio * np.exp(-Z * s), ratio * np.exp(Z * s)
    p = 2 * norm.sf(abs(np.log(ratio) / s))
    WITHIN[k] = (ratio, lo, hi, p)
    o01 = d["OR01"]
    print(f"   {k:23s} present: {ratio:.2f} (approx {lo:.2f}-{hi:.2f}, p{"<0.001" if p < 0.001 else "~" + format(p, ".3f")})"
          f" | absent: OR01 {f2(o01[0])} ({f2(o01[1])}-{f2(o01[2])}), CI excludes 1={o01[2] < 1}")
    # identity: interaction OR = (present-stratum effect) / (absent-stratum effect)
    print(f"   {'':23s} check: (OR11/OR10)/OR01 = {ratio / o01[0]:.3f} = interaction OR {REC[k]['MI']:.3f}")
rng = [WITHIN[k][0] for k in ROWS]
print(f"   range OR11/OR10: {min(rng):.2f}-{max(rng):.2f}; range OR01: "
      f"{min(T2[k]['OR01'][0] for k in ROWS):.2f}-{max(T2[k]['OR01'][0] for k in ROWS):.2f}")

# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
    "font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
TICKS = [0.1, 0.2, 0.5, 1, 2, 4]


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
    plt.close(fig)


def log_axis(ax, lo=0.1, hi=4.6):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    tk = [t for t in TICKS if lo <= t <= hi]
    ax.set_xticks(tk)
    ax.set_xticklabels([str(t) for t in tk])
    ax.set_xlim(lo, hi)
    ax.minorticks_off()
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.axvline(1, color=INK2, lw=0.9, zorder=1)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


# ---------------------------------------------------------------- candidate 1: two-series forest plot
def cand1(name):
    fig, ax = plt.subplots(figsize=(W, 4.0))
    y = np.arange(len(ROWS))[::-1].astype(float)
    off = 0.17
    for yi, k in zip(y, ROWS):
        a = ABSTRACT[k]
        ax.plot([a[1], a[2]], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
        ax.plot(a[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.text(4.5, yi + off, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}), p≈{P_ABS[k]:.2f}",
                va="center", ha="left", fontsize=10, color=INK)
        w = WITHIN[k]
        ax.plot([w[1], w[2]], [yi - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
        ax.plot(w[0], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
        ax.text(4.5, yi - off, f"{f2(w[0])} (≈{f2(w[1])}–{f2(w[2])})", va="center", ha="left",
                fontsize=10, color=INK2)
    log_axis(ax, 0.17, 4.4)
    ax.set_yticks(y)
    ax.set_yticklabels([SHORT[k] for k in ROWS], fontsize=10.5, color=INK)
    ax.set_ylim(-0.7, len(ROWS) - 0.3)
    ax.set_xlabel("Odds ratio for early death (log scale)", color=INK2)
    leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported: 'AOR' cited in the abstract for correct fluid\n(= Table 2 multiplicative-interaction OR)"),
           Line2D([], [], color=BLUE, marker="s", ms=8, mec=BG, label="Recomputed: correct vs wrong fluid, symptom present\n(OR11/OR10 from Table 2; approximate CI)")]
    ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, -0.17), ncol=2, frameon=False, fontsize=9.6 if False else 10,
              handlelength=1.6, columnspacing=1.2)
    ax.set_title("Abstract: “significantly reduced the risk of early mortality … in all subgroups”",
                 loc="left", fontsize=11, color=INK, pad=10)
    save(fig, name)


# ---------------------------------------------------------------- candidate 2: abstract vs Table 2 text panels
def cand2(name):
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(-6, 54)
    ax.axis("off")

    def panel(x0, w, title, color):
        ax.add_patch(FancyBboxPatch((x0, 4), w, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
                                    fc="#ffffff", ec=color, lw=2))
        ax.text(x0 + 2, 49, title, fontsize=11, weight="bold", va="top", color=INK)

    panel(1, 43, "Abstract, Findings", ORANGE)
    abs_txt = ("“Correct fluid prescription significantly\n reduced the risk of early mortality\n"
               " (within 2 days) in all subgroups:”")
    ax.text(3, 43.5, abs_txt, fontsize=10, va="top", color=INK, style="italic")
    yy = 31
    for k in ROWS:
        a = ABSTRACT[k]
        ax.text(4, yy, SHORT[k], fontsize=10, va="center", color=INK2)
        ax.text(42, yy, f"AOR {f2(a[0])} ({f2(a[1])}–{f2(a[2])})", fontsize=10, va="center", ha="right", color=INK)
        yy -= 5
    ax.text(3, 6.5, "values as printed (reported)", fontsize=10, color=INK2)

    panel(56, 43, "Table 2, “Multiplicative interaction”", BLUE)
    ax.text(58, 43.5, "Footnote: interaction = OR11/(OR10 × OR01)\nRecomputed from the table's own cells:",
            fontsize=10, va="top", color=INK)
    yy = 31
    for k in ROWS:
        a = T2[k]["MI"]
        ax.text(58, yy, SHORT[k], fontsize=10, va="center", color=INK2)
        ax.text(97, yy, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}) → {REC[k]['MI']:.2f}",
                fontsize=10, va="center", ha="right", color=INK)
        yy -= 5
    ax.text(58, 6.5, "printed (CI) → recomputed from cells", fontsize=10, color=INK2)

    ax.add_patch(FancyArrowPatch((44.8, 21), (55.2, 21), arrowstyle="<|-|>", mutation_scale=14, color=INK2, lw=2))
    ax.text(50, 23.5, "same\nfive\nnumbers", ha="center", va="bottom", fontsize=10, color=INK)
    ax.text(50, -1.5, "The abstract cites the interaction ORs as “AOR” for correct fluid. "
            "They are ratios of two effects, not effects of correct fluid; 0 of 5 CIs exclude 1.",
            ha="center", va="center", fontsize=10, color=INK, wrap=True)
    save(fig, name)


# ---------------------------------------------------------------- candidate 3: calculation flow grid
def cand3(name):
    fig = plt.figure(figsize=(W, 3.9))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(0, 52)
    ax.axis("off")
    cols = [(20, "OR10\nwrong fluid,\nsign present"), (33, "OR11\ncorrect fluid,\nsign present"),
            (46, "OR01\ncorrect fluid,\nsign absent"), (66, "OR11/(OR10·OR01)\nrecomputed ≈\nabstract 'AOR'"),
            (86, "OR11/OR10\ncorrect vs wrong,\nsign present")]
    for x, lab in cols:
        ax.text(x, 49, lab, ha="center", va="top", fontsize=10, color=INK, linespacing=1.15)
    ax.text(33, 51.5, "Table 2, as printed (reported)", ha="center", fontsize=10, color=ORANGE, weight="bold")
    ax.text(75, 51.5, "computed from those cells (recomputed)", ha="center", fontsize=10, color=BLUE, weight="bold")
    ax.plot([13, 52], [37.5, 37.5], color=ORANGE, lw=2)
    ax.plot([55, 97], [37.5, 37.5], color=BLUE, lw=2)
    yy = 32
    for k in ROWS:
        d = T2[k]
        ax.text(1, yy, SHORT[k], va="center", fontsize=10, color=INK)
        for x, key in ((20, "OR10"), (33, "OR11"), (46, "OR01")):
            ax.text(x, yy, f2(d[key][0]), ha="center", va="center", fontsize=10.5, color=INK)
        ax.annotate("", xy=(54.5, yy), xytext=(50.5, yy), arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.2))
        ax.text(66, yy, f"{REC[k]['MI']:.2f}  (CI {f2(d['MI'][1])}–{f2(d['MI'][2])})", ha="center", va="center", fontsize=10.5, color=INK)
        ax.text(86, yy, f"{WITHIN[k][0]:.2f}", ha="center", va="center", fontsize=10.5, color=INK, weight="bold")
        yy -= 5.6
    ax.text(65, 2.5, "all 5 CIs include 1", ha="center", fontsize=10, color=INK2)
    ax.text(88, 2.5, "all 5 below 1", ha="center", fontsize=10, color=INK2)
    save(fig, name)


# ---------------------------------------------------------------- candidate 4: interaction plot, small multiples
def cand4(name):
    fig, axes = plt.subplots(1, 5, figsize=(W, 3.5), sharey=True)
    for ax, k in zip(axes, ROWS):
        d = T2[k]
        ax.plot([0, 1], [1, d["OR01"][0]], color=ORANGE, lw=2, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2)
        ax.plot([0, 1], [d["OR10"][0], d["OR11"][0]], color=ORANGE, lw=2, marker="o", ms=8, mec=ORANGE, mew=1.2)
        ax.set_yscale("log")
        ax.set_ylim(0.13, 5)
        ax.set_yticks([0.25, 0.5, 1, 2, 4])
        ax.set_yticklabels(["0.25", "0.5", "1", "2", "4"])
        ax.minorticks_off()
        ax.axhline(1, color=INK2, lw=0.9, zorder=0)
        ax.grid(axis="y", color=GRID, lw=0.8)
        ax.set_xlim(-0.35, 1.35)
        ax.set_xticks([0, 1])
        ax.set_xticklabels(["wrong", "correct"], fontsize=10)
        for s in ("top", "right"):
            ax.spines[s].set_visible(False)
        ax.set_title(f"{SHORT[k].replace(' only', '')}\ninteraction {f2(d['MI'][0])}", fontsize=10, color=INK)
        ax.text(0.5, 0.03, f"×{WITHIN[k][0]:.2f} | ×{d['OR01'][0]:.2f}", transform=ax.transAxes,
                ha="center", fontsize=10, color=BLUE)
    axes[0].set_ylabel("OR for early death vs\nwrong fluid, sign absent (log)", color=INK2)
    fig.supxlabel("Fluid prescription at admission", fontsize=10, color=INK2, y=0.06)
    leg = [Line2D([], [], color=ORANGE, lw=2.5, marker="o", ms=8, mec=ORANGE, label="Reported, sign present (OR10 → OR11): solid line, filled dots"),
           Line2D([], [], color=ORANGE, lw=2.5, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2, label="Reported, sign absent (1 → OR01): dashed line, open dots"),
           Line2D([], [], color=BLUE, lw=0, marker="$×$", ms=9, label="Recomputed change with correct fluid: present | absent")]
    fig.legend(handles=leg, loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, -0.2), handlelength=4)
    fig.tight_layout(rect=(0, 0.04, 1, 1))
    save(fig, name)


# ---------------------------------------------------------------- candidate 5: cited values only, with p
def cand5(name):
    fig, ax = plt.subplots(figsize=(W, 3.2))
    y = np.arange(len(ROWS))[::-1].astype(float)
    ax.axvspan(0.3, 1, color="#f1f0ec", zorder=0)
    ax.text(0.31, len(ROWS) - 0.3, "OR < 1 (shaded)",
            ha="left", va="center", fontsize=10, color=INK2)
    for yi, k in zip(y, ROWS):
        a = ABSTRACT[k]
        ax.plot([a[1], a[2]], [yi, yi], color=ORANGE, lw=2, solid_capstyle="round")
        ax.plot(a[0], yi, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.text(3.6, yi, f"p ≈ {P_ABS[k]:.2f}", va="center", fontsize=10.5, color=BLUE)
    log_axis(ax, 0.3, 3.4)
    ax.set_xticks([0.3, 0.5, 1, 2, 3])
    ax.set_xticklabels(["0.3", "0.5", "1", "2", "3"])
    ax.set_yticks(y)
    ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
    ax.set_ylim(-0.6, len(ROWS) + 0.05)
    ax.set_xlabel("'AOR' (95% CI) cited in the abstract, log scale", color=INK2)
    leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported in the abstract"),
           Line2D([], [], color=BLUE, lw=0, marker="$p$", ms=9, label="Recomputed two-sided p from the printed CI")]
    ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
    save(fig, name)


# ---------------------------------------------------------------- figure 2: RERI cells
def fig2(name):
    fig, ax = plt.subplots(figsize=(W, 3.4))
    y = np.arange(len(ROWS))[::-1].astype(float)
    off = 0.16
    for yi, k in zip(y, ROWS):
        d = T2[k]["RERI"]
        lo, hi = sorted((d[1], d[2]))
        ax.plot([lo, hi], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
        ax.plot(d[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.plot(REC[k]["RERI"], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
        mid = (d[1] + d[2]) / 2
        ax.plot(mid, yi + off, "|", ms=12, mew=2, color=INK2, zorder=4)
    k = "Dehydration signs only"
    yd = y[ROWS.index(k)]
    ax.annotate(f"printed {f2(T2[k]['RERI'][0])}; cells give {REC[k]['RERI']:.2f};\nCI midpoint {(T2[k]['RERI'][1] + T2[k]['RERI'][2]) / 2:.2f}",
                xy=(0.5, yd), xytext=(1.2, yd), va="center", fontsize=10, color=INK,
                arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
    k = "Respiratory"
    yr = y[ROWS.index(k)]
    ax.annotate("printed CI “−2·16 to −9·45”:\nupper limit below point and lower limit",
                xy=(-2.16, yr + off), xytext=(-7.5, yr - 0.62), va="center", fontsize=10, color=INK,
                arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
    ax.axvline(0, color=INK2, lw=0.9)
    ax.set_xlim(-10, 3.2)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_yticks(y)
    ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
    ax.set_xlabel("RERI (relative excess risk due to interaction), linear scale; null = 0", color=INK2)
    leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported RERI (95% CI as printed)"),
           Line2D([], [], color=BLUE, lw=0, marker="s", ms=8, mec=BG, label="Recomputed OR11 − OR10 − OR01 + 1"),
           Line2D([], [], color=INK2, lw=0, marker="|", ms=12, mew=2, label="Midpoint of printed CI")]
    ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=3, frameon=False, fontsize=10,
              columnspacing=1.0, handletextpad=0.4)
    save(fig, name)


for fn, nm in ((cand1, "candidate1"), (cand2, "candidate2"), (cand3, "candidate3"),
               (cand4, "candidate4"), (cand5, "candidate5"), (cand1, "fig1"), (fig2, "fig2")):
    fn(nm)
print("\nFigures written to", HERE)

Several adjusted odds ratios in the abstract have confidence intervals narrower than the data allow

Lancet Global Health (2025) · doi:10.1016/S2214-109X(25)00061-0 · PMC12208783

Summary

The Abstract, Results and Table 2 report adjusted odds ratios (aORs) for catastrophic health expenditure (CHE; 76 cases among 296 participants) of 2·0 (95% CI 1·9–2·1, p<0·0001) for the second vs first poverty quintile (60 participants each) and 9·7 (9·6–9·8) for subcounty hospitals vs health centres (69 vs 164 participants; 9·8 (9·7–9·9) in Table 2). For a 0/1 covariate held by this many participants, no model-based maximum-likelihood logistic regression can give a standard error below a floor set by the group sizes, and the standard errors implied by the two reported intervals are 11 and 52 times smaller than that floor; three further abstract aORs (female sex, education above secondary school, county hospital) are also below it, by smaller margins. With the standard error implied by the Table 1 counts, the second-quintile aOR becomes 2.0 (0.86–4.67, p = 0.11), close to the paper's own five-facility sensitivity estimate of 2.1 (0.70–6.1) in Supplementary Table 4. The second quintile is the only poverty quintile reported with significantly raised adjusted odds, and the abstract's conclusion that CHE is "driven by socioeconomic … inequalities" leans on it; the facility-level associations keep their direction and significance, but their stated precision is overstated about 50-fold.

What the paper reports

CHE was associated with being aged 30–44 years old (aOR 2·6, 95% CI 1·3–5·3, p=0·010), being female (1·8, 1·3–2·7, p=0·0021), having higher than secondary school education (1·6, 1·1–2·3, p=0·0083), being accompanied during health-care-seeking visits (3·2, 1·7–5·9, p<0·0001), belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty hospitals (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001).

Abstract, Findings

…belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001; table 2).

Results, paragraph on CHE incidence

Table 2 ("Univariable and multivariable logistic regression of health and social factors associated with catastrophic health expenditure"). Its columns are the unadjusted OR (n=296), p, the adjusted OR (n=296) and p. Two of its rows read:

Second quintile | 1·4 (0·8–2·4) | 0·30 | 2·0 (1·9–2·1) | <0·0001

Subcounty hospital | 9·7 (8·4–11·1) | <0·0001 | 9·8 (9·7–9·9) | <0·0001

Table 2

Table 1 gives the counts behind these rows. The first quintile has 13 CHE and 47 no CHE, the second quintile 15 and 45, health centres 11 and 153, subcounty hospitals 27 and 42, and the county hospital 38 and 25.

How the paper uses these results:

…driven by socioeconomic and sex inequalities and impaired access to health care and social protection.

Abstract, Interpretation

Independent drivers of CHE included being female, poverty, being accompanied by a carer during health-care seeking, and seeking care at subcounty or county hospitals.

Research in context, Added value of this study

Our findings showed that poorer quintiles were more likely to incur CHE than the richest quintile, although the relationship across poverty quintiles did not always reach significance and appeared non-linear.

Discussion, paragraph 3

On the analysis, the paper says "Univariable and multivariable logistic regression models generated unadjusted and adjusted odds ratios", that "for the 21 participants without household asset data, multiple imputation with chained equations was also used to impute poverty scores", and "Data were analysed in STATA (version 15) and R (version 4)." (Methods, Statistical analysis). The paper does not describe a clustered, robust or otherwise non-standard variance estimator.

What the paper's own numbers show

Method. For a logistic regression coefficient on a 0/1 indicator held by n₁ of N participants, the model-based variance is the corresponding diagonal element of the inverse Fisher information. For any positive-definite matrix, a diagonal element of the inverse is at least the corresponding element of the inverse of any principal submatrix that contains it, so [I⁻¹]ⱼⱼ ≥ [(I_sub)⁻¹]ⱼⱼ, where I_sub is the intercept-and-indicator block of the same information matrix, evaluated at the full model's fitted values p. That block's inverse gives 1/S₁ + 1/S₀, where S₁ and S₀ sum p(1 − p) over participants with and without the indicator. Since p(1 − p) ≤ 1/4 always, SE ≥ √(4/n₁ + 4/(N − n₁)) for any covariate set and any fitted values. Pooling multiply imputed fits by Rubin's rules can only add variance. Separately, I derived each reported interval's implied SE, ln(upper/lower)/(2 × 1.96), including the full range allowed by rounding the printed limits to one decimal place. I computed crude ORs with Wald intervals from the Table 1 counts. All numbers come from recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
aOR, second vs first poverty quintile Abstract; Results; Table 2 2·0 (1·9–2·1), p<0·0001 Implied SE 0.0255 (0.013–0.038 allowing for rounding), z = 27. Minimum possible SE with n₁ = 60: 0.289, which is 11× the implied SE and 7.5× the widest SE rounding allows. Narrowest possible 95% CI at aOR 2.0: 1.13–3.53 (p = 0.017)
Same, allowing for imputation of quintile in 21 participants derived — If n₁ is anywhere from 39 to 81, the minimum SE is still 0.261, 10× the implied SE
Crude OR, second vs first quintile (15/60 vs 13/60) Table 1 Table 2 unadjusted: 1·4 (0·8–2·4), p=0·30 1.21 (0.52–2.81), SE 0.433, p = 0.67 (Fisher exact p = 0.83). Crude SE ÷ reported adjusted SE = 17 (11–34 allowing for rounding)
aOR 2.0 with the count-based SE derived — 95% CI 0.86–4.67, p = 0.11
Second quintile, same outcome and covariates with a five-level facility term Supplementary Table 4 2.1 (0.70–6.1), p=0.099 Implied SE 0.552, 22× the Table 2 SE. The interval includes 1 (the Wald p from this interval is 0.18)
aOR, subcounty hospital vs health centre Abstract; Results 9·7 (9·6–9·8), p<0·0001 Implied SE 0.0053. Minimum possible SE with n₁ = 69: 0.275, 52× larger (35× the widest SE rounding allows). Narrowest possible CI at 9.7: 5.66–16.63
aOR, subcounty hospital Table 2 9·8 (9·7–9·9) Differs from the Abstract and Results (9·7, 9·6–9·8). Same problem: minimum possible CI at 9.8 is 5.72–16.80
Crude OR, subcounty vs health centre (27/69 vs 11/164) Table 1 Table 2 unadjusted: 9·7 (8·4–11·1) 8.94 (4.10–19.50), SE 0.398, which is 76× the SE implied by the abstract's adjusted interval. aOR 9.7 with this SE: 4.4–21.2
aOR, female Abstract; Table 2 1·8 (1·3–2·7), p=0·0021 Implied SE 0.186 (at most 0.201 allowing for rounding) vs minimum 0.236. Narrowest possible CI 1.13–2.86, p = 0.013
aOR, above secondary education Abstract; Table 2 1·6 (1·1–2·3), p=0·0083 Implied SE 0.188 (at most 0.206) vs minimum 0.344. Narrowest possible CI 0.82–3.14, p = 0.17. Supplementary Table 4 gives 1.4 (0.50–4.2), p = 0.50
aOR, county hospital Abstract; Table 2 25·1 (15·7–40·2) Implied SE 0.240 (at most 0.241) vs minimum 0.284. Narrowest possible CI 14.4–43.8. Still strongly significant
aOR, third quintile Table 2 0·7 (0·6–0·9), p=0·0009 Implied SE 0.103 (at most 0.139) vs minimum 0.293. Narrowest possible CI 0.39–1.24. Supplementary Table 4 gives 0.73 (0.22–2.4), p = 0.60
aORs for age 30–44, accompanied, fourth and fifth quintile, NHIF Abstract; Table 2 e.g. 2·6 (1·3–5·3); 3·2 (1·7–5·9) Consistent with the minimum (implied SE ≥ minimum)
Unadjusted secondary vs primary education (23/94 vs 42/163) Table 1; Table 2 1·3 (1·2–1·5), p<0·0001 Crude 0.93 (0.52–1.68), p = 0.82. Table 1 gives p = 0·90 for education
Unadjusted age 30–44 vs 18–29 (39/117 vs 14/103) Table 1; Table 2 3·7 (2·9–4·6) Crude 3.18 (1.61–6.29), SE 0.348 vs implied 0.118
Unadjusted county vs health centre (38/63 vs 11/164) Table 1; Table 2 24 (19·9–28·6) Crude 21.1 (9.56–46.7), SE 0.405 vs implied 0.093

In the Abstract, 5 of the 7 adjusted associations have reported intervals narrower than any maximum-likelihood logistic fit to these data can produce. The violation is severe for the second quintile (11×) and subcounty hospital (52×). It is moderate for education above secondary school (1.8×). It is small for female sex (1.3×) and county hospital (1.2×). The bound is conservative: it assumes every fitted probability is 0.5, whereas the overall CHE rate is 26%. Realistic standard errors are larger, as the crude SEs and Supplementary Table 4 show.

The univariable column of Table 2 has the same pattern. Several of its intervals are much narrower than the crude counts give, and the secondary-education OR points the opposite way from the counts. That column appears unchanged in Supplementary Table 4, for example 3.7 (2.9–4.6), 1.3 (1.2–1.5) and 1.4 (0.76–2.4).

Figure

Second-quintile and subcounty odds ratios: reported intervals vs the intervals the paper's counts allow

Figure 1. Odds ratios of catastrophic health expenditure with 95% CIs on a log scale. The thin vertical line marks OR = 1. Orange shows values the paper reports. For the second poverty quintile these are the main-model aOR 2.0 (1.9–2.1) from the Abstract, Results and Table 2 (diamond), and the same model with a five-level facility term from Supplementary Table 4 (square). For subcounty hospitals they are the Abstract and Results value 9.7 (9.6–9.8) and the Table 2 value 9.8 (9.7–9.9). Blue shows values recomputed from the paper's own numbers. The first blue row in each group is the narrowest interval any maximum-likelihood logistic fit could give at the reported point estimate. The second keeps the reported point estimate but uses the standard error of the crude log OR from the Table 1 counts. The third is the crude OR and Wald interval from those counts. The reported main-model intervals are hidden behind their markers. For the second quintile, the interval from the paper's own sensitivity model, the interval with the count-based SE (0.86–4.67) and the crude interval all include 1.

Implied standard errors of the seven abstract aORs vs the minimum attainable

Figure 2. For each of the seven adjusted odds ratios in the Abstract (point estimate in brackets), the orange diamond is the standard error of the log OR implied by the reported 95% CI. Its bar is the range allowed by rounding the printed limits. The blue dot is the smallest standard error any maximum-likelihood logistic regression can give for that indicator, √(4/n₁ + 4/(296 − n₁)), with n₁ from Table 1. The shaded band to its right is the attainable range. Five of the seven reported intervals fall below this floor. The second-poverty-quintile and subcounty-hospital estimates are 11 and 52 times below it. The axis is logarithmic.

Why it matters

Explanations considered

Status

Suggested correction

  1. Re-estimate Table 2, both columns, with model-based maximum-likelihood standard errors, or state and justify the variance estimator used. With five facilities, a cluster-robust estimator is not appropriate without small-cluster corrections. Report the method in the Statistical analysis section.
  2. Replace the second-quintile, subcounty-hospital, female, above-secondary, county-hospital and third-quintile aOR intervals and p values in the Abstract, Results and Table 2. Also replace the univariable intervals that do not match the Table 1 counts.
  3. Reconcile the subcounty-hospital aOR between the Abstract and Results (9·7, 9·6–9·8) and Table 2 (9·8, 9·7–9·9).
  4. If the corrected second-quintile and above-secondary-education intervals include 1, revise the statements that poverty is an independent driver of CHE (Abstract Interpretation, Research in context, Discussion) and the abstract's list of associations accordingly.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12208783 write-up and regenerate fig1/fig2.

Run:  python recompute.py.txt   (needs numpy, scipy, matplotlib)
All inputs are copied from the article (Table 1, Table 2, Abstract) or from its
Supplementary appendix 1 (mmc1.pdf: Supplementary Tables 1b and 4).
"""
import os
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
N, CHE = 296, 76


def crude(a, b, c, d):
    """a = CHE exposed, b = no CHE exposed, c = CHE reference, d = no CHE reference."""
    OR = (a / b) / (c / d)
    se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    lo, hi = np.exp(np.log(OR) - Z * se), np.exp(np.log(OR) + Z * se)
    p = 2 * stats.norm.sf(abs(np.log(OR)) / se)
    return OR, lo, hi, se, p


def se_from_ci(lo, hi):
    return np.log(hi / lo) / (2 * Z)


def se_rounding_range(lo, hi, h=0.05):
    """(narrowest, widest) SE consistent with limits printed to one decimal place."""
    return se_from_ci(lo + h, hi - h), se_from_ci(lo - h, hi + h)


def se_min(n1, n=N):
    """Hard lower bound on the model-based (ML) SE of a logistic coefficient for a
    0/1 indicator held by n1 of n participants, whatever the other covariates:
    Var(b_j) = [I^-1]_jj >= [(I restricted to intercept and x_j)^-1]_jj = 1/S1 + 1/S0,
    with S1 = sum over x_j = 1 of p(1-p) <= n1/4 and S0 <= (n - n1)/4."""
    return np.sqrt(4 / n1 + 4 / (n - n1))


def ci(OR, se):
    return OR * np.exp(-Z * se), OR * np.exp(Z * se)


def p_of(OR, se):
    return 2 * stats.norm.sf(abs(np.log(OR)) / se)


print("Totals from Table 1: CHE", CHE, "of", N, f"({CHE / N:.1%})")

# ---------------------------------------------------------------------------
print("\n=== 1. Crude ORs from Table 1 counts vs Table 2 'unadjusted' column ===")
rows = [
    ("Age 30-44 vs 18-29", 39, 78, 14, 89, "3.7 (2.9-4.6), p=0.0010"),
    ("Female vs male", 42, 131, 34, 89, "0.7 (0.4-2.0), p=0.71"),
    ("Secondary vs primary", 23, 71, 42, 121, "1.3 (1.2-1.5), p<0.0001"),
    ("Above secondary vs primary", 11, 28, 42, 121, "1.7 (1.0-2.8), p=0.053"),
    ("Quintile 2 vs 1", 15, 45, 13, 47, "1.4 (0.8-2.4), p=0.30"),
    ("Quintile 3 vs 1", 10, 48, 13, 47, "1.0 (0.8-1.3), p=0.91"),
    ("Quintile 5 vs 1", 24, 35, 13, 47, "2.2 (1.1-4.7), p=0.035"),
    ("Subcounty vs health centre", 27, 42, 11, 153, "9.7 (8.4-11.1), p<0.0001"),
    ("County vs health centre", 38, 25, 11, 153, "24 (19.9-28.6), p<0.0001"),
]
CR = {}
for lab, a, b, c, d, rep in rows:
    OR, lo, hi, se, p = crude(a, b, c, d)
    CR[lab] = (OR, lo, hi, se, p)
    print(f"{lab:28s} crude {OR:5.2f} ({lo:5.2f}-{hi:5.2f}) SE {se:.3f} p={p:.2g}   | Table 2: {rep}")
print("Fisher exact p, quintile 2 vs 1:", f"{stats.fisher_exact([[15, 45], [13, 47]])[1]:.2f}")
print("Fisher exact p, secondary vs primary:", f"{stats.fisher_exact([[23, 71], [42, 121]])[1]:.2f}")

# ---------------------------------------------------------------------------
print("\n=== 2. Reported adjusted ORs vs the hard lower bound on any ML standard error ===")
# label, n1 (participants in the category, Table 1), aOR, lo, hi, where, in_abstract
adj = [
    ("Age 30-44", 117, 2.6, 1.3, 5.3, True),
    ("Female", 173, 1.8, 1.3, 2.7, True),
    ("Above secondary", 39, 1.6, 1.1, 2.3, True),
    ("Accompanied", 59, 3.2, 1.7, 5.9, True),
    ("Second quintile", 60, 2.0, 1.9, 2.1, True),
    ("Subcounty (abstract/text)", 69, 9.7, 9.6, 9.8, True),
    ("County", 63, 25.1, 15.7, 40.2, True),
    ("Subcounty (Table 2)", 69, 9.8, 9.7, 9.9, False),
    ("Third quintile", 58, 0.7, 0.6, 0.9, False),
    ("Fourth quintile", 59, 0.5, 0.2, 1.2, False),
    ("Fifth quintile", 59, 2.1, 0.6, 7.7, False),
    ("NHIF no", 212, 1.5, 0.7, 3.5, False),
]
print(f"{'term':26s} {'reported':>18s} {'SE_rep':>7s} {'SE range (rounding)':>20s} {'SE_min':>7s} {'SE_min/SE_rep':>13s}  min CI at reported aOR   verdict")
FIG2 = []
for lab, n1, OR, lo, hi, abst in adj:
    se = se_from_ci(lo, hi)
    s_lo, s_hi = se_rounding_range(lo, hi)
    smin = se_min(n1)
    verdict = "BELOW BOUND" if s_hi < smin else "consistent"
    clo, chi = ci(OR, smin)
    print(f"{lab:26s} {OR:5.1f} ({lo:4.1f}-{hi:4.1f})  {se:7.4f} {s_lo:9.4f}-{s_hi:.4f} {smin:7.3f} {smin / se:13.1f}  {clo:6.2f}-{chi:6.2f}   {verdict}")
    if abst:
        FIG2.append((lab, OR, lo, hi, se, s_lo, s_hi, smin))
n_abs_fail = sum(1 for r in FIG2 if r[6] < r[7])
print(f"Abstract aORs whose reported CI is narrower than any ML fit allows: {n_abs_fail} of {len(FIG2)}")
for _lab, _n1, _lo, _hi in [("Second quintile", 60, 1.9, 2.1), ("Subcounty (abstract/text)", 69, 9.6, 9.8)]:
    _r = se_min(_n1) / se_from_ci(_lo, _hi)
    print(f"Inverse-variance weight inflation vs the floor, {_lab}: ({_r:.1f})^2 = {_r ** 2:.0f}x")
print("p-value if each abstract aOR kept its point estimate but had the minimum possible SE:")
for lab, OR, lo, hi, se, s_lo, s_hi, smin in FIG2:
    print(f"   {lab:26s} reported-CI p {p_of(OR, se):.2g}   p at minimum SE {p_of(OR, smin):.2g}")

# ---------------------------------------------------------------------------
print("\n=== 3. Second quintile: the reported aOR 2.0 (1.9-2.1) in context ===")
se_rep = se_from_ci(1.9, 2.1)
s_lo, s_hi = se_rounding_range(1.9, 2.1)
print(f"implied SE {se_rep:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); implied z = {np.log(2.0) / se_rep:.1f}")
crude_q2 = CR["Quintile 2 vs 1"]
print(f"crude SE {crude_q2[3]:.3f}; ratio crude/reported {crude_q2[3] / se_rep:.0f} (rounding range {crude_q2[3] / s_hi:.0f}-{crude_q2[3] / s_lo:.0f})")
print(f"bound/reported {se_min(60) / se_rep:.1f}x; bound/widest rounding-consistent SE {se_min(60) / s_hi:.1f}x")
# Poverty quintile was imputed for 21 participants: allow n1 anywhere in 60 +/- 21
bmin = min(se_min(n1) for n1 in range(60 - 21, 60 + 22))
print(f"lower bound if n1 for Q2 is anywhere in 39..81 (imputation): {bmin:.3f}  -> {bmin / se_rep:.0f}x the reported SE, {bmin / s_hi:.1f}x the widest rounding-consistent SE")
for name, s in [("hard minimum SE", se_min(60)), ("count-based crude SE", crude_q2[3])]:
    lo2, hi2 = ci(2.0, s)
    print(f"aOR 2.0 with {name} {s:.3f}: 95% CI {lo2:.2f}-{hi2:.2f}, p={p_of(2.0, s):.3f}")
se_s4 = se_from_ci(0.70, 6.1)
print(f"Supplementary Table 4 (same model, five-facility term): 2.1 (0.70-6.1), p=0.099; implied SE {se_s4:.3f} = {se_s4 / se_rep:.0f}x the Table 2 SE")
print(f"   Wald p recomputed from that CI: {p_of(2.1, se_s4):.2f}")
se_q3 = se_from_ci(0.6, 0.9)
print(f"Third quintile: Table 2 aOR 0.7 (0.6-0.9), p=0.0009, implied SE {se_q3:.3f} vs bound {se_min(58):.3f}; "
      f"narrowest CI at 0.7: {ci(0.7, se_min(58))[0]:.2f}-{ci(0.7, se_min(58))[1]:.2f}; "
      f"Supplementary Table 4: 0.73 (0.22-2.4), p=0.60")

# ---------------------------------------------------------------------------
print("\n=== 4. Subcounty hospital: 9.7 (9.6-9.8) in the abstract and text, 9.8 (9.7-9.9) in Table 2 ===")
for lab, OR, lo, hi in [("abstract/text", 9.7, 9.6, 9.8), ("Table 2", 9.8, 9.7, 9.9)]:
    se = se_from_ci(lo, hi)
    s_lo, s_hi = se_rounding_range(lo, hi)
    print(f"{lab:14s} implied SE {se:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); bound {se_min(69):.3f} = {se_min(69) / se:.0f}x; bound/widest rounding-consistent SE {se_min(69) / s_hi:.0f}x")
cs = CR["Subcounty vs health centre"]
print(f"crude SE {cs[3]:.3f} = {cs[3] / se_from_ci(9.6, 9.8):.0f}x the abstract SE; aOR 9.7 with crude SE: CI {ci(9.7, cs[3])[0]:.1f}-{ci(9.7, cs[3])[1]:.1f}")
print(f"aOR 9.7 with hard minimum SE: CI {ci(9.7, se_min(69))[0]:.2f}-{ci(9.7, se_min(69))[1]:.2f}")
cc = CR["County vs health centre"]
print(f"County: crude SE {cc[3]:.3f}; reported adjusted SE {se_from_ci(15.7, 40.2):.3f}; bound {se_min(63):.3f}")

# ---------------------------------------------------------------------------
print("\n=== 5. Could a facility-clustered (sandwich) SE explain the narrow intervals? ===")
# Supplementary Table 1b ('Incurred catastrophic health expenditure' row): CHE / n by facility; level: 0 = health centre, 1 = subcounty, 2 = county
fac = [("Mitunguu HC", 0, 5, 85), ("Laare HC", 0, 6, 79), ("Mutuati SCH", 1, 12, 38),
       ("Kanyakine SCH", 1, 15, 31), ("Meru TRH", 2, 38, 63)]
lev_y = {L: sum(y for _, l, y, _ in fac if l == L) for L in range(3)}
lev_n = {L: sum(n for _, l, _, n in fac if l == L) for L in range(3)}
print("level CHE/n from facilities:", [f"{lev_y[L]}/{lev_n[L]}" for L in range(3)], "(Table 1: 11/164, 27/69, 38/63)")
# Univariable model with level dummies is saturated at the level, so fitted p = level proportion.
# Scores per facility for the level log-odds theta_L: u_g = y_g - n_g p_L. A_L = n_L p_L (1 - p_L).
G = len(fac)
var = {}
for L in range(3):
    pL = lev_y[L] / lev_n[L]
    A = lev_n[L] * pL * (1 - pL)
    B = sum((y - n * pL) ** 2 for _, l, y, n in fac if l == L)
    var[L] = B / A ** 2
for lab, k in [("without small-sample factor", 1.0), ("with G/(G-1)", G / (G - 1))]:
    se_sub = np.sqrt(k * (var[0] + var[1]))
    se_cty = np.sqrt(k * (var[0] + var[2]))
    print(f"cluster-robust SE {lab}: subcounty {se_sub:.3f}, county {se_cty:.3f}")
print(f"Table 2 unadjusted implied SEs: subcounty {se_from_ci(8.4, 11.1):.3f}, county {se_from_ci(19.9, 28.6):.3f}")

# ---------------------------------------------------------------------------
# Figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def style(ax):
    ax.set_facecolor(BG)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(INK2)
    ax.grid(axis="x", color=GRID, lw=0.8, which="both")
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)


# ---- Figure 1: second quintile and subcounty hospital
q2min = ci(2.0, se_min(60))
scmin = ci(9.7, se_min(69))
items = [
    # group, label, OR, lo, hi, kind (rep / rec), marker
    ("Second vs first poverty quintile", "Reported aOR (Abstract, Results, Table 2)", 2.0, 1.9, 2.1, "rep", "D"),
    ("Second vs first poverty quintile", "Reported aOR, Supplementary Table 4 (five-facility model)", 2.1, 0.70, 6.1, "rep", "s"),
    ("Second vs first poverty quintile", "Narrowest CI any logistic ML fit allows at aOR 2.0", 2.0, *q2min, "rec", "o"),
    ("Second vs first poverty quintile", "aOR 2.0 with the count-based SE from Table 1", 2.0, *ci(2.0, crude_q2[3]), "rec", "o"),
    ("Second vs first poverty quintile", "Crude OR from Table 1 counts (15/60 vs 13/60)", crude_q2[0], crude_q2[1], crude_q2[2], "rec", "o"),
    ("Subcounty hospital vs health centre", "Reported aOR (Abstract, Results)", 9.7, 9.6, 9.8, "rep", "D"),
    ("Subcounty hospital vs health centre", "Reported aOR (Table 2)", 9.8, 9.7, 9.9, "rep", "D"),
    ("Subcounty hospital vs health centre", "Narrowest CI any logistic ML fit allows at aOR 9.7", 9.7, *scmin, "rec", "o"),
    ("Subcounty hospital vs health centre", "aOR 9.7 with the count-based SE from Table 1", 9.7, *ci(9.7, cs[3]), "rec", "o"),
    ("Subcounty hospital vs health centre", "Crude OR from Table 1 counts (27/69 vs 11/164)", cs[0], cs[1], cs[2], "rec", "o"),
]
fig, ax = plt.subplots(figsize=(7.2, 7.0), facecolor=BG)
style(ax)
STEP, GAP = 1.25, 1.1
ypos, y, last = [], 0.0, None
for g, *_ in items:
    if g != last and last is not None:
        y += GAP
    ypos.append(y)
    y += STEP
    last = g
ypos = -np.array(ypos)
reptxt = {0: "2.0 (1.9–2.1)", 1: "2.1 (0.70–6.1)", 5: "9.7 (9.6–9.8)", 6: "9.8 (9.7–9.9)"}
for i, ((g, lab, OR, lo, hi, kind, mk), yy) in enumerate(zip(items, ypos)):
    col = ORANGE if kind == "rep" else BLUE
    ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt", zorder=2)
    ax.plot([lo, lo], [yy - 0.13, yy + 0.13], color=col, lw=2)
    ax.plot([hi, hi], [yy - 0.13, yy + 0.13], color=col, lw=2)
    ax.plot(OR, yy, marker=mk, ms=8, color=col, mec=BG, mew=1.2, zorder=3)
    txt = reptxt[i] if kind == "rep" else f"{OR:.2f} ({lo:.2f}–{hi:.2f})"
    ax.text(1.02, yy, txt, transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=INK)
    ax.text(0.34, yy + 0.2, lab, va="bottom", ha="left", fontsize=10, color=INK2)
    # OR = 1 reference: short segment through each data row only, so it never crosses a label
    ax.plot([1, 1], [yy - 0.3, yy + 0.18], color=INK2, lw=0.8, zorder=1)
ax.text(0.34, ypos[0] + 0.85, "Second vs first poverty quintile", fontsize=11, weight="bold", va="bottom")
ax.text(0.34, ypos[5] + 0.85, "Subcounty hospital vs health centre", fontsize=11, weight="bold", va="bottom")
ax.plot([1, 1], [ypos[-1] - 0.3, ypos[-1] - 0.6], color=INK2, lw=0.8, zorder=1)
ax.text(1.04, ypos[-1] - 0.75, "OR = 1 (no association)", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.33, 30)
ax.set_xticks([0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.5", "1", "2", "5", "10", "20"])
ax.set_ylim(ypos[-1] - 1.2, ypos[0] + 1.5)
ax.set_yticks([])
ax.set_xlabel("Odds ratio of catastrophic health expenditure, 95% CI (log scale)")
from matplotlib.lines import Line2D
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG, label="Reported in the paper"),
       Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8, mec=BG, label="Recomputed from the paper's own numbers")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.11), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.03, right=0.77, top=0.98, bottom=0.17)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)

# ---- Figure 2: standard errors for the seven aORs in the abstract
order = ["Age 30-44", "Accompanied", "County", "Female", "Above secondary", "Second quintile", "Subcounty (abstract/text)"]
names = {"Age 30-44": "Aged 30–44 years (2.6)", "Accompanied": "Accompanied (3.2)", "County": "County hospital (25.1)",
         "Female": "Female (1.8)", "Above secondary": "Above secondary education (1.6)",
         "Second quintile": "Second poverty quintile (2.0)", "Subcounty (abstract/text)": "Subcounty hospital (9.7)"}
d = {r[0]: r for r in FIG2}
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
style(ax)
for i, k in enumerate(order):
    lab, OR, lo, hi, se, s_lo, s_hi, smin = d[k]
    yy = -i
    ax.plot([s_lo, s_hi], [yy, yy], color=ORANGE, lw=2)
    ax.plot(se, yy, marker="D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
    ax.plot(smin, yy, marker="o", ms=8, color=BLUE, mec=BG, mew=1.2, zorder=3)
    ax.plot([smin, 1.2], [yy, yy], color=BLUE, lw=6, alpha=0.18, solid_capstyle="butt", zorder=1)
    status = "too small" if s_hi < smin else "possible"
    ax.text(1.03, yy, f"{se:.3f} vs {smin:.3f}  {status}", transform=ax.get_yaxis_transform(),
            va="center", fontsize=10, color=INK)
ax.set_yticks(-np.arange(len(order)))
ax.set_yticklabels([names[k] for k in order], fontsize=10, color=INK)
ax.set_xscale("log")
ax.set_xlim(0.002, 1.2)
ax.set_xticks([0.003, 0.01, 0.03, 0.1, 0.3, 1])
ax.set_xticklabels(["0.003", "0.01", "0.03", "0.1", "0.3", "1"])
ax.text(1.03, 0.55, "reported vs minimum", transform=ax.get_yaxis_transform(), va="bottom", fontsize=10, color=INK2)
ax.set_ylim(-len(order) + 0.4, 0.9)
ax.set_xlabel("Standard error of log odds ratio (log scale)")
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG,
              label="Implied by reported 95% CI (bar: range allowed by rounding)"),
       Line2D([0], [0], color=BLUE, lw=0, marker="o", ms=8, mec=BG,
              label="Smallest SE any logistic ML fit can give (shaded: attainable)")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(-0.55, -0.17), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.35, right=0.72, top=0.93, bottom=0.3)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
    fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nFigures written to", HERE)

Abstract swaps the DALYs-averted estimates of the two dynamic choice HIV prevention policies

Lancet HIV (2025) · doi:10.1016/S2352-3018(25)00169-9 · PMC12854106

Summary

The abstract's Findings give 18 400 DALYs averted per year (95% CI 16 700–20 100) for dynamic choice HIV prevention (DCP) including long-acting cabotegravir and 56 400 (52 300–60 500) for DCP without cabotegravir. Table 5 prints the same two estimates and intervals with the labels reversed: 18 400 for DCP without cabotegravir and 56 400 for DCP including it. Every other number in the paper agrees with Table 5. Each policy's cost difference divided by its Table 5 DALYs gives the reported ICERs ($13.2M / 56 400 = $234; $8.6M / 18 400 = $467), and the reported net DALYs and the extended-dominance result also fit Table 5. With the abstract's labels, the same costs give $717 and $152, and DCP including cabotegravir would be dominated. As printed, the abstract reverses which policy delivers more health benefit, although its ICER and conclusion are right.

What the paper reports

Abstract, Findings:

Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP including cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP without cabotegravir in 10 million adults. Compared with no DCP, there was a mean increase in annual discounted costs over 50 years: $8·6 million (7·7–9·4) for DCP without cabotegravir and $13·2 million (11·6–14·8) for DCP including cabotegravir. Addition of long-acting cabotegravir PrEP to DCP was cost-effective (vs DCP without cabotegravir); the incremental cost-effectiveness ratio for DCP including cabotegravir (vs no DCP) was $234 per DALY averted.

Abstract, Findings (the incidence result, in the same paragraph):

These increases in PrEP use led to HIV incidence reductions, with incidence rate ratios of 0·89 (0·67–1·17) for DCP without cabotegravir and 0·64 (0·44–0·97) for DCP including cabotegravir, relative to no DCP.

Table 5, "HIV-related deaths, DALYs, and costs over 50 years and cost-effectiveness analysis compared with no DCP". Columns: No DCP | DCP without cabotegravir | DCP including cabotegravir. "Data are mean (95% CI); 90% range":

Difference in number of HIV-related deaths per year | ·· | -1300 (-1400 to -1200); -4500 to 800 | -3700 (-4000 to -3400); -12 300 to 200

Difference in DALYs per year over 50 years | ·· | -18 400 (-20 100 to -16 700); -68 600 to 19 000 | -56 400 (-60 500 to -52 300); -178 800 to 8200

Difference in annual cost over 50 years | ·· | $8·6 million (7·7 to 9·4); -6·1 to 32·3 | $13·2 million (11·6 to 14·8); -16·9 to 61·0

Difference in net DALYs* per year over 50 years | ·· | -1200 (-3600 to 1200); -65 900 to 55 800 | -29 900 (-35 400 to -24 400); -179 700 to 81 700

Incremental cost-effectiveness ratio (cost per DALY averted) | ·· | Dominated | $234

Incremental cost-effectiveness ratio (cost per DALY averted) for DCP compared with no DCP (if long-acting cabotegravir PrEP not available) | ·· | $467 | NA

Table 5 footnote: "*Given a cost-effectiveness threshold of $500 per DALY averted."

Results, paragraph 3:

Overall, as a mean across setting-scenarios, both DCP policies lead to DALYs being averted (table 5). There was, however, variability over setting-scenarios (appendix p 3); the policy with the least DALYs incurred was the status quo (no DCP or long-acting cabotegravir) in 6% of setting-scenarios, DCP without cabotegravir in 11% of setting-scenarios, and DCP including cabotegravir in 83% of setting-scenarios.

Results, paragraph 5:

The policy of DCP without cabotegravir was not a candidate for the cost-effective choice of intervention, as a combination of the policies of status quo and DCP including cabotegravir (represented by a point half way along the line between the two policies) was more effective and less costly than this policy (ie, the policy was extendedly dominated). The incremental cost-effectiveness ratio (ICER) for the policy of DCP including cabotegravir was $234 per DALY averted (figure). This policy incurred the lowest overall mean net DALYs at a cost-effectiveness threshold of $500 per DALY averted (table 5). If long-acting cabotegravir was not available, the ICER for DCP was $467.

Methods, "Cost-effectiveness analysis":

Net DALYs averted was calculated by DALYs averted plus the difference in costs divided by the cost-effectiveness threshold.

What the paper's own numbers show

Method. The abstract and Table 5 report the same quantity: mean DALYs per year over 50 years, discounted at 3%, for 10 million adults, compared with no DCP. Table 5 signs it as a difference, so a negative value means DALYs averted. I first compared the two pairs of estimates and confidence intervals. I then tested each possible assignment of the two DALY estimates to the two policies against three relationships the paper defines or reports. (1) ICER vs no DCP = difference in annual cost / DALYs averted, compared with the reported $234 and $467. (2) Difference in net DALYs = −DALYs averted + difference in cost / $500, compared with the Table 5 net-DALY row. (3) Extended dominance, which the Results state for DCP without cabotegravir. Finally I compared the ordering with the paper's other effectiveness outcomes: HIV deaths averted and incidence rate ratios. All inputs are the rounded values printed in the paper. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
DALYs averted/yr, DCP without cabotegravir Table 5 18 400 (16 700–20 100) consistent with every check below
DALYs averted/yr, DCP including cabotegravir Table 5 56 400 (52 300–60 500) consistent with every check below
DALYs averted/yr, DCP including cabotegravir Abstract, Findings 18 400 (16 700–20 100) equals the Table 5 value for DCP without cabotegravir, CI included
DALYs averted/yr, DCP without cabotegravir Abstract, Findings 56 400 (52 300–60 500) equals the Table 5 value for DCP including cabotegravir, CI included
ICER vs no DCP, DCP including cabotegravir Abstract; Results p5; Table 5 $234 $13.2M / 56 400 = $234.04 (Table 5 labels); $13.2M / 18 400 = $717.39 (abstract labels)
ICER vs no DCP, DCP without cabotegravir Results p5; Table 5 $467 $8.6M / 18 400 = $467.39 (Table 5 labels); $8.6M / 56 400 = $152.48 (abstract labels)
Net DALY difference, DCP without cabotegravir Table 5 −1200 −18 400 + 17 200 = −1200 (Table 5 labels); −56 400 + 17 200 = −39 200 (abstract labels)
Net DALY difference, DCP including cabotegravir Table 5 −29 900 −56 400 + 26 400 = −30 000 (Table 5 labels; a cost of $13.15M–$13.25M, which rounds to $13.2M, gives −30 100 to −29 900); −18 400 + 26 400 = +8000 (abstract labels)
Extended dominance of DCP without cabotegravir Results p5; Table 5 "Dominated" stated Table 5 labels: $467 > $234, so it is extendedly dominated; the midpoint of no DCP and DCP including cabotegravir averts 28 200 DALYs at $6.6M, against 18 400 at $8.6M. Abstract labels: DCP including cabotegravir would avert fewer DALYs (18 400 < 56 400) at a higher cost, so it would be strongly dominated, which contradicts the paper
Incremental ICER, including vs without cabotegravir Abstract ("cost-effective (vs DCP without cabotegravir)") not printed Table 5 labels: $4.6M / 38 000 = $121.05 per DALY, below the $500 threshold, consistent with the statement
HIV-related deaths averted per year Table 5 1300 (without); 3700 (including) ratio including/without 2.85; DALY ratio 3.07 with the Table 5 labels, 0.33 with the abstract labels
Incidence reduction vs no DCP Abstract IRR 0·89 (without); 0·64 (including) 11% vs 36% reduction, so the policy including cabotegravir is the more effective one

Figure

The abstract and Table 5 print the same two DALY estimates with the policy labels swapped

Figure 1. Left: the DALYs-averted sentence in the abstract's Findings, with its policy labels (orange) as printed. Right: the same estimates as printed in Table 5, where a negative difference means DALYs averted. The arrows join each abstract value to the identical value and interval in Table 5, which belongs to the other policy. Below, in blue: each policy's cost difference from Table 5 divided by its DALYs averted under each labelling. Only the Table 5 labelling reproduces the reported ICERs of $234 and $467. All values are from the paper or are printed by recompute.py.txt.

Reported ICERs against the ICERs implied by each labelling

Figure 2. The ICER of each DCP policy against no DCP. Orange rings: the reported ICERs. Filled blue dots: cost difference / DALYs averted with the Table 5 labels. Open blue dots: the same calculation with the abstract's labels. The filled dots sit on the reported values. The open dots fall at $717, above the $500 threshold, for DCP including cabotegravir and at $152 for DCP without cabotegravir, which would reverse the paper's cost-effectiveness ranking. Dashed line: the $500 per DALY threshold used in the paper.

Why it matters

The sentence is the abstract's only statement of health benefit in DALYs. As printed, it says the policy without cabotegravir averts about three times as many DALYs as the policy including cabotegravir (56 400 vs 18 400). Table 5, the Results and the paper's conclusions all say the reverse. A reader who has only the abstract, or who takes the per-policy health gains from it for a review, a meta-analysis or a budget-impact estimate, would get the two policies' benefits the wrong way round. The abstract is then also inconsistent with itself. Its own costs ($8.6M and $13.2M) and its reported ICER of $234 cannot both hold with its DALY figures: with those figures, DCP including cabotegravir costs more and averts fewer DALYs than DCP without it, so it would be dominated rather than cost-effective.

The error does not affect the model results or the paper's conclusions. Table 5, the ICERs of $234 and $467, the net-DALY results, the extended-dominance finding and the Interpretation all use the correct assignment. The error is confined to the policy labels in one sentence of the abstract.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

If the May 2026 erratum does not already address it, the abstract's Findings sentence should read:

Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP without cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP including cabotegravir in 10 million adults.

The same change should be made wherever the abstract is reproduced (PubMed, PMC, Europe PMC). No change to Table 5, the Results, the ICERs or the conclusions is needed.

Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for PMC12854106
(Lancet HIV 2025, doi:10.1016/S2352-3018(25)00169-9).

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.

All inputs are copied from the paper as archived in PMC (same values in the
publisher PDF):
  Table 5 (columns: No DCP | DCP without cabotegravir | DCP including cabotegravir),
    rows "Difference in DALYs per year over 50 years", "Difference in annual cost
    over 50 years", "Difference in net DALYs* per year over 50 years",
    "Difference in number of HIV-related deaths per year", and the two ICER rows.
  Abstract, Findings: the DALYs-averted sentence, costs, ICER and incidence rate ratios.
  Methods: threshold of US$500 per DALY averted.
All figures are means per year over 50 years, discounted at 3%, for 10 million adults,
compared with no DCP.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
THRESHOLD = 500.0                                   # Methods, US$ per DALY averted
POL = ("without", "including")                      # DCP without / including cabotegravir
NAME = {"without": "DCP without cabotegravir", "including": "DCP including cabotegravir"}

# Table 5: difference in DALYs per year vs no DCP (negative = averted), mean (95% CI)
T5_DALY = {"without": (-18400, -20100, -16700), "including": (-56400, -60500, -52300)}
# Abstract, Findings: DALYs averted per year, as labelled in the abstract
ABS_DALY = {"including": (18400, 16700, 20100), "without": (56400, 52300, 60500)}
# Table 5 and abstract: difference in annual discounted cost vs no DCP, US$
COST = {"without": 8.6e6, "including": 13.2e6}
# Table 5: difference in net DALYs per year (threshold $500)
T5_NET = {"without": -1200, "including": -29900}
# Table 5: difference in HIV-related deaths per year
T5_DEATHS = {"without": -1300, "including": -3700}
# Abstract: incidence rate ratios vs no DCP
IRR = {"without": 0.89, "including": 0.64}
# Reported ICERs vs no DCP: $234 (abstract, Results p5, Table 5) and $467
# (Results p5, Table 5 "if long-acting cabotegravir PrEP not available")
ICER_REPORTED = {"without": 467, "including": 234}

# ---------------------------------------------------------------- derived
averted_t5 = {p: -T5_DALY[p][0] for p in POL}       # Table 5 attribution
averted_abs = {p: ABS_DALY[p][0] for p in POL}      # abstract attribution


def line(s=""):
    print(s)


line("1. Same estimates, same CIs, opposite labels")
for p in POL:
    m, lo, hi = T5_DALY[p]
    line(f"   Table 5  {NAME[p]:28s}: averted {-m} (95% CI {-hi} to {-lo})")
for p in POL:
    m, lo, hi = ABS_DALY[p]
    line(f"   Abstract {NAME[p]:28s}: averted {m} (95% CI {lo} to {hi})")
same_set = sorted(-T5_DALY[p][0] for p in POL) == sorted(ABS_DALY[p][0] for p in POL)
swapped = all(ABS_DALY[p][0] == -T5_DALY[q][0] for p, q in (("including", "without"), ("without", "including")))
line(f"   same pair of values: {same_set}; abstract value for each policy = Table 5 value of the other: {swapped}")

line()
line("2. ICER vs no DCP = cost difference / DALYs averted")
icer_t5 = {p: COST[p] / averted_t5[p] for p in POL}
icer_abs = {p: COST[p] / averted_abs[p] for p in POL}
for p in POL:
    line(f"   {NAME[p]:28s}: reported ${ICER_REPORTED[p]}; "
         f"Table 5 attribution ${COST[p]/1e6:.1f}M/{averted_t5[p]} = ${icer_t5[p]:.2f}; "
         f"abstract attribution ${COST[p]/1e6:.1f}M/{averted_abs[p]} = ${icer_abs[p]:.2f}")

line()
line("3. Net DALY difference = -(DALYs averted) + cost difference / threshold")
net_t5 = {p: -averted_t5[p] + COST[p] / THRESHOLD for p in POL}
net_abs = {p: -averted_abs[p] + COST[p] / THRESHOLD for p in POL}
for p in POL:
    line(f"   {NAME[p]:28s}: reported {T5_NET[p]}; cost/threshold = {COST[p]/THRESHOLD:.0f}; "
         f"Table 5 attribution {net_t5[p]:.0f}; abstract attribution {net_abs[p]:+.0f}")
line("   (the 100 gap for 'including' is consistent with the costs being printed to one decimal "
     f"in $ millions: $13.15M-$13.25M gives {-56400+13.15e6/THRESHOLD:.0f} to {-56400+13.25e6/THRESHOLD:.0f})")

line()
line("4. Dominance")
mid_daly = averted_t5["including"] / 2
mid_cost = COST["including"] / 2
line(f"   Table 5 attribution: ICER(without) {icer_t5['without']:.0f} > ICER(including) {icer_t5['including']:.0f} "
     f"-> 'without' extendedly dominated: {icer_t5['without'] > icer_t5['including']}")
line(f"   midpoint of no DCP and 'including': {mid_daly:.0f} DALYs averted at ${mid_cost/1e6:.1f}M, "
     f"vs 'without' {averted_t5['without']} at ${COST['without']/1e6:.1f}M")
inc_d = averted_t5["including"] - averted_t5["without"]
inc_c = COST["including"] - COST["without"]
line(f"   incremental, including vs without: {inc_d} DALYs, ${inc_c/1e6:.1f}M -> ${inc_c/inc_d:.2f} per DALY")
strong = averted_abs["including"] < averted_abs["without"] and COST["including"] > COST["without"]
line(f"   abstract attribution: 'including' averts {averted_abs['including']} < {averted_abs['without']} "
     f"at higher cost -> strongly dominated by 'without': {strong}")

line()
line("5. Other outcomes in the paper")
line(f"   HIV deaths averted per year: without {-T5_DEATHS['without']}, "
     f"including {-T5_DEATHS['including']}; ratio {T5_DEATHS['including']/T5_DEATHS['without']:.2f}")
line(f"   DALYs averted ratio including/without, Table 5: {averted_t5['including']/averted_t5['without']:.2f}; "
     f"abstract: {averted_abs['including']/averted_abs['without']:.2f}")
line(f"   incidence reduction vs no DCP: without {100*(1-IRR['without']):.0f}%, including {100*(1-IRR['including']):.0f}%")

# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"
BLUE = "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "text.parse_math": False,
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


def fmt(n):
    return f"{n:,}".replace(",", " ")


# ---------------------------------------------------------------- fig1: abstract vs Table 5
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")

for x0, title in ((2, "Abstract, Findings"), (54, "Table 5 and Results")):
    ax.add_patch(FancyBboxPatch((x0, 30), 44, 62, boxstyle="round,pad=0.6,rounding_size=1.5",
                                fc="white", ec=INK2, lw=1))
    ax.text(x0 + 2, 88, title, fontsize=11.5, fontweight="bold", va="top")

# Abstract panel (as printed)
a_inc = ABS_DALY["including"]
a_wo = ABS_DALY["without"]
ax.text(4, 78, f"{fmt(a_inc[0])} DALYs per year\n(95% CI {fmt(a_inc[1])}–{fmt(a_inc[2])})",
        fontsize=10.5, va="top", color=INK)
ax.text(4, 67, "for DCP including cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 56, f"{fmt(a_wo[0])} DALYs per year\n(95% CI {fmt(a_wo[1])}–{fmt(a_wo[2])})",
        fontsize=10.5, va="top", color=INK)
ax.text(4, 45, "for DCP without cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 37, "Reported ICER, including: $234 per DALY", fontsize=10, va="top", color=INK2)

# Table 5 panel (as printed)
t_wo = T5_DALY["without"]
t_inc = T5_DALY["including"]
ax.text(56, 78, "DCP without cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 73, f"−{fmt(-t_wo[0])} DALYs per year\n(−{fmt(-t_wo[1])} to −{fmt(-t_wo[2])})",
        fontsize=10.5, va="top")
ax.text(56, 56, "DCP including cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 51, f"−{fmt(-t_inc[0])} DALYs per year\n(−{fmt(-t_inc[1])} to −{fmt(-t_inc[2])})",
        fontsize=10.5, va="top")
ax.text(56, 39, "Reported ICERs: $467 (without),\n$234 (including)", fontsize=10, va="top", color=INK2)

# crossing connectors: each abstract number to the same number in Table 5
ax.add_patch(FancyArrowPatch((31, 75), (55, 69.5), arrowstyle="-|>", mutation_scale=12,
                             color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.add_patch(FancyArrowPatch((31, 53), (55, 49.5), arrowstyle="-|>", mutation_scale=12,
                             color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.text(50, 95.5, "Same estimates and CIs; the two policy labels are swapped",
        ha="center", fontsize=11, fontweight="bold", va="center")

# recomputed check underneath
ax.text(2, 24, "Check with the paper's own costs (Table 5: +$8.6M without, +$13.2M including, vs no DCP):",
        fontsize=10, va="top", color=INK)
ax.text(2, 18,
        f"Table 5 labels:  $13.2M / {fmt(averted_t5['including'])} = ${icer_t5['including']:.0f}  and  "
        f"$8.6M / {fmt(averted_t5['without'])} = ${icer_t5['without']:.0f}   (both match)",
        fontsize=10, va="top", color=BLUE)
ax.text(2, 12,
        f"Abstract labels: $13.2M / {fmt(averted_abs['including'])} = ${icer_abs['including']:.0f}  and  "
        f"$8.6M / {fmt(averted_abs['without'])} = ${icer_abs['without']:.0f}   (match neither)",
        fontsize=10, va="top", color=BLUE)
ax.text(2, 2, "Orange: abstract labels as printed.  Blue: recomputed from the paper's numbers.",
        fontsize=10, ha="left", va="bottom", color=INK2)
save(fig, "fig1")

# ---------------------------------------------------------------- fig2: ICER dot plot
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.20)
ypos = {"including": 1, "without": 0}
for p in POL:
    y = ypos[p]
    ax.plot([min(icer_t5[p], icer_abs[p]), max(icer_t5[p], icer_abs[p])], [y, y], color=GRID, lw=2, zorder=1)
    ax.plot(ICER_REPORTED[p], y, "o", ms=13, mfc="none", mec=ORANGE, mew=2, zorder=3)
    ax.plot(icer_t5[p], y, "o", ms=8, color=BLUE, zorder=4)
    ax.plot(icer_abs[p], y, "o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
    right = icer_abs[p] > icer_t5[p]
    ax.annotate(f"${icer_abs[p]:.0f}", (icer_abs[p], y), xytext=(10 if right else -10, 0),
                textcoords="offset points", ha="left" if right else "right", va="center",
                fontsize=10, color=BLUE)
    ax.annotate(f"${ICER_REPORTED[p]} reported\n${icer_t5[p]:.0f} recomputed", (icer_t5[p], y),
                xytext=(0, -12), textcoords="offset points", ha="center", va="top", fontsize=10, color=INK)
ax.axvline(THRESHOLD, color=INK2, lw=1, ls=(0, (3, 3)), zorder=2)
ax.text(THRESHOLD + 8, 1.55, "threshold $500", ha="left", fontsize=10, color=INK2)
ax.set_yticks([0, 1])
ax.set_yticklabels([NAME["without"], NAME["including"]])
ax.set_ylim(-0.75, 1.75)
ax.set_xlim(0, 800)
ax.set_xticks(range(0, 801, 100))
ax.set_xticklabels([f"${v}" for v in range(0, 801, 100)])
ax.set_xlabel("ICER vs no DCP (US$ per DALY averted)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", ms=13, mfc="none", mec=ORANGE, mew=2, ls="")
h2, = ax.plot([], [], "o", ms=8, color=BLUE, ls="")
h3, = ax.plot([], [], "o", ms=8, mfc=BG, mec=BLUE, mew=2, ls="")
fig.legend([h1, h2, h3], ["Reported ICER", "Cost / DALYs with Table 5 labels", "Cost / DALYs with abstract labels"],
           loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99),
           handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
line()
line("Figures written: fig1.svg/png, fig2.svg/png")

95% upper bound for zero neural tube defects among 384 dolutegravir-exposed outcomes is tenfold too small

Lancet HIV (2021) · doi:10.1016/S2352-3018(20)30268-X · PMC7869891

Summary

The paper's primary safety outcome is neural tube defect (NTD) prevalence after periconceptional dolutegravir (DTG) exposure. It observes no NTDs among 384 DTG-exposed and 1068 efavirenz (EFV)-exposed birth outcomes (Table 3). In the Abstract and the Results it reports a 95% Wilson confidence interval of 0–0·0010 for the DTG group and 0–0·0036 for the EFV group. For 0 events in n, the Wilson 95% upper bound is z²/(n + z²). With n = 384 that is 0·0099, about ten times the printed value; a bound of 0·0010 would need about 3838 outcomes, and no DTG-exposed pregnancy count in the paper comes close. The same method reproduces every other Wilson (prevalence) interval in the paper, including the EFV bound. The paper cites five DTG-exposed NTD rates from other studies: 1·04% (the Botswana estimate used in the Methods power calculation), 0·94%, 0·66% and 0·30% (Botswana) and 0·4% (the Antiretroviral Pregnancy Registry). The printed DTG bound (0·10%) lies below all five, so it appears to exclude every one of them. The bound implied by the paper's own counts (0·99%) contains four of them, 0·94%, 0·66%, 0·4% and 0·30%. Only the 1·04% power-calculation rate lies above it, by 0·05 percentage points.

What the paper reports

Abstract, Methods:

NTD incidences were calculated with 95% Wilson confidence intervals (95%CI).

Abstract, Findings:

There were no NTD in either DTG-exposed (0 [95%CI 0, 0·0010]) or EFV-exposed groups (0 [95%CI 0, 0·0036]).

Methods, "Variable definitions and statistical analysis":

We computed Wilson 95% confidence intervals (95%CI) for prevalence estimates of the primary outcomes.

Results, third paragraph:

The estimated NTD prevalence for DTG-exposed infants was 0 (95%CI 0-0·0010) and 0 (95%CI 0-0·0036) in EFV-exposed infants. Our composite outcome was more frequent in DTG-exposed fetuses (n=25, 7% [95%CI 0·044-0·094]) compared to those EFV-exposed (n=43, 4% [95%CI 0·030-0·054])

Table 3 ("Birth outcomes of periconceptionally DTG- and EFV- exposed women, Brazil, 2015-2018"), excerpt (selected rows; the P-value column and the "Outcome" row that carries its P value are omitted):

| Only EFV regimen(N=1068) | Any DTG regimen(N=384) |

Neural tube defects, n (%) | 0 (0) | 0 (0)

Live birth | 1025 (96) | 359 (93)

Stillbirth | 15 (1) | 2 (1)

Abortion | 28 (3) | 23 (6)

Results, "Updated NTD incidence estimate":

2 / (0·78 * 900 + 382) = 0·0018 or 1·8 per 1,000 DTG-exposed pregnancies (95%CI 0·5-6·7 per 1,000 DTG-exposed pregnancies)

Research in context panel, "Evidence before this study" (the comparison rates):

out of 426 women exposed to DTG at conception, four gave birth to infants with NTD (0·94%) compared to 0·10% among those exposed to other antiretrovirals. Since that initial warning, updated data from the Botswana study showed a decreased prevalence compared to the initial evaluation though statistically increased compared to women on non-DTG regimens (0·30% vs 0·10%). Other recent smaller studies from Botswana demonstrated increased NTD prevalence among women with DTG exposure (1 among 152 women or 0·66%) but statistical analyses comparing risk to other groups included the null. Lastly, data from North America and Europe using the Antiretroviral Pregnancy registry observed 1 NTD event among 248 DTG-exposed women (0·4%).

Methods, "Study design and data sources" (the power calculation):

Assuming a NTD prevalence of 0·06% for births among DTG-unexposed women and a prevalence of 1·04% in the DTG-exposed women (the estimated prevalence from Botswana available at the time)

Abstract, Interpretation:

Neither DTG or EFV exposure was associated with NTD in our national cohort; incidence of NTD is likely well under 1% among DTG-exposed HIV-positive women

The version of record repeats the bound. The publisher's edited abstract, as indexed in PubMed (PMID 33387477), reads:

There were no NTDs in either dolutegravir-exposed (0, 95% CI 0-0·0010) or efavirenz-exposed groups (0, 95% CI 0-0·0036).

What the paper's own numbers show

Method. I computed the Wilson score interval at the stated 95% level (z = 1·959964) for 0 events, using every denominator the paper gives for each group: birth outcomes (Table 3 N), women, and live births. For x = 0 the upper bound has the closed form z²/(n + z²). I inverted that formula to find the n at which the bound equals 0·0010. To rule out a different method or level, I also computed Clopper–Pearson, the rule of three, and Wilson bounds at 90% (equal to a one-sided 95% bound) and 99%. To test whether the authors' method and denominators are as stated, I recomputed the paper's other Wilson intervals: the composite outcome in each group and the updated DTG estimate. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
DTG NTD, 0/384 birth outcomes, Wilson 95% upper bound Abstract (Findings); Results ¶3; Table 3 N 0·0010 0·0099 (0·99%), 9·9 times the reported value
DTG NTD, 0/382 women Abstract; Results (same 0·0010) 0·0100
DTG NTD, 0/359 live births Table 3 (same 0·0010) 0·0106
EFV NTD, 0/1068 birth outcomes, Wilson 95% upper bound Abstract (Findings); Results ¶3; Table 3 N 0·0036 0·0036 (0·00358), matches
EFV NTD, 0/1045 women; 0/1025 live births Results; Table 3 (same 0·0036) 0·0037; 0·0037
DTG 0/384, other methods and levels derived 0·0010 Clopper–Pearson 95% 0·0096; rule of three 0·0078; Wilson 90% (one-sided 95%) 0·0070; Wilson 99% 0·0170
n at which a 0/n Wilson 95% upper bound equals 0·0010 derived — 3838 (the value rounds to 0·0010 only for n from 3655 to 4039)
Largest DTG-exposed pregnancy or outcome counts Abstract; Results; Table 3 382 women; 384 birth outcomes; updated DTG denominator 1084 (whole analysed cohort: 1427 women, 1452 outcomes) all below 3655
Larger counts in the paper that are not DTG-exposed pregnancy denominators Results ¶1; Introduction 3390 possible pregnancies screened; 2507 selected for investigation; 22 624 Brazilian women on DTG aged 15–49 0/n Wilson 95% upper bounds 0·0011, 0·0015 and 0·0002; none falls in the 3655–4039 range that gives 0·0010
Composite outcome DTG, 25/384 Results ¶3; Table 3 (2 stillbirths + 23 abortions) 7% (0·044–0·094) 6·5% (0·0445–0·0943), matches
Composite outcome EFV, 43/1068 Results ¶3; Table 3 (15 + 28) 4% (0·030–0·054) 4·0% (0·0300–0·0538), matches
Updated DTG estimate, 2/(0·78 × 900 + 382) = 2/1084 Results, updated estimate; Abstract 0·0018 (0·0005–0·0067) 0·00185 (0·00051–0·00670), matches
Table 3 column totals Table 3 N = 384; N = 1068 359 + 2 + 23 = 384; 1025 + 15 + 28 = 1068

The DTG NTD bound is the only Wilson (prevalence) interval in the paper that does not reproduce. Every other Wilson interval reproduces with the stated method and the Table 3 denominators. The bootstrap 95% CIs for the propensity-weighted odds ratios (Methods; Results; Figure 2 of the paper) depend on individual-level data and were not recomputed.

Figure

Reported and recomputed 95% confidence intervals for NTD prevalence, with the comparison rates the paper cites

Figure 1. NTD prevalence intervals on one axis, in percent. Each pair of rows shows the interval as reported in the paper (orange) and as recomputed from the paper's own counts with the Wilson 95% method the paper states (blue). Dots are point estimates and the end ticks are upper bounds, with values printed next to them. For EFV (0/1068) and for the updated DTG estimate (2/1084) the reported and recomputed intervals coincide. For DTG (0/384) the recomputed upper bound is 0·99%, 9·9 times the reported 0·10%. The lower rows show every NTD rate the paper cites for comparison, each with a thin dashed line running up through the intervals: five DTG-exposed rates (filled diamonds: Botswana 1·04% from the power calculation, 0·94%, 0·66% and 0·30%, and the Pregnancy Registry 0·4%) and two rates without DTG exposure (hollow diamonds: Botswana non-DTG 0·10%, Brazil general population 0·06%). As printed, the DTG interval ends at 0·10% and lies below all five DTG-exposed rates. As recomputed, it contains four of them; only the 1·04% power-calculation rate lies above it.

Wilson 95% upper bound for zero events as a function of sample size

Figure 2. The Wilson 95% upper bound for 0 events in n, z²/(n + z²), on log–log axes (blue line). The recomputed DTG (n = 384, 0·99%) and EFV (n = 1068, 0·36%) bounds lie on the line, and the EFV value matches the paper. The reported DTG bound of 0·0010 at n = 384 (orange dot, with a dashed orange line at its level) lies far off the line. The line reaches 0·0010 only at n = 3838 (hollow diamond), about ten times the DTG sample and larger than the whole analysed cohort (1452 birth outcomes).

Why it matters

NTD prevalence among women exposed to DTG at conception is the outcome the study was set up to measure, after the 2018 safety signal from Botswana. Its confidence interval is the cohort's own statement of how high the DTG-associated NTD risk could plausibly be. The misprinted bound is in the Abstract Findings, which is also the version of record's abstract and is indexed in PubMed, and in the Results. The correct value is printed nowhere in the paper, so a reader cannot recover it without recomputing.

The error changes what the cohort appears to exclude.

Also unaffected: the point estimates (0 NTDs in each group), the Table 3 counts, the composite-outcome intervals and regressions, and the updated estimate (0·18%, 95% CI 0·05–0·67%). The Interpretation's "well under 1%" is supported by the updated estimate, which is computed correctly. On the cohort data alone, though, the upper bound is 0·99%, just under 1%. The overall conclusions survive. What is overstated, about tenfold, is the precision of the primary-outcome estimate in the Abstract and Results.

Explanations considered

Status

Suggested correction

  1. In the Abstract (Findings) and the Results (third paragraph), replace the DTG NTD interval "0–0·0010" with the Wilson 95% interval for 0/384, 0–0·0099 (0–0·0100 if the 382 women are the intended denominator). Leave the EFV interval (0–0·0036) as it is.
  2. State the denominator used for the NTD prevalence (birth outcomes, women or live births), since the paper uses all three elsewhere.
  3. Optionally, note that the corrected DTG interval contains the DTG-exposed prevalences from the Research in context panel (0·94%, 0·66%, 0·4% and 0·30%), though not the 1·04% assumed in the power calculation, which fits the Discussion's statement that the study was underpowered for NTDs. The updated estimate, 0·0018 (0·0005–0·0067), needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the 95% Wilson confidence intervals for neural tube defect (NTD)
prevalence in PMC7869891 (Lancet HIV 2021, doi:10.1016/S2352-3018(20)30268-X).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PubMed Central (author manuscript):
  Abstract (Findings) and Results paragraph 3: reported NTD intervals
  Table 3: N = 384 (any DTG regimen) and N = 1068 (only EFV regimen) birth outcomes,
           0 NTDs in each; live births 359 / 1025; stillbirths 2 / 15; abortions 23 / 28
  Results (Characteristics): 382 DTG-exposed and 1045 EFV-exposed women
  Results (Updated NTD incidence estimate): 2 / (0.78 * 900 + 382)
  Methods: "We computed Wilson 95% confidence intervals (95%CI) for prevalence
           estimates of the primary outcomes."
  Research in context panel / Discussion: comparison rates 0.94%, 0.30%, 0.66%
           (1/152) and 0.10% (Botswana), 0.4% (1/248, Antiretroviral Pregnancy
           Registry) and 0.06% (Brazil, general population)
  Methods (power calculation): assumed DTG prevalence 1.04% ("the estimated
           prevalence from Botswana available at the time")
"""
import math
import os

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)  # 1.959964

# ------------------------------------------------------------------ inputs (paper)
N_DTG_OUT, N_EFV_OUT = 384, 1068          # Table 3 column Ns (birth outcomes)
N_DTG_WOMEN, N_EFV_WOMEN = 382, 1045      # Results / Abstract
N_DTG_LIVE, N_EFV_LIVE = 359, 1025        # Table 3 live births
STILL = {"DTG": 2, "EFV": 15}             # Table 3
ABORT = {"DTG": 23, "EFV": 28}            # Table 3
COMPOSITE = {"DTG": 25, "EFV": 43}        # Abstract / Results (manuscript)
REPORTED = {                              # Abstract Findings and Results paragraph 3
    "DTG NTD upper": 0.0010,
    "EFV NTD upper": 0.0036,
    "DTG composite": (0.044, 0.094),
    "EFV composite": (0.030, 0.054),
    "updated": (0.0018, 0.0005, 0.0067),
}
UPD_NEW, UPD_FRAC, UPD_EVENTS = 900, 0.78, 2   # Results, updated estimate
CHART_CONFIRMED, CHART_REVIEWED = 382, 490
POWER_ASSUMED_DTG = 0.0104  # Methods power calculation: assumed DTG prevalence 1.04%
COMPARATORS = [  # (rate, label, DTG-exposed?) rates cited in the paper
    (POWER_ASSUMED_DTG, "Botswana DTG, power calc. 1.04%", True),   # Methods
    (0.0094, "Botswana DTG, initial 0.94%", True),                  # Research in context
    (0.0066, "Botswana DTG, 1/152, 0.66%", True),                   # Research in context
    (0.0040, "Pregnancy Registry DTG, 1/248, 0.4%", True),          # Research in context
    (0.0030, "Botswana DTG, updated 0.30%", True),                  # Research in context
    (0.0010, "Botswana non-DTG 0.10%", False),                      # Research in context
    (0.0006, "Brazil, general population 0.06%", False),            # Methods / Discussion
]


def wilson(x, n, conf=0.95):
    z = stats.norm.ppf(1 - (1 - conf) / 2)
    p = x / n
    den = 1 + z * z / n
    c = (p + z * z / (2 * n)) / den
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
    return max(0.0, c - h), c + h


def cp_upper(x, n, conf=0.95):
    return stats.beta.ppf(1 - (1 - conf) / 2, x + 1, n - x)


def n_for_zero_upper(u, z=Z):
    """n at which the Wilson upper bound for 0/n equals u: z^2/(n+z^2) = u."""
    return z * z / u - z * z


print("=" * 78)
print("1. Wilson 95% upper bound for 0 NTDs, every denominator in the paper")
print("   closed form for x = 0: upper = z^2 / (n + z^2), z = %.6f" % Z)
rows = [("DTG birth outcomes (Table 3 N)", N_DTG_OUT), ("DTG women", N_DTG_WOMEN),
        ("DTG live births (Table 3)", N_DTG_LIVE), ("EFV birth outcomes (Table 3 N)", N_EFV_OUT),
        ("EFV women", N_EFV_WOMEN), ("EFV live births (Table 3)", N_EFV_LIVE)]
for lab, n in rows:
    w = wilson(0, n)[1]
    print(f"   {lab:32s} n={n:5d}  Wilson={w:.5f} ({100*w:.2f}%)  "
          f"Clopper-Pearson={cp_upper(0, n):.5f}  rule of three={3/n:.5f}")
print(f"   reported: DTG {REPORTED['DTG NTD upper']:.4f}   EFV {REPORTED['EFV NTD upper']:.4f}")
dtg_w = wilson(0, N_DTG_OUT)[1]
efv_w = wilson(0, N_EFV_OUT)[1]
print(f"   ratio recomputed/reported, DTG: {dtg_w / REPORTED['DTG NTD upper']:.2f}")
print(f"   EFV recomputed {efv_w:.5f} rounds to {round(efv_w, 4):.4f} (reported 0.0036)")

print("\n2. Other levels / methods for 0/384")
for conf in (0.90, 0.95, 0.99):
    print(f"   Wilson {conf:.0%} upper = {wilson(0, N_DTG_OUT, conf)[1]:.5f}")
print(f"   one-sided 95% Wilson (z=1.645) = {wilson(0, N_DTG_OUT, 0.90)[1]:.5f}")
print(f"   Clopper-Pearson 95% = {cp_upper(0, N_DTG_OUT):.5f};  rule of three = {3/N_DTG_OUT:.5f}")
print(f"   0.0010 read as a percentage = {0.0010/100:.6f} as a proportion "
      f"(would need n of about {n_for_zero_upper(0.0010/100):,.0f})")

print("\n3. Sample size at which a 0/n Wilson 95% upper bound is 0.0010")
print(f"   exactly 0.0010: n = {n_for_zero_upper(0.0010):.1f}")
print(f"   rounds to 0.0010 (0.00095 <= upper < 0.00105): "
      f"{n_for_zero_upper(0.00105):.1f} < n <= {n_for_zero_upper(0.00095):.1f}")
print(f"   largest DTG-exposed pregnancy/outcome counts: {N_DTG_WOMEN} women, {N_DTG_OUT} outcomes; "
      f"updated DTG denominator {UPD_FRAC*UPD_NEW + N_DTG_WOMEN:.0f}; "
      f"whole analysed cohort {N_DTG_WOMEN + N_EFV_WOMEN} women, {N_DTG_OUT + N_EFV_OUT} outcomes")
LARGER = [  # larger counts printed in the paper that are not DTG-exposed pregnancy denominators
    ("possible pregnancies screened (Results para 1)", 3390),
    ("selected for investigation (Results para 1)", 2507),
    ("Brazilian women on DTG aged 15-49, May 2018 (Introduction)", 22624),
]
print("   larger counts in the paper (not DTG-exposed pregnancy denominators), 0/n Wilson 95% upper:")
for lab, n in LARGER:
    u = wilson(0, n)[1]
    print(f"     {lab:60s} n={n:6d}  upper={u:.5f}  rounds to {round(u, 4):.4f}  "
          f"in rounding range for 0.0010? {n_for_zero_upper(0.00105) < n <= n_for_zero_upper(0.00095)}")

print("\n4. Checks that the stated method and denominators reproduce the paper's other CIs")
for arm, n in (("DTG", N_DTG_OUT), ("EFV", N_EFV_OUT)):
    x = COMPOSITE[arm]
    lo, hi = wilson(x, n)
    print(f"   composite {arm} {x}/{n}: {100*x/n:.1f}%  Wilson {lo:.4f}-{hi:.4f}  "
          f"(reported {REPORTED[arm + ' composite'][0]:.3f}-{REPORTED[arm + ' composite'][1]:.3f})")
    print(f"     Table 3 stillbirths + abortions {arm}: {STILL[arm]} + {ABORT[arm]} = {STILL[arm]+ABORT[arm]}")
    print(f"     Table 3 live + still + abortion {arm}: "
          f"{(N_DTG_LIVE if arm == 'DTG' else N_EFV_LIVE) + STILL[arm] + ABORT[arm]} (N = {n})")
print(f"   chart-review fraction 382/490 = {CHART_CONFIRMED/CHART_REVIEWED:.4f}")
n_upd = UPD_FRAC * UPD_NEW + N_DTG_WOMEN
lo, hi = wilson(UPD_EVENTS, n_upd)
print(f"   updated estimate 2/(0.78*900+382) = 2/{n_upd:.0f} = {UPD_EVENTS/n_upd:.5f}  "
      f"Wilson {lo:.5f}-{hi:.5f}  (reported 0.0018, 0.0005-0.0067)")

print("\n5. What 0/384 can exclude: comparator rates cited in the paper")
print("   (a rate on the bound itself counts as not excluded)")
for r, lab, _ in COMPARATORS:
    p0 = (1 - r) ** N_DTG_OUT
    print(f"   {lab:36s} inside reported CI (<=0.0010)? {r <= 0.0010!s:5s}  "
          f"inside recomputed CI (<={dtg_w:.4f})? {r <= dtg_w!s:5s}  P(0 events in 384) = {p0:.4f}")
print(f"   gap between 1.04% and the recomputed bound: {100*(POWER_ASSUMED_DTG - dtg_w):.3f} "
      f"percentage points")
print("   two-sided 95% interval: each tail has probability 0.025")

# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, REF = "#eb6834", "#2a78d6", "#e4e3df", "#9a9893"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
    "axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
    plt.close(fig)


# Figure 1: reported vs recomputed intervals on one axis (percent), with every
# rate the paper cites drawn as its own row and a thin dashed line up through
# the interval rows
from matplotlib.lines import Line2D  # noqa: E402
upd_lo, upd_hi = wilson(UPD_EVENTS, n_upd)
items = [  # (label, point, lo, hi, color)
    ("DTG, 0/384 — reported", 0, 0, REPORTED["DTG NTD upper"], ORANGE),
    ("DTG, 0/384 — recomputed (Wilson)", 0, 0, dtg_w, BLUE),
    ("EFV, 0/1068 — reported", 0, 0, REPORTED["EFV NTD upper"], ORANGE),
    ("EFV, 0/1068 — recomputed (Wilson)", 0, 0, efv_w, BLUE),
    ("Updated DTG, 2/1084 — reported", 0.0018, 0.0005, 0.0067, ORANGE),
    ("Updated DTG, 2/1084 — recomputed", UPD_EVENTS / n_upd, upd_lo, upd_hi, BLUE),
]
ys = [12.2, 11.6, 10.4, 9.8, 8.6, 8.0]
rate_ys = [6.6 - 0.6 * i for i in range(len(COMPARATORS))]  # descending rates, top to bottom
TOP = 12.9
LBL_BOX = dict(boxstyle="square,pad=0.12", fc=BG, ec="none")
fig, ax = plt.subplots(figsize=(7.2, 6.6))
for (r, lab, is_dtg), y in zip(COMPARATORS, rate_ys):
    ax.plot([100 * r, 100 * r], [y, TOP], color=REF, lw=0.9, ls=(0, (3, 3)), zorder=1)
    ax.plot([100 * r], [y], marker="D", ms=8, color=SEC if is_dtg else BG, mec=SEC, mew=1.5,
            zorder=4)
    ax.text(100 * r + 0.02, y, f"{100*r:.2f}%", va="center", ha="left", fontsize=10,
            color=SEC, zorder=5)
for (lab, p, lo, hi, col), y in zip(items, ys):
    ax.plot([100 * lo, 100 * hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
    ax.plot([100 * hi], [y], marker="|", ms=10, mew=2, color=col, zorder=3)
    ax.plot([100 * p], [y], marker="o", ms=8, color=col, zorder=4, mfc=col, mec=BG, mew=1)
    ax.text(100 * hi + 0.015, y, f"{100*hi:.2f}%", va="center", ha="left", fontsize=10,
            color=col, bbox=LBL_BOX, zorder=5)
ax.set_yticks(ys + rate_ys)
ax.set_yticklabels([it[0] for it in items] + [c[1].rsplit(" ", 1)[0].rstrip(",").replace("power calc.", "power calculation")
                                               for c in COMPARATORS],
                   fontsize=10)
for t, c in zip(ax.get_yticklabels(), [TXT] * len(items) + [SEC] * len(COMPARATORS)):
    t.set_color(c)
ax.text(-0.03, 7.25, "Rates the paper cites (filled = DTG-exposed, hollow = not DTG-exposed)",
        ha="left", va="center", fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.set_ylim(rate_ys[-1] - 0.5, TOP)
ax.set_xlim(-0.03, 1.2)
ax.set_xlabel("NTD prevalence, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(100 * dtg_w, 11.9), xytext=(100 * REPORTED["DTG NTD upper"], 11.9),
            arrowprops=dict(arrowstyle="->", color=SEC, lw=1.2), zorder=5)
ax.text(100 * (dtg_w + REPORTED["DTG NTD upper"]) / 2, 11.93,
        f"{dtg_w / REPORTED['DTG NTD upper']:.1f}× wider", ha="center", va="bottom",
        fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in the paper"),
                   Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
                          label="Recomputed from the paper's counts (Wilson 95%)"),
                   Line2D([], [], color=SEC, lw=0.9, ls=(0, (3, 3)), marker="D", ms=8,
                          label="Rate cited in the paper")],
          loc="upper center", bbox_to_anchor=(0.3, -0.1), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")

# Figure 2: Wilson 0/n upper bound as a function of n
n_grid = np.logspace(2, 4.2, 400)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.plot(n_grid, 100 * Z * Z / (n_grid + Z * Z), color=BLUE, lw=2)
n_req = n_for_zero_upper(REPORTED["DTG NTD upper"])
ax.axhline(100 * REPORTED["DTG NTD upper"], color=ORANGE, lw=1.2, ls=(0, (4, 3)))
ax.plot([N_DTG_OUT], [100 * REPORTED["DTG NTD upper"]], "o", ms=9, color=ORANGE)
ax.plot([N_DTG_OUT], [100 * dtg_w], "o", ms=9, color=BLUE, mec=BG)
ax.plot([N_EFV_OUT], [100 * efv_w], "s", ms=9, color=BLUE, mec=BG)
ax.plot([n_req], [100 * REPORTED["DTG NTD upper"]], "D", ms=8, mfc=BG, mec=BLUE, mew=2,
        )
ax.annotate(f"DTG, n = 384: {100*dtg_w:.2f}%", (N_DTG_OUT, 100 * dtg_w), xytext=(10, 6),
            textcoords="offset points", color=BLUE, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(100, 15000)
ax.set_ylim(0.02, 5)
ax.set_xticks([100, 384, 1068, 3838, 10000])
ax.set_xticklabels(["100", "384", "1068", "3838", "10 000"])
ax.set_yticks([0.05, 0.1, 0.36, 1, 2])
ax.set_yticklabels(["0.05%", "0.10%", "0.36%", "1%", "2%"])
ax.minorticks_off()
ax.set_xlabel("Number of birth outcomes, n (log scale)")
ax.set_ylabel("Upper 95% bound, % (log scale)")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.annotate("Reported: 0.0010 at n = 384", (N_DTG_OUT, 100 * REPORTED["DTG NTD upper"]),
            xytext=(10, -16), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(f"EFV, n = 1068: {100*efv_w:.2f}%\n(matches reported 0.0036)", (N_EFV_OUT, 100 * efv_w),
            xytext=(12, 4), textcoords="offset points", color=BLUE, fontsize=10)
ax.annotate(f"n = {n_req:,.0f} needed for 0.0010", (n_req, 100 * REPORTED["DTG NTD upper"]),
            xytext=(8, 8), textcoords="offset points", color=BLUE, fontsize=10)
ax.text(700, 2.3, "Line: Wilson 95% upper bound for 0 events in n", color=BLUE, fontsize=10, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")

Reported p<0.001 for mouse leak frequencies cannot be reproduced; the paper's counts give p=0.11–0.36

Lancet Neurology (2026) · doi:10.1016/S1474-4422(26)00140-7 · PMC13275636

Summary

The Results report that CSF leaks were significantly more frequent in all three Fbn2 mutant mouse strains than in wild-type mice: relative risks (RR) of 1.38, 1.33 and 1.27, 95% confidence intervals that exclude 1, and "all comparisons p<0·001". The legend of Figure 4A goes further, marking p<0.0001 for two of the strains. The paper names the χ² test for this comparison. Applied to the paper's own counts (15/19, 22/29 and 8/11 against 20/35 in wild-type), that test gives p = 0.109, 0.117 and 0.356. Fisher's exact test gives 0.14, 0.19 and 0.49, and the 95% CIs for the RRs are 0.96–2.00, 0.93–1.89 and 0.80–2.02, so all three include 1. The reported CIs are reproduced to two decimals only if each percentage is treated as a count out of 100 mice per group. The comparison is the main in-vivo support for the abstract's statement that the mutant mice "showed a predisposition for dural rupture". The paper's genetic burden results do not depend on it.

What the paper reports

Results, in-vivo paragraph (Marfan comparison, then Fbn2 strains):

There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p>0·05, relative risk [RR]=0·94, 95% CI 0·74–1·22, figure 4A). Contrastingly, these frequencies were significantly higher for Fbn2A1052T/+ mice (15 [79%] of 19), RR 1·38, 95% CI 1·14–1·69), for Fbn2D1581V/+ mice (22 [76%] of 29], RR 1·33, 95% CI 1·09–1·63), and for Fbn2M2387T/+ mice (eight [73%] of 11], RR 1·27, 95% CI 1·04–1·58), with all comparisons p<0·001 (figure 4A).

(The unmatched brackets are in the published text.)

Methods, Statistical analysis:

Proportions of mice with leaks were compared between mutant and wild-type mice by use of χ2 tests. p<0·05 was used as significance threshold.

Figure 4 legend:

(A) Proportion of CSF leak occurrence in mouse strains following intrathecal infusion.

*p<0·001. †p<0·0001.

The earlier check of the figure image (gr4.jpg from the publisher's supplementary package) found † on the A1052T and D1581V bars and * on the M2387T bar. The bar heights (about 57%, 54%, 79%, 76% and 73%) match the counts in the text.

Abstract, Findings:

Mice carrying variants equivalent to the three FBN2 variants in TB domains found in individuals with type 1b spontaneous spinal CSF leak (Fbn2A1052T/+, Fbn2D1581V/+, and Fbn2M2387T/+) showed a predisposition for dural rupture on controlled leak induction.

Research in context, Added value of this study:

Functional in-vitro studies and animal model data are supportive of a causal effect of the identified variants in FBN2.

Discussion, first paragraph:

We also showed increased CSF leak susceptibility and severity in fibrillin-2 mutant mice, and increased dural compliance in Marfan syndrome mice.

Discussion, mouse paragraph:

Our findings revealed that type 1b spontaneous spinal CSF leak-associated variants in the TB3, TB4, and TB7 domains of fibrillin-2 led to increased CSF leak frequency and severity in mice, a characteristic not attributable to increased pressure or volume at the time of leak.

Marfan syndrome mice also showed increased meningeal compliance without increased leak frequency or severity.

All quotes were checked verbatim against the PMC-archived text. Superscripts on the allele names (A1052T/+ etc.) are lost in plain text.

What the paper's own numbers show

Method. Each strain was compared with the wild-type group (20 of 35 mice with leaks) in a 2 × 2 table built from the counts printed in the Results. For each comparison I calculated the RR with its usual log-scale (Katz) 95% CI and the Pearson χ² test without continuity correction, which is the test named in the Methods. As checks I also ran the Yates-corrected χ², Fisher's exact test (two-sided and one-sided), Barnard's exact test, a pooled comparison of all Fbn2 mice, and a comparison against pooled controls (wild-type plus Marfan). To find where the reported intervals come from, I repeated the calculation with each printed percentage used as a count out of 100. All of this is in recompute.py.txt.

Quantity Location Reported Recomputed from the paper's counts
Leak frequencies Results; Fig 4A 79%, 76%, 73%, 54% vs 57% 15/19 = 78.9%, 22/29 = 75.9%, 8/11 = 72.7%, 7/13 = 53.8%, 20/35 = 57.1% (consistent)
Fbn2 A1052T/+ vs WT, p Results; Fig 4A (†) p<0.001 (text); p<0.0001 (figure) χ² p = 0.109 (Yates 0.192; Fisher 0.142; Barnard 0.123)
Fbn2 A1052T/+ RR (95% CI) Results 1.38 (1.14–1.69) 1.38 (0.96–2.00)
Fbn2 D1581V/+ vs WT, p Results; Fig 4A (†) p<0.001 (text); p<0.0001 (figure) χ² p = 0.117 (Yates 0.192; Fisher 0.186; Barnard 0.133)
Fbn2 D1581V/+ RR (95% CI) Results 1.33 (1.09–1.63) 1.33 (0.93–1.89)
Fbn2 M2387T/+ vs WT, p Results; Fig 4A (*) p<0.001 χ² p = 0.356 (Yates 0.569; Fisher 0.486; Barnard 0.449)
Fbn2 M2387T/+ RR (95% CI) Results 1.27 (1.04–1.58) 1.27 (0.80–2.02)
Fbn1 C1039G/+ (Marfan) RR (95% CI), p Results 0.94 (0.74–1.22), p>0.05 0.94 (0.53–1.68), χ² p = 0.838. The conclusion (not significant) holds; the CI is too narrow
One-sided Fisher (mutant > WT) check n/a 0.095, 0.095, 0.289
SE(log RR): from the reported CI vs from the counts check 0.100, 0.103, 0.107, 0.128 0.188, 0.180, 0.236, 0.296 (1.75–2.32 times wider)
RR CI with percentages used as counts out of 100 diagnostic 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.22 79/100, 76/100, 73/100, 54/100 vs 57/100: 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.21
χ² p with percentages used as counts out of 100 diagnostic p<0.001 (all three) 0.0009 / 0.0044 / 0.018. Even this does not give p<0.001 for all three
All Fbn2 mice pooled (45/59) vs WT 20/35 sensitivity n/a RR 1.33 (0.97–1.84); χ² p = 0.052; Fisher p = 0.066
Each Fbn2 strain vs pooled controls (WT + Marfan, 27/48) sensitivity n/a χ² p = 0.083, 0.083, 0.316
Global 2 × 5 χ² across all five strains sensitivity n/a χ² = 4.99, df = 4, p = 0.288
Leaks needed for χ² p<0.001 vs WT 20/35 sensitivity observed 15/19, 22/29, 8/11 ≥19/19, ≥28/29; not reachable with 11 mice (11/11 gives p = 0.008)

Every printed percentage matches its count, so the counts are internally consistent. No standard test of these counts gives p<0.05 for any single strain, let alone p<0.001. For the M2387T group, p<0.001 cannot be reached against 20/35 even if all 11 mice had leaked.

Figure

Reported and recomputed relative risks of CSF leak, with 95% CIs, on a log scale

Figure 1. Relative risk of CSF leak in each mutant strain compared with wild-type mice (20/35), on a log scale. Orange circles and bars are the RRs and 95% CIs printed in the Results. Blue squares and bars are the RRs and log-scale 95% CIs recomputed from the counts printed alongside them (shown under each strain name). The thin vertical line marks RR = 1. The point estimates agree. The recomputed intervals are 1.75 to 2.3 times wider on the log scale, and for all three Fbn2 strains they cross 1, whereas the reported intervals do not. The Marfan interval is also too narrow as printed, although both versions include 1.

Reported and recomputed p-values for each Fbn2 strain against wild-type, on a log scale

Figure 2. p-values for each Fbn2 strain against wild-type, on a log scale. Orange triangles show the reported thresholds: filled for "p<0·001" in the Results text, open for the "p<0·0001" marker (†) in the Figure 4 legend. Each triangle marks an upper bound. Blue markers show the values recomputed from the paper's counts: filled squares for the Pearson χ² test named in the Methods, open diamonds for Fisher's exact test. The thin vertical line marks the paper's significance threshold, p = 0.05. The recomputed values lie two to three orders of magnitude above the reported bounds, and all lie above 0.05.

Why it matters

The in-vivo leak experiment is the paper's test of whether the FBN2 variants found in patients impair dural integrity in a living animal. The leak-frequency comparison is the only in-vivo measure the paper reports as showing a difference in leak occurrence between Fbn2 mice and wild-type mice. The abstract ("showed a predisposition for dural rupture on controlled leak induction"), the Research in context panel ("animal model data are supportive of a causal effect") and the Discussion ("led to increased CSF leak frequency") all rely on it. The contrast the Discussion draws with Marfan mice, which showed increased compliance "without increased leak frequency", also relies on it. Recomputed from the paper's own counts, none of the three strain-level differences is statistically significant at the paper's own threshold of 0.05. The pooled Fbn2 group is borderline (p = 0.052). The direction of effect is unchanged: every Fbn2 strain leaked more often than wild-type (73–79% vs 57%). With 11 to 29 mice per group, the data are compatible both with no difference and with a doubling of risk.

Several results are unaffected. The exome burden enrichment (p = 0.041, 0.004 and 0.0003 in the three control cohorts) rests on human data, and so do the domain-distribution and fibroblast-binding results. The separately reported larger pressure drop at leak onset in Fbn2 mice (Figure 4E–F) and the Marfan compliance result (Figure 4D) are different measurements; they could not be checked here because no per-mouse data are published. The genetic conclusion therefore still stands. What changes is that the leak-frequency data do not, by themselves, provide statistically significant in-vivo evidence for it.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the Results, in-vivo paragraph, the RR confidence intervals and p-values should be recomputed from the mouse-level counts. If the counts printed are correct, the passage would read approximately:

There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p=0·84, relative risk [RR]=0·94, 95% CI 0·53–1·68, figure 4A). Leak frequencies were numerically higher for Fbn2A1052T/+ mice (15 [79%] of 19; RR 1·38, 95% CI 0·96–2·00; p=0·11), for Fbn2D1581V/+ mice (22 [76%] of 29; RR 1·33, 95% CI 0·93–1·89; p=0·12), and for Fbn2M2387T/+ mice (eight [73%] of 11; RR 1·27, 95% CI 0·80–2·02; p=0·36), but none of these differences was statistically significant (χ² tests).

The significance markers on Figure 4A for the three Fbn2 strains would need to be removed or replaced with "NS". The Abstract ("showed a predisposition for dural rupture"), the Research in context panel and the two Discussion passages describing increased leak frequency should be revised to match. For example, they could state that leak frequency was higher in all three Fbn2 strains but not significantly so, and rest any in-vivo claim on the pressure-drop results, if those are confirmed. Depositing per-mouse leak data would let readers verify the corrected analysis.

Recomputation script (python recompute.py.txt)
"""Recompute the in-vivo CSF-leak frequency comparisons in PMC13275636
(Lancet Neurol 2026; doi:10.1016/S1474-4422(26)00140-7) from the paper's own printed
counts, and redraw fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy, scipy >= 1.7, matplotlib)
Inputs are copied from the published article (Results, in-vivo paragraph; Methods,
Statistical analysis; Figure 4 legend).
"""
import os
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (Results)
WT = (20, 35)                      # "20 [57%] of 35" wild-type mice with leaks
GROUPS = [
    # label, (leaks, mice), reported RR, CI low, CI high, reported p, printed %
    ("Fbn2 A1052T/+", (15, 19), 1.38, 1.14, 1.69, "p<0.001 (Fig 4A: p<0.0001)", 79),
    ("Fbn2 D1581V/+", (22, 29), 1.33, 1.09, 1.63, "p<0.001 (Fig 4A: p<0.0001)", 76),
    ("Fbn2 M2387T/+", (8, 11), 1.27, 1.04, 1.58, "p<0.001 (Fig 4A: p<0.001)", 73),
    ("Fbn1 C1039G/+ (Marfan)", (7, 13), 0.94, 0.74, 1.22, "p>0.05", 54),
]
WT_PCT = 57
REPORTED_P_BOUND = {"Fbn2 A1052T/+": 1e-4, "Fbn2 D1581V/+": 1e-4, "Fbn2 M2387T/+": 1e-3}


def katz(a, n1, c, n0):
    """Relative risk with the usual log-scale (Katz) 95% CI."""
    rr = (a / n1) / (c / n0)
    se = np.sqrt(1 / a - 1 / n1 + 1 / c - 1 / n0)
    return rr, rr * np.exp(-Z * se), rr * np.exp(Z * se), se


def wald_linear(a, n1, c, n0):
    """Delta-method CI on the untransformed RR scale (an alternative some software uses)."""
    rr, _, _, se = katz(a, n1, c, n0)
    return rr - Z * rr * se, rr + Z * rr * se


def table(a, n1, c, n0):
    return np.array([[a, n1 - a], [c, n0 - c]])


def pvals(a, n1, c, n0):
    t = table(a, n1, c, n0)
    chi2_stat, p_chi, _, _ = stats.chi2_contingency(t, correction=False)
    p_yates = stats.chi2_contingency(t, correction=True)[1]
    p_fisher = stats.fisher_exact(t)[1]
    p_fisher_1s = stats.fisher_exact(t, alternative="greater")[1]
    p_barnard = stats.barnard_exact(t).pvalue
    return dict(chi2=chi2_stat, chi=p_chi, yates=p_yates, fisher=p_fisher,
                fisher1=p_fisher_1s, barnard=p_barnard)


print("=== 1. Percentages printed in the Results ===")
for lab, (a, n), *_rest in GROUPS:
    print(f"{lab:24s} {a}/{n} = {100*a/n:.1f}%  (printed {_rest[-1]}%)")
print(f"{'Wild-type':24s} {WT[0]}/{WT[1]} = {100*WT[0]/WT[1]:.1f}%  (printed {WT_PCT}%)")

print("\n=== 2. Each strain vs wild-type 20/35, with the paper's counts ===")
RES = {}
for lab, (a, n1), rr_r, lo_r, hi_r, p_r, _ in GROUPS:
    rr, lo, hi, se = katz(a, n1, *WT)
    wl, wh = wald_linear(a, n1, *WT)
    p = pvals(a, n1, *WT)
    RES[lab] = dict(rr=rr, lo=lo, hi=hi, se=se, **p)
    print(f"{lab}: {a}/{n1} vs {WT[0]}/{WT[1]}")
    print(f"   RR {rr:.3f}; log-scale 95% CI {lo:.2f}-{hi:.2f}; linear-scale 95% CI {wl:.2f}-{wh:.2f}"
          f"   | reported RR {rr_r} (95% CI {lo_r}-{hi_r})")
    print(f"   Pearson chi2 = {p['chi2']:.2f}, p = {p['chi']:.3f}; Yates p = {p['yates']:.3f}; "
          f"Fisher p = {p['fisher']:.3f}; one-sided Fisher p = {p['fisher1']:.3f}; "
          f"Barnard p = {p['barnard']:.3f}   | reported {p_r}")

print("\n=== 3. Width of the reported CIs vs the width the counts imply ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, _ in GROUPS:
    se_rep = (np.log(hi_r) - np.log(lo_r)) / (2 * Z)
    se_true = RES[lab]["se"]
    print(f"{lab:24s} SE(log RR) implied by reported CI = {se_rep:.3f}; from counts = {se_true:.3f}; "
          f"ratio {se_true/se_rep:.2f}; implied effective sample size inflation ~{(se_true/se_rep)**2:.1f}x")

print("\n=== 4. Diagnostic: percentages used as if they were counts out of 100 per arm ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, pct in GROUPS:
    rr, lo, hi, _ = katz(pct, 100, WT_PCT, 100)
    p = pvals(pct, 100, WT_PCT, 100)
    print(f"{lab:24s} {pct}/100 vs {WT_PCT}/100: RR {rr:.3f}, 95% CI {lo:.2f}-{hi:.2f} "
          f"(reported {lo_r}-{hi_r}); chi2 p = {p['chi']:.4f}, Fisher p = {p['fisher']:.4f}")
    if lab in REPORTED_P_BOUND:
        g = stats.chisquare([pct, 100 - pct], f_exp=[WT_PCT, 100 - WT_PCT])
        print(f"{'':24s} goodness-of-fit chi2 of {pct}% vs expected {WT_PCT}%, n=100: p = {g.pvalue:.1e}")

print("\n=== 5. Sensitivity: other comparisons that do not rescue p<0.001 ===")
a = sum(g[1][0] for g in GROUPS[:3]); n1 = sum(g[1][1] for g in GROUPS[:3])
rr, lo, hi, _ = katz(a, n1, *WT); p = pvals(a, n1, *WT)
print(f"All Fbn2 strains pooled {a}/{n1} vs WT 20/35: RR {rr:.2f} (95% CI {lo:.2f}-{hi:.2f}); "
      f"chi2 p = {p['chi']:.3f}; Fisher p = {p['fisher']:.3f}")
ctrl = (WT[0] + GROUPS[3][1][0], WT[1] + GROUPS[3][1][1])
print(f"Controls pooled (WT + Marfan) = {ctrl[0]}/{ctrl[1]}")
for lab, (a, n1), *_ in GROUPS[:3]:
    p = pvals(a, n1, *ctrl)
    print(f"   {lab} vs pooled controls: chi2 p = {p['chi']:.3f}, Fisher p = {p['fisher']:.3f}")
t = np.array([[g[1][0] for g in GROUPS] + [WT[0]],
              [g[1][1] - g[1][0] for g in GROUPS] + [WT[1] - WT[0]]])
c5 = stats.chi2_contingency(t)
print(f"Global 2x5 chi2 across all five strains: chi2 = {c5[0]:.2f}, df = {c5[2]}, p = {c5[1]:.3f}")

print("\n=== 6. Leak count needed for Pearson chi2 p<0.001 vs WT 20/35 ===")
for lab, (a, n1), *_ in GROUPS[:3]:
    need = [k for k in range(n1 + 1)
            if k / n1 > WT[0] / WT[1] and pvals(k, n1, *WT)["chi"] < 0.001]
    req = f"needs >= {need[0]}/{n1} leaks" if need else "impossible at any leak count"
    print(f"{lab:24s} n = {n1}: {req} (observed {a}); "
          f"if all {n1} leaked: p = {pvals(n1, n1, *WT)['chi']:.4f}")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.ticker import FixedLocator, FixedFormatter, NullLocator

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
                     "svg.fonttype": "none", "font.family": "DejaVu Sans"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
    fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
    plt.close(fig)


def style(ax):
    ax.set_facecolor(BG)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.xaxis.grid(True, color=GRID, lw=0.8, which="major")
    ax.set_axisbelow(True)


# ---- fig1: relative risk, reported vs recomputed 95% CI, log scale
labels = [f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in GROUPS]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
style(ax)
ys = np.arange(len(GROUPS))[::-1].astype(float)
off = 0.17
for y, (lab, (a, n1), rr_r, lo_r, hi_r, _, _) in zip(ys, GROUPS):
    r = RES[lab]
    ax.plot([lo_r, hi_r], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
    ax.plot(rr_r, y + off, "o", color=ORANGE, ms=8, zorder=3)
    ax.text(hi_r * 1.04, y + off, f"{lo_r:.2f}–{hi_r:.2f}", va="center", fontsize=10, color=ORANGE)
    ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
    ax.plot(r["rr"], y - off, "s", color=BLUE, ms=8, zorder=3)
    ax.text(r["hi"] * 1.04, y - off, f"{r['lo']:.2f}–{r['hi']:.2f}", va="center", fontsize=10,
            color=BLUE)
ax.axvline(1, color=SEC, lw=0.8)
ax.text(0.98, ys[0] + 0.55, "RR = 1 (no difference)", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
ticks = [0.5, 0.75, 1, 1.5, 2, 3]
ax.xaxis.set_major_locator(FixedLocator(ticks))
ax.xaxis.set_major_formatter(FixedFormatter([str(t) for t in ticks]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(0.45, 3.0)
ax.set_ylim(-0.6, len(GROUPS) - 0.2)
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Relative risk of CSF leak vs wild-type (log scale), with 95% CI")
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2, label="reported in Results")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=2, label="recomputed from the paper's counts")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.03), ncol=2, fontsize=10, frameon=False)
save(fig, "fig1")

# ---- fig2: p-values, reported bound vs recomputed, log scale
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
style(ax)
fb = GROUPS[:3]
ys2 = np.arange(len(fb))[::-1].astype(float)
for y, (lab, (a, n1), *_r) in zip(ys2, fb):
    r = RES[lab]
    b = REPORTED_P_BOUND[lab]
    ax.plot(1e-3, y + off, "<", color=ORANGE, ms=9, zorder=3)
    if b < 1e-3:
        ax.plot(b, y + off, "<", color=ORANGE, ms=9, mfc=BG, mew=2, zorder=3)
    ax.plot(r["chi"], y - off, "s", color=BLUE, ms=8, zorder=3)
    ax.plot(r["fisher"], y - off, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=3)
    ax.plot([r["chi"], r["fisher"]], [y - off] * 2, color=BLUE, lw=2, zorder=2)
    ax.text(max(r["chi"], r["fisher"]) * 1.25, y - off, f"{r['chi']:.2f} / {r['fisher']:.2f}",
            va="center", fontsize=10, color=BLUE)
ax.axvline(0.05, color=SEC, lw=0.8)
ax.text(0.045, ys2[0] + 0.5, "p = 0.05", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
pt = [1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1]
ax.xaxis.set_major_locator(FixedLocator(pt))
ax.xaxis.set_major_formatter(FixedFormatter(["0.00001", "0.0001", "0.001", "0.01", "0.05", "1"]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(5e-6, 1.6)
ax.set_ylim(-0.55, len(fb) - 0.25)
ax.set_yticks(ys2)
ax.set_yticklabels([f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in fb], fontsize=10)
ax.set_xlabel("p-value, mutant vs wild-type leak frequency (log scale)")
ax.plot([], [], "<", color=ORANGE, ms=9, label="reported: p<0.001 (Results text)")
ax.plot([], [], "<", color=ORANGE, ms=9, mfc=BG, mew=2, label="reported: p<0.0001 (Fig 4A legend)")
ax.plot([], [], "s", color=BLUE, ms=8, label="recomputed: Pearson χ² (named in Methods)")
ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2, label="recomputed: Fisher's exact")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)

The firearm death-in-ED estimate is reported as 88.82, 83.82, 77.63 and 379.00

Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2025.101343 · PMC12811418

Summary

The paper's headline mortality result is the adjusted relative risk (RR) of dying in the emergency department (ED) for women injured by firearms compared with women injured by other assault mechanisms. The same estimate appears with four different values: 88.82 ("89 times") in the abstract, 83.82 in Table 3, 77.63 in the Results text, and 379.00 in the Discussion. The paper's own numbers show where each value comes from. The abstract prints Table 3's confidence interval (72.38–97.06), and that interval is centred on 83.82, not 88.82. The Results paragraph that cites Table 3 reproduces the age 18–29 column of Table 4. The Discussion's 379.00 (266.15–539.7) is the odds ratio from the supplementary logistic regression, which the Methods say was set aside because it gave inflated estimates. All versions are far above 1, so the direction and significance of the association hold. But the size of the paper's key finding is misstated in the abstract, and the Discussion overstates it by a factor of about 4.5.

What the paper reports

The abstract:

Female patients who were injured by firearms had 89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm injury mechanisms.

— Abstract, Findings (the stray "6" is in the source, and in the PubMed abstract too)

Table 3, model (b), "firearm-related assault compared to all other adult female assault patients". The columns are all-assault RR, its CI, firearm RR, its CI:

In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06

— Table 3, Death: In ED row

The Results paragraph on the firearm model, which ends by citing Table 3:

Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; 95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic (RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater risk of firearm injury (Table 3).

— Results, paragraph 4 ("In the firearm injury model for women…")

Table 4 gives the age-stratified models. In the In ED row, the second estimate is the 18–29 y firearm model:

In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)

— Table 4, Death: In ED row (first two of eight columns)

The same 18–29 y value restated:

There appeared to be a bimodal distribution of increased mortality for firearm injuries for 18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; 95% CI 2.57–4.22)

— Results, paragraph 5

The Discussion:

Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients.

— Discussion, paragraph 1

On the logistic model:

We initially employed a survey-weighted binary logistic regression model. However, several results were affected by sparse-data bias and quasi-complete separation, leading to inflated odds ratio estimates.

The results from the initial survey-weighted logistic regression are provided in the Supplement for comparison.

— Methods, statistical analysis

The initial logistic regression was affected by sparse-data bias and quasi-complete separation, which inflated the odds ratios for several variables.

— Discussion, final paragraph

Supplementary Table 1 ("Adjusted Logistic Regression Model…", file mmc1.docx), firearm model, Death row: "ED | 1.04 | 0.89 – 1.21 | 379.00 | 266.15 – 539.71".

Every quotation from the article was checked character for character against the text archived in PubMed Central (src/show.py PMC12811418). Each occurs exactly once, and recompute.py.txt reruns that check. The supplementary row is transcribed from the publisher's mmc1.docx. Paragraphs are numbered from 1 under the Results and Discussion headings.

What the paper's own numbers show

Method. The model is a survey-weighted Poisson GLM with a log link (Methods; Table 3 caption), so its 95% intervals are Wald intervals on the log scale: L = exp(b − 1.96·SE) and U = exp(b + 1.96·SE). The point estimate that belongs with a printed interval is therefore its geometric midpoint, exp(b) = √(L·U). Comparing each printed value with √(L·U), and then searching the paper's tables for the matching cell, shows which numbers belong together. No outside data are used. The underlying NEDS microdata are licensed and not public, so the model itself cannot be refitted. The check is limited to the internal consistency of the reported values.

Quantity Location Reported Recomputed from the paper's own numbers
Firearm vs other assault, death in ED, all ages Table 3 (b) 83.82 (72.38, 97.06) √(72.38 × 97.06) = 83.816 → 83.82. Consistent (log distance 0.147 to each limit)
Same estimate Abstract "89 times", RR = 88.82 (72.38–97.06) Same CI as Table 3, whose centre is 83.82. 88.82 is off-centre (log distance 0.205 to L, 0.089 to U), 0.77 SE above the centre, and 6.0% above Table 3. "89" is round(88.82); round(83.82) = 84
Same estimate, "firearm injury model … (Table 3)" Results para 4 77.63 (62.60–96.27) √(62.60 × 96.27) = 77.631. Identical to Table 4, 18–29 y firearm column; 7.4% below Table 3
Same paragraph: death in hospital, admitted, Black, Hispanic, Other Results para 4 3.29 (2.57–4.22); 19.94 (18.04–22.02); 4.15 (3.61–4.78); 1.77 (1.48–2.13); 2.01 (1.51–2.68) Table 3 gives 3.34 (2.79, 4.00); 16.50 (15.29, 17.80); 4.12 (3.75, 4.52); 1.87 (1.64, 2.12); 1.90 (1.55, 2.33); none of these 15 numbers agree. Table 4 18–29 y gives 3.29 (2.57, 4.22); 19.94 (18.05, 22.02); 4.16 (3.61, 4.78); 1.78 (1.48, 2.14); 2.01 (1.51, 2.68). Across all six outcome and race rows, 14 of 18 numbers are identical to Table 4 18–29 y and the other 4 are within 0.01
Same paragraph: age Results para 4 0.98 (0.97–0.98) Matches Table 3 (0.98, 0.97–0.98), not Table 4 18–29 y (0.96, 0.95–0.98). This is the only row of the paragraph that comes from Table 3
18–29 y firearm, death in ED Results para 5 77.65 (62.60–96.27) Table 4 gives 77.63; √(L·U) = 77.631
Firearm death in ED, "times higher risk" Discussion para 1 379.00 (266.15–539.7) √(266.15 × 539.7) = 379.00. Equal to Suppl. Table 1 logistic adjusted odds ratio 379.00 (266.15–539.71). It appears in no RR table and is 4.52 × Table 3's RR
Firearm model, Black vs White Discussion para 1, "nearly five times" — Table 3 RR = 4.12 (3.75, 4.52), as in the abstract. Suppl. Table 1 OR = 4.74 (4.29–5.24). "Nearly five" fits the odds ratio
All-assault model, Native American vs White Abstract RR = 2.81 (2.67–2.97), no age group stated Equal to Table 4, 18–29 y column (stated as such in Results para 7). The all-ages Table 3 value is 2.68 (2.60, 2.77)
All-assault model, 18–29 y, death in ED Results para 5 3.14 (2.45–4.20) Table 4 gives 3.14 (2.45, 4.02). √(2.45 × 4.02) = 3.138 fits 3.14, while √(2.45 × 4.20) = 3.208 does not, so 4.20 is a transposition of 4.02

So the one value that is internally consistent is Table 3's 83.82 (72.38–97.06). The abstract pairs a different point estimate with Table 3's interval. The two differ in a single digit (88.82 vs 83.82), which looks like a typographical slip. The Results paragraph on the firearm model reports the 18–29 y stratum of Table 4 instead of the all-ages model it cites. The Discussion reports the logistic odds ratio as a risk. The last three rows of the table are minor slips of the same kind and are listed for completeness.

Figure

Dot-and-interval plot on a log scale of the six printed versions of the firearm death-in-ED estimate, each with the geometric centre of its printed CI, and a zoom panel on the abstract and Table 3 rows

Figure 1. The printed versions of one estimate, and the centres of their printed intervals. Panel A shows each printed value (orange dot) with its printed 95% CI (orange line), on a logarithmic axis that runs from the null value (thin line at 1) to 1000. Next to it is the geometric centre of that CI, √(L×U), recomputed (blue open diamond). The Results para 4 value equals the age 18–29 y value in Table 4. The Discussion value equals the logistic odds ratio in Supplementary Table S1 and lies about 4.5 times above the Poisson estimates. Panel B zooms in (log axis, 70–100) on the two rows that print the same interval, 72.38–97.06. Table 3's 83.82 sits at the centre (log distance 0.147 to each limit). The abstract's 88.82 does not (0.205 and 0.089). All intervals lie far from 1.

Table-style diagram comparing the seven estimates quoted in Results paragraph 4 with the all-ages firearm column of Table 3 and the 18–29 y firearm column of Table 4

Figure 2. Results paragraph 4 against the two table columns it could come from. The left column (orange) holds the values printed in the Results paragraph that cites Table 3. The middle and right columns hold the firearm-model values printed in Table 3 (all ages) and Table 4 (age 18–29 y). A cell is blue and bold when all three of its numbers (estimate and both limits) equal the Results-text value to within 0.01, and grey otherwise. For death in the ED, death in hospital, admission and the three race contrasts, the Results text matches Table 4's 18–29 y column (14 of 18 numbers identical, the rest within 0.01) and none of Table 3's 18 numbers. Only the age coefficient matches Table 3.

Why it matters

Explanations considered

  1. The abstract reports a different model or population than Table 3. This does not work. The abstract's comparison ("compared to female patients injured by non-firearm injury mechanisms") is Table 3 model (b), and it prints exactly Table 3's interval, 72.38–97.06. No estimate of 88.82 appears in any table or in the supplement. A point estimate of 88.82 cannot come from a log-scale Wald interval of 72.38–97.06, because it lies 0.77 standard errors from the centre. The abstract's own interval therefore implies 83.82.
  2. A rounding or significant-figure difference. This does not work. 88.82 and 83.82 differ by 5.00 (6%), and "89 times" rounds 88.82, not 83.82 (84).
  3. Results paragraph 4 deliberately describes the youngest age group. This does not work. The paragraph opens "In the firearm injury model for women", gives no age group, and ends "(Table 3)". Its age coefficient (0.98, 0.97–0.98) is the all-ages Table 3 value; within a single age stratum the coefficient differs (0.96 for 18–29 y). The age-stratified results are introduced only in the next paragraph ("We re-ran 8 survey-weighted Poisson regression models by age category … in Table 4"). That paragraph restates the same 18–29 y firearm values, which confirms where they came from.
  4. The Discussion intentionally quotes the logistic model. The value is indeed the logistic odds ratio (Supplementary Table 1: 379.00, 266.15–539.71). But the sentence calls it "times higher risk" without saying it is an odds ratio from the supplementary model, and it sits in the summary of the paper's own findings. The Methods say that model gave inflated odds ratios and was replaced. The comparator in the sentence ("all-cause female assault patients") also differs from the model's comparator (assault by all other mechanisms). So even read as intentional, the sentence misdescribes the measure and does not report the paper's primary estimate.
  5. A conversion or typesetting artifact in PMC. This is ruled out. The same values appear in the Europe PMC full-text JATS XML (each of "RR = 88.82", "RR = 77.63", "RR = 77.65", "379.00" and "83.82" exactly once) and in the PubMed abstract (PMID 41551920), which also carries the stray "95% CI 6 =". The table cells line up with their headers, and no minus signs, superscripts or decimal commas are involved.
  6. A table rather than the text is wrong. This is unlikely. Each of the 15 Table 3 and Table 4 cells examined sits at the geometric centre of its own interval to within 0.006 (rounding), against 5.00 for the abstract's 88.82. The textual values match specific cells of other tables (Table 4 18–29 y; Supplementary Table 1). The only value with no source in any table is the abstract's 88.82, and its interval is Table 3's.

None of these explanations reconciles the four values. The consistent reading is that Table 3 holds the primary estimate, 83.82 (72.38–97.06), and that the abstract, the Results paragraph and the Discussion each misreport it in a different way.

Status

As of 2026-10-23:

Suggested correction

  1. Abstract, Findings: replace "89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06)" with "84 times higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06)", matching Table 3.
  2. Results, paragraph 4: replace the age 18–29 y values with the all-ages Table 3 values: death in ED RR = 83.82 (72.38–97.06), death in hospital RR = 3.34 (2.79–4.00), admitted RR = 16.50 (15.29–17.80), Black RR = 4.12 (3.75–4.52), Hispanic RR = 1.87 (1.64–2.12), Other RR = 1.90 (1.55–2.33). Alternatively, label them explicitly as the 18–29 y estimates and cite Table 4.
  3. Results, paragraph 5: change "RR = 77.65" to "RR = 77.63", and change the upper limit of the 18–29 y all-assault death-in-ED interval from "4.20" to "4.02", matching Table 4.
  4. Discussion, paragraph 1: replace "379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients" with the primary Poisson estimate: "an 84-fold higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06) compared to female patients injured by other assault mechanisms". If the logistic estimate is kept, label it as an odds ratio from the supplementary model. Similarly, "nearly five times more likely to identify as Black" should read about four times (RR = 4.12; 95% CI 3.75–4.52), matching the abstract and Table 3.
  5. Abstract, Native American estimate (optional): either state that RR = 2.81 (2.67–2.97) refers to women aged 18–29 y, or give the all-ages Table 3 value, RR = 2.68 (2.60–2.77).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12811418 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2025.101343)
# One adjusted estimate (firearm vs other assault mechanisms, death in the ED) is printed
# with four different values in the abstract, Table 3, the Results text and the Discussion.
#
# Run:  python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs (all transcribed from the paper; nothing is estimated from outside data)
#  * Text quotes and Tables 3 and 4: the article as archived in PubMed Central (PMC12811418).
#    When run inside the /home/ubuntu/papers repo, the script re-checks that every quote
#    occurs exactly once in the archived full text via src/show.py (skipped elsewhere).
#  * Supplementary Tables S1/S2: the article's only supplementary file, mmc1.docx
#    ("Supplementary Tables"), from
#    https://ars.els-cdn.com/content/image/1-s2.0-S2667193X25003540-mmc1.docx
#    These are the survey-weighted LOGISTIC regression results (adjusted odds ratios), which
#    the Methods say were set aside as inflated by sparse-data bias / quasi-complete separation.
#
# Method. For a ratio estimate with a Wald interval on the log scale,
#   L = exp(b - z*SE), U = exp(b + z*SE)  =>  exp(b) = sqrt(L*U).
# So the point estimate that belongs with a printed interval is the interval's geometric
# midpoint. Comparing each printed point estimate with sqrt(L*U) identifies which printed
# values belong together, and the matching table cell identifies where each value came from.

import math, os, subprocess

import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC12811418"

# ------------------------------------------------------------------ quotes (verbatim)
QUOTES = [
    ("Abstract, Findings",
     "Female patients who were injured by firearms had 89 times higher risk of dying in the ED "
     "(RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm "
     "injury mechanisms."),
    ("Abstract, Findings",
     "Racial disparities were prevalent, with Native American women experiencing the greatest risk "
     "of being assaulted (RR = 2.81; 95% CI 2.67–2.97)."),
    ("Abstract, Findings",
     "Victims of firearm related assaults had nearly 4.12 times the risk of identifying as Black "
     "compared with those assaulted without firearms (95% CI 3.75–4.52)."),
    ("Methods, statistical analysis",
     "We initially employed a survey-weighted binary logistic regression model. However, several "
     "results were affected by sparse-data bias and quasi-complete separation, leading to inflated "
     "odds ratio estimates."),
    ("Methods, statistical analysis",
     "The results from the initial survey-weighted logistic regression are provided in the "
     "Supplement for comparison."),
    ("Results, paragraph 4 (firearm injury model)",
     "Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; "
     "95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially "
     "higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to "
     "White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic "
     "(RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater "
     "risk of firearm injury (Table 3)."),
    ("Results, paragraph 5 (age strata, Table 4)",
     "There appeared to be a bimodal distribution of increased mortality for firearm injuries for "
     "18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; "
     "95% CI 2.57–4.22)"),
    ("Results, paragraph 5 (age strata, Table 4)",
     "Female patients 18–29 years old, had the greatest risk of death, whether in the ED "
     "(RR = 3.14; 95% CI 2.45–4.20)"),
    ("Discussion, paragraph 1",
     "Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) "
     "times higher risk of dying in the ED compared to all-cause female assault patients."),
    ("Discussion, paragraph 1",
     "victims of firearm-related assaults nearly five times more likely to identify as Black than "
     "those assaulted without firearms"),
    ("Discussion, final paragraph",
     "The initial logistic regression was affected by sparse-data bias and quasi-complete "
     "separation, which inflated the odds ratios for several variables."),
    ("Table 3, Death: In ED row (all-assault RR, CI, firearm RR, CI)",
     "In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06"),
    ("Table 4, Death: In ED row (18-29 firearm = 2nd estimate)",
     "In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)"),
]

def check_quotes():
    repo = "/home/ubuntu/papers"
    py = "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.\n")
        return
    full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
                          capture_output=True, text=True).stdout
    print("Quote check against the PMC-archived text (occurrences; must be 1):")
    for loc, q in QUOTES:
        n = full.count(q)
        print(f"  [{'ok ' if n == 1 else 'BAD'}] {n}  {loc}: {q[:70]}...")
    print()

# ------------------------------------------------------------------ the estimates
# (key, label, location, measure, point, lo, hi)
ED_DEATH = [
    ("abs",  "Abstract",                        "Abstract, Findings",              "RR",  88.82, 72.38, 97.06),
    ("t3",   "Table 3 (b), all ages",           "Table 3, Death: In ED",           "RR",  83.82, 72.38, 97.06),
    ("rp3",  "Results p4 (cites Table 3)",      "Results para 4",                  "RR",  77.63, 62.60, 96.27),
    ("t4",   "Table 4, firearm, 18-29 y",       "Table 4, Death: In ED",           "RR",  77.63, 62.60, 96.27),
    ("rp4",  "Results p5, 18-29 y",             "Results para 5",                  "RR",  77.65, 62.60, 96.27),
    ("disc", "Discussion",                      "Discussion para 1",               "'risk'", 379.00, 266.15, 539.7),
    ("s1",   "Suppl. Table S1 (logistic OR)",   "Suppl. Table S1, Death: ED",      "OR", 379.00, 266.15, 539.71),
]
# Other age strata, Table 4 firearm In ED (for context in the figure)
T4_OTHER = [("30-39 y", 78.40, 59.30, 103.65), ("40-49 y", 92.63, 66.98, 128.10),
            ("50+ y", 110.20, 70.04, 173.38)]

def gmid(lo, hi):
    return math.sqrt(lo * hi)

def logasym(p, lo, hi):
    # (log U - log p) - (log p - log L); 0 when p is the geometric midpoint
    return (math.log(hi) - math.log(p)) - (math.log(p) - math.log(lo))

def part1():
    print("1. Death in the ED, firearm vs other assault mechanisms: printed value vs geometric "
          "midpoint of its printed 95% CI")
    print(f"   {'source':34s} {'meas':6s} {'printed':>8s} {'CI':>17s} {'sqrt(L*U)':>10s} "
          f"{'log-asym':>9s}")
    for k, lab, loc, m, p, lo, hi in ED_DEATH:
        print(f"   {lab:34s} {m:6s} {p:8.2f} {lo:8.2f}-{hi:<8.2f} {gmid(lo, hi):10.3f} "
              f"{logasym(p, lo, hi):9.4f}")
    p, lo, hi = 88.82, 72.38, 97.06
    se = (math.log(hi) - math.log(lo)) / (2 * 1.959964)
    print(f"   Abstract CI: log-distance from 88.82 to L = {math.log(p/lo):.4f}, to U = "
          f"{math.log(hi/p):.4f}; from 83.82: {math.log(83.82/lo):.4f} and {math.log(hi/83.82):.4f}")
    print(f"   Implied SE(log RR) = {se:.4f}; 88.82 lies {(math.log(p)-math.log(gmid(lo,hi)))/se:.2f} "
          f"SE above the CI centre")
    print(f"   88.82 / 83.82 = {88.82/83.82:.4f}  (abstract {100*(88.82/83.82-1):.1f}% above Table 3)")
    print(f"   '89 times' = round(88.82) = {round(88.82)};  round(83.82) = {round(83.82)}")
    print(f"   77.63 / 83.82 = {77.63/83.82:.4f}  (Results p4 {100*(1-77.63/83.82):.1f}% below Table 3)")
    print(f"   379.00 / 83.82 = {379.00/83.82:.2f}  (Discussion vs Table 3)")
    print(f"   Abstract 88.82 and Table 3 83.82 differ in one digit (8 vs 3, second digit)")
    print()

# Results p4 values vs Table 3 firearm column and Table 4 18-29 firearm column
P3_ROWS = [
    # quantity, Results p4, Table 3 firearm (all ages), Table 4 firearm 18-29
    ("Death in ED",      (77.63, 62.60, 96.27), (83.82, 72.38, 97.06), (77.63, 62.60, 96.27)),
    ("Death in hospital",(3.29, 2.57, 4.22),    (3.34, 2.79, 4.00),    (3.29, 2.57, 4.22)),
    ("Admitted",         (19.94, 18.04, 22.02), (16.50, 15.29, 17.80), (19.94, 18.05, 22.02)),
    ("Black",            (4.15, 3.61, 4.78),    (4.12, 3.75, 4.52),    (4.16, 3.61, 4.78)),
    ("Hispanic",         (1.77, 1.48, 2.13),    (1.87, 1.64, 2.12),    (1.78, 1.48, 2.14)),
    ("Other race",       (2.01, 1.51, 2.68),    (1.90, 1.55, 2.33),    (2.01, 1.51, 2.68)),
    ("Age (per year)",   (0.98, 0.97, 0.98),    (0.98, 0.97, 0.98),    (0.96, 0.95, 0.98)),
]

def part2():
    print("2. Results paragraph 4 ('firearm injury model ... (Table 3)'): which table column do its "
          "numbers match?")
    print(f"   {'quantity':18s} {'Results p4':>20s} {'Table 3 firearm':>20s} {'Table 4 18-29':>20s}  "
          f"exact(T3) exact(T4) within0.01(T4)")
    tot3 = tot4 = near4 = n = 0
    for q, r, t3, t4 in P3_ROWS:
        e3 = sum(a == b for a, b in zip(r, t3)); e4 = sum(a == b for a, b in zip(r, t4))
        w4 = sum(abs(a - b) <= 0.0101 for a, b in zip(r, t4))
        fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
        print(f"   {q:18s} {fmt(r):>20s} {fmt(t3):>20s} {fmt(t4):>20s}  {e3:>8d}/3 {e4:>8d}/3 {w4:>8d}/3")
        if q != "Age (per year)":
            tot3 += e3; tot4 += e4; near4 += w4; n += 3
    print(f"   Six outcome/race rows (18 numbers): exact match Table 3 = {tot3}/{n}, "
          f"Table 4 18-29 = {tot4}/{n}, within 0.01 of Table 4 18-29 = {near4}/{n}")
    print("   Age row matches Table 3 (0.98, 0.97-0.98), not Table 4 18-29 (0.96, 0.95-0.98).")
    print()

def part3():
    print("3. Other values checked")
    # Discussion 'nearly five times ... Black' vs RR and OR
    print(f"   Black, firearm model: Table 3 RR 4.12 (3.75-4.52), centre {gmid(3.75,4.52):.3f}; "
          f"Suppl. S1 OR 4.74 (4.29-5.24), centre {gmid(4.29,5.24):.3f}; "
          f"Discussion 'nearly five times' is closer to the OR (5 - 4.74 = {5-4.74:.2f}; 5 - 4.12 = {5-4.12:.2f})")
    # Abstract Native American 2.81
    print(f"   Native American, all-assault model: abstract 2.81 (2.67-2.97), centre "
          f"{gmid(2.67,2.97):.3f}; Table 4 18-29 = 2.81 (2.67, 2.97); Table 3 all ages = 2.68 (2.60, 2.77), "
          f"centre {gmid(2.60,2.77):.3f}")
    # Results p5: 18-29 all-assault death in ED, upper limit 4.20 vs 4.02
    print(f"   18-29 all-assault death in ED: Results p5 3.14 (2.45-4.20), centre {gmid(2.45,4.20):.3f}; "
          f"Table 4 3.14 (2.45, 4.02), centre {gmid(2.45,4.02):.3f}")
    # Results p5 18-29 firearm in hospital matches Table 4
    print(f"   18-29 firearm in hospital: Results p5 3.29 (2.57-4.22) = Table 4; centre {gmid(2.57,4.22):.3f}")
    # admission: Results text vs Table 3
    print(f"   Admitted: Results 19.94 / Table 3 16.50 = {19.94/16.50:.4f} ({100*(19.94/16.50-1):.0f}% higher)")
    # internal consistency of the table cells used here: printed estimate vs CI centre
    cells = [("T3 In ED", 83.82, 72.38, 97.06), ("T3 In hospital", 3.34, 2.79, 4.00),
             ("T3 Admit", 16.50, 15.29, 17.80), ("T3 Black", 4.12, 3.75, 4.52),
             ("T3 Hispanic", 1.87, 1.64, 2.12), ("T3 Other", 1.90, 1.55, 2.33),
             ("T4 18-29 In ED", 77.63, 62.60, 96.27), ("T4 18-29 In hospital", 3.29, 2.57, 4.22),
             ("T4 18-29 Admit", 19.94, 18.05, 22.02), ("T4 18-29 Black", 4.16, 3.61, 4.78),
             ("T4 18-29 Hispanic", 1.78, 1.48, 2.14), ("T4 18-29 Other", 2.01, 1.51, 2.68),
             ("T4 30-39 In ED", 78.40, 59.30, 103.65), ("T4 40-49 In ED", 92.63, 66.98, 128.10),
             ("T4 50+ In ED", 110.20, 70.04, 173.38)]
    worst = max(abs(p - gmid(lo, hi)) for _, p, lo, hi in cells)
    print("   Table cells used here, printed vs sqrt(L*U): " +
          "; ".join(f"{k} {p:.2f}/{gmid(lo, hi):.2f}" for k, p, lo, hi in cells))
    print(f"   Largest |printed - sqrt(L*U)| among these table cells = {worst:.3f} "
          f"(abstract 88.82 vs 83.82: {88.82 - gmid(72.38, 97.06):.3f})")
    # every version excludes 1
    print("   Lower CI limit of every version of the firearm ED-death estimate > 1: "
          + str(all(lo > 1 for *_, lo, hi in ED_DEATH)))
    print()

# ------------------------------------------------------------------ figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
                     "axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
                     "svg.fonttype": "none"})

def fig1():
    rows = [  # label, sublabel, point, lo, hi
        ("Abstract", "“89 times”, RR = 88.82", 88.82, 72.38, 97.06),
        ("Table 3 (b), all ages", "Poisson RR = 83.82", 83.82, 72.38, 97.06),
        ("Results para 4", "“firearm injury model … (Table 3)”", 77.63, 62.60, 96.27),
        ("Table 4, age 18–29 y only", "Poisson RR = 77.63", 77.63, 62.60, 96.27),
        ("Discussion para 1", "“379.00 … times higher risk”", 379.00, 266.15, 539.7),
        ("Suppl. Table S1", "logistic odds ratio = 379.00", 379.00, 266.15, 539.71),
    ]
    from matplotlib.lines import Line2D
    fig, (ax, zx) = plt.subplots(2, 1, figsize=(7.2, 6.6), facecolor=BG,
                                 gridspec_kw=dict(height_ratios=[6, 1.9], hspace=0.55))
    ax.set_facecolor(BG); zx.set_facecolor(BG)
    ys = [5 - i for i in range(len(rows))]
    off = 0.24   # recomputed centre drawn just below the printed value
    for y, (lab, sub, p, lo, hi) in zip(ys, rows):
        ax.plot([lo, hi], [y + 0.08, y + 0.08], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
        ax.plot([p], [y + 0.08], "o", ms=8, color=ORANGE, zorder=4)
        g = gmid(lo, hi)
        ax.plot([g], [y - off], marker="D", ms=8, mfc="none", mec=BLUE, mew=2, zorder=5, ls="none")
        his = "539.7" if hi == 539.7 else f"{hi:.2f}"   # Discussion prints 539.7
        ax.text(1.01, y + 0.08, f"{p:.2f} ({lo:.2f}–{his})", transform=ax.get_yaxis_transform(),
                va="center", fontsize=10, color=ORANGE)
        ax.text(1.01, y - off, f"{g:.2f}", transform=ax.get_yaxis_transform(),
                va="center", fontsize=10, color=BLUE)
    ax.axvline(1, color=SEC, lw=0.8, zorder=1)
    ax.text(1.07, -0.65, "null (ratio = 1)", fontsize=10, color=SEC, va="center")
    ax.set_xscale("log")
    ax.set_xlim(0.9, 1000)
    ax.set_ylim(-0.95, 5.55)
    ax.set_xticks([1, 10, 100, 1000]); ax.set_xticklabels(["1", "10", "100", "1000"])
    ax.set_yticks(ys)
    ax.set_yticklabels([f"{lab}\n{sub}" for lab, sub, *_ in rows], fontsize=10)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    for sp in ("top", "right", "left"):
        ax.spines[sp].set_visible(False)
    ax.spines["bottom"].set_color(SEC)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Firearm vs other assault mechanisms, death in the ED (ratio, log scale)")
    ax.set_title("A. All printed versions of the estimate (full range, null at 1)", loc="left",
                 fontsize=10.5, color=TXT, x=-0.6)
    # panel B: zoom on the two rows that print the same CI
    for yy, p, lab in ((1, 88.82, "Abstract\nRR = 88.82"), (0, 83.82, "Table 3 (b)\nRR = 83.82")):
        zx.plot([72.38, 97.06], [yy + 0.0, yy], color=ORANGE, lw=2, solid_capstyle="butt")
        for e in (72.38, 97.06):
            zx.plot([e, e], [yy - 0.15, yy + 0.15], color=ORANGE, lw=2)
        zx.plot([p], [yy], "o", ms=8, color=ORANGE, zorder=4)
        zx.plot([gmid(72.38, 97.06)], [yy], marker="D", ms=9, mfc="none", mec=BLUE, mew=2,
                ls="none", zorder=5)
    zx.set_yticks([1, 0]); zx.set_yticklabels(["Abstract\nRR = 88.82", "Table 3 (b)\nRR = 83.82"],
                                              fontsize=10)
    zx.set_xscale("log"); zx.set_xlim(70, 100); zx.set_ylim(-0.5, 1.5)
    zx.set_xticks([72.38, 83.82, 88.82, 97.06])
    zx.set_xticklabels(["72.38", "83.82", "88.82", "97.06"], fontsize=10)
    zx.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
    zx.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    for sp in ("top", "right", "left"):
        zx.spines[sp].set_visible(False)
    zx.spines["bottom"].set_color(SEC); zx.tick_params(axis="y", length=0)
    zx.set_title("B. Zoom (log scale, 70–100): one printed CI, two printed point estimates",
                 loc="left", fontsize=10.5, color=TXT, x=-0.6)
    zx.text(1.01, 1, f"log distance\nto L {math.log(88.82/72.38):.3f}, to U {math.log(97.06/88.82):.3f}",
            transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
    zx.text(1.01, 0, f"log distance\nto L {math.log(83.82/72.38):.3f}, to U {math.log(97.06/83.82):.3f}",
            transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
    h = [Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8,
                label="value printed in the paper, with its 95% CI"),
         Line2D([0], [0], ls="none", marker="D", ms=8, mfc="none", mec=BLUE, mew=2,
                label="recomputed: geometric centre of that CI, √(L×U)")]
    fig.legend(handles=h, loc="lower left", bbox_to_anchor=(0.2, 0.0), ncol=1, frameon=False,
               fontsize=10, columnspacing=1.2, handletextpad=0.5)
    fig.subplots_adjust(left=0.36, right=0.75, top=0.95, bottom=0.12)
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)

def fig2():
    fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(-0.3, 4.7); ax.axis("off")
    ax.text(0.15, 4.5, "Results paragraph 4 says it reports the all-ages firearm model (Table 3).",
            fontsize=11, fontweight="bold", va="center")
    ax.text(0.15, 4.2, "Its numbers are those of the age 18–29 y column of Table 4.",
            fontsize=11, color=SEC, va="center")
    cols = [(2.55, "Results para 4\n(text, “Table 3”)", ORANGE),
            (4.15, "Table 3 (b)\nfirearm, all ages", TXT),
            (5.75, "Table 4 (b)\nfirearm, 18–29 y", TXT)]
    top = 3.62
    for x, h, c in cols:
        ax.text(x, top, h, ha="center", va="center", fontsize=10, fontweight="bold", color=c,
                linespacing=1.25)
    rows = [r for r in P3_ROWS]
    dy = 0.39
    y0 = top - 0.55
    fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
    for i, (q, r, t3, t4) in enumerate(rows):
        y = y0 - i * dy
        if i % 2 == 0:
            ax.add_patch(FancyBboxPatch((0.12, y - dy / 2 + 0.02), 6.96, dy - 0.04,
                                        boxstyle="round,pad=0,rounding_size=0.04",
                                        fc="#f1f0ec", ec="none", zorder=0))
        ax.text(0.2, y, q, va="center", fontsize=10)
        ax.text(cols[0][0], y, fmt(r), ha="center", va="center", fontsize=10, color=ORANGE,
                fontweight="bold")
        for (x, _, _), t in ((cols[1], t3), (cols[2], t4)):
            same = all(abs(a - b) <= 0.0101 for a, b in zip(r, t))
            ax.text(x, y, fmt(t), ha="center", va="center", fontsize=10,
                    color=BLUE if same else SEC, fontweight="bold" if same else "normal")
    yb = y0 - len(rows) * dy + 0.05
    six = [r for r in P3_ROWS if r[0] != "Age (per year)"]
    e4 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t4))
    e3 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t3))
    ax.text(0.2, yb - 0.08,
            "Blue bold = equal to the Results-text value to within 0.01 on all three numbers;  grey = differs.\n"
            f"Six outcome and race rows: {e4} of 18 numbers equal Table 4 18–29 y, the other {18-e4} within 0.01;\n"
            f"{e3} of 18 identical to Table 3. Only the age row (last) matches Table 3, not Table 4 18–29 y.",
            fontsize=10, color=SEC, va="top", linespacing=1.35)
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)

if __name__ == "__main__":
    check_quotes()
    part1()
    part2()
    part3()
    fig1()
    fig2()
    print("Figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))

Abstract's adjusted odds ratios for education and prenatal care differ from the paper's own Results

Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101434 · PMC12995907

Summary

The Abstract reports that higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with documented adequate partner treatment. The paper's national multivariable model, given in the Results text and in Fig. 2, has different values for the same two associations: 1·48 (1·39–1·59) for higher vs primary education and 3·05 (2·92–3·20) for prenatal care. In both cases the two intervals do not overlap. The race estimates in the same Abstract sentence match the Results exactly. Neither Abstract value appears anywhere else in the paper, and the Abstract's education interval is too narrow to be possible for a higher-vs-primary contrast given the Table 1 counts. Both associations keep their direction and statistical significance, but the Abstract overstates the education association, understates the prenatal-care association, and presents the two as about equal, while the Results show prenatal care as much the stronger of the two.

What the paper reports

Abstract, Findings:

Higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with partner treatment, whereas odds were lower among Black (aOR 0·85, 95% CI 0·82–0·89) and Mixed women (aOR 0·81, 95% CI 0·79–0·84) and in the North region.

Abstract, Methods:

Multivariable logistic regression estimated adjusted odds ratios (aORs) and 95% CIs.

Results, paragraph on the national multivariable model:

Multivariable analysis included 214,259 participants, with 4069 exclusions due to missing maternal age. Prenatal care was the factor most strongly associated with adequate partner treatment (adjusted odds ratio [aOR] 3·05, 95% CI 2·92–3·20; p < 0·001).

Higher maternal education remained positively associated with partner treatment (higher vs primary: aOR 1·48, 1·39–1·59; secondary vs primary: aOR 1·23, 1·20–1·27; both p < 0·001)

Compared with White women, lower odds persisted among Black (aOR 0·85, 0·82–0·89; p < 0·001) and Mixed women (aOR 0·81, 0·79–0·84; p < 0·001)

Fig. 2 caption:

Unadjusted ORs are from separate univariable logistic regression models; aORs are from the multivariable logistic regression model including maternal age, race, education, prenatal care, and macro-region.

Fig. 2 itself is an image. Read from the figure file (gr2.jpg): pre-natal care "Yes", unadjusted 3·33 (3·22 – 3·46) and adjusted 3·05 (2·92 – 3·20); "Higher education", unadjusted 1·85 (1·73 – 1·97) and adjusted 1·48 (1·39 – 1·59). Fig. 3 (gr3.jpg), the region-stratified models, gives prenatal care "Yes" aORs of 2·11, 2·58, 3·70, 4·50 and 4·87 and higher-education aORs of 1·39, 1·72, 1·48, 1·38 and 1·41 (North, Northeast, Central-West, Southeast, South).

Discussion, second paragraph:

Prenatal care emerged as the strongest predictor of partner treatment, with approximately threefold higher odds compared with women without prenatal follow-up.

Table 1 (columns: all cases, n = 218,328 | adequate partner treatment, n = 50,609 | no adequate partner treatment, n = 167,719 | p-value), education and prenatal-care rows:

Primary | 94,267 (43·2%) | 19,367 (38·3%) | 74,900 (44·7%) | <0·001

Higher | 4427 (2·03%) | 1431 (2·83%) | 2996 (1·79%) | <0·001

No | 32,162 (14·7%) | 2627 (5·2%) | 29,535 (17·6%) | <0·001

Yes | 180,009 (82·4%) | 46,943 (92·8%) | 133,066 (79·3%) | <0·001

What the paper's own numbers show

Method. I compared each Abstract estimate with every other estimate of the same association in the paper: the Results text, Fig. 2 (adjusted and unadjusted) and Fig. 3 (five regional models). From the Table 1 counts I computed crude odds ratios with Woolf 95% CIs. From each printed 95% CI I recovered the implied standard error of the log odds ratio, SE = ln(upper/lower)/(2 × 1·96), together with the largest SE the two-decimal rounding allows. I then compared those SEs with a lower bound that holds for any maximum-likelihood logistic model fitted to these cases with model-based standard errors. When the model contains the levels of a categorical variable, the fitted probabilities within each level sum to the observed number of events in that level. As a result, the variance of the log odds ratio between two levels can be no smaller than the crude Woolf variance 1/a + 1/b + 1/c + 1/d computed from the level counts, whatever other covariates are in the model. Any subset of the cases, such as the n = 214,259 model sample or a single region, has smaller counts, so the bound only increases. Finally I searched every merge of levels of the model variables in Table 1 for a crude OR near 2·41 or 2·64. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Higher education, adjusted Abstract 2·41 (2·35–2·48) Appears nowhere else in the paper
Higher vs primary education, adjusted Results text; Fig. 2 1·48 (1·39–1·59) Does not overlap the Abstract interval; the Abstract OR is 1·63 times larger (2·24 times on the log scale)
Higher vs primary education, crude Fig. 2 (unadjusted); Table 1 counts 1·85 (1·73–1·97) 1·85 (1·73–1·97) from 1431/2996 vs 19,367/74,900, matching Fig. 2, so the Abstract value is not the crude OR either
SE(log OR) implied by the education interval Abstract 2·35–2·48 0·0137 (at most 0·0148 allowing for rounding)
SE(log OR) implied by the education interval Results; Fig. 2 1·39–1·59 0·0343
Smallest SE(log OR) possible for higher vs primary Table 1 counts — 0·0331. The Abstract interval is 0·41 times this. Even counting only the 1431 treated women with higher education, SE ≥ 0·0264; the Abstract interval would need at least 4570 such women
Prenatal care, adjusted Abstract 2·64 (2·56–2·73) Appears nowhere else in the paper
Prenatal care yes vs no, adjusted Results text; Fig. 2 3·05 (2·92–3·20) Does not overlap the Abstract interval; the Abstract OR is 0·87 times the Results value
Prenatal care yes vs no, crude Table 1 counts — 3·97 (3·81–4·13)
SE(log OR) implied by the prenatal interval Abstract 2·56–2·73 0·0164 (at most 0·0174 allowing for rounding)
SE(log OR) implied by the prenatal interval Results; Fig. 2 2·92–3·20 0·0234
Smallest SE(log OR) possible for prenatal yes vs no Table 1 counts — 0·0211. The Abstract interval is 0·78 times this. Even pooling "Ignored/Unknown" with "No", the bound is 0·0182
Regional aORs, prenatal care Fig. 3 2·11, 2·58, 3·70, 4·50, 4·87 None is 2·64. The closest, Northeast 2·58 (2·37–2·80), has SE 0·0425, about 2.6 times the Abstract's
Regional aORs, higher education Fig. 3 1·39, 1·72, 1·48, 1·38, 1·41 None is near 2·41
Black and Mixed vs White, adjusted Abstract vs Results 0·85 (0·82–0·89) and 0·81 (0·79–0·84) in both Identical, so the Abstract sentence uses the national model for race
Relative strength, log OR of prenatal care ÷ log OR of higher education Abstract vs Results 2·64 vs 2·41; 3·05 vs 1·48 1·10 in the Abstract, 2·84 in the Results
Crude OR near 2·41 or 2·64 from any merge of levels Table 1 (race, region, education, prenatal care) — None within 0·06

Table 1 is internally consistent: every variable sums to 50,609 treated and 167,719 untreated cases. The Results and Fig. 2 intervals are compatible with the bound (1·04 and 1·11 times it), as they should be. The Abstract intervals are not.

One smaller observation is separate from the claim. The Fig. 2 unadjusted OR for prenatal care "Yes", 3·33 (3·22–3·46), is not the Table 1 crude OR for yes vs no (3·97). It is reproduced exactly by yes vs "No" and "Ignored/Unknown" pooled: 3·33 (3·22–3·46). The "Ignored/Unknown" row in Fig. 2, 2·28 (2·11–2·47), is reproduced by ignored vs no alone. This suggests that the unadjusted "Yes" estimate may use a different reference group from the rest of the block. It does not affect the adjusted estimates.

Figure

The Abstract and the Results give different adjusted odds ratios for education and prenatal care but the same ones for race

Figure 1. Short excerpts from the Abstract (Findings, left) and from the Results text, whose values Fig. 2 repeats (right), for the four associations the Abstract quantifies. Orange arrows join two reported values of the same association that disagree: higher education 2·41 (2·35–2·48) vs 1·48 (1·39–1·59), and prenatal care 2·64 (2·56–2·73) vs 3·05 (2·92–3·20). In each case the intervals do not overlap. Dashed gray lines join values that are identical: Black women 0·85 (0·82–0·89) and Mixed women 0·81 (0·79–0·84). The excerpts are shortened; the full sentences are quoted above.

The Abstract confidence intervals are narrower than any model fitted to these cases allows

Figure 2. The standard error of the log odds ratio implied by each printed 95% CI, for higher vs primary education (top) and prenatal care yes vs no (bottom), on one linear axis that starts at zero. Orange diamonds are the Abstract intervals and hollow orange circles the Results / Fig. 2 intervals. The short orange bars show the range of SEs consistent with two-decimal rounding of the printed limits. The blue line in each group is the smallest SE any logistic model fitted to these cases can have, computed from the Table 1 counts (0·0331 and 0·0211). The shaded area to its left is unattainable. Both Results intervals lie just above the bound, and both Abstract intervals lie inside the unattainable area, the education interval at less than half the bound.

Why it matters

The Abstract is the most-read part of the paper and the only place, apart from the Results text and figures, where the paper puts numbers on its main associations. The title, the Research in context panel and the Conclusions describe these associations without numbers. In the Abstract:

  1. The education association is overstated. 2·41 is 63% larger than the Results' 1·48, and 2·24 times larger on the log-odds scale. The Abstract's interval (2·35–2·48) also suggests a precision that the 4427 women with higher education cannot provide.
  2. The prenatal-care association is understated. 2·64 is 13% smaller than 3·05. The Discussion's statement of "approximately threefold higher odds" matches the Results, not the Abstract.
  3. The relative ranking is altered. The Abstract presents education and prenatal care as roughly equal (log-OR ratio 1·10). In the Results, prenatal care is the strongest predictor, with a log OR 2·84 times that of higher education. The Discussion states that "public health prioritization should be guided primarily by effect size", so the size and ranking of these two associations are what a reader would act on.

The direction and statistical significance of both associations are the same in the Abstract and the Results. So are the paper's qualitative conclusions: partner treatment is low, it is more likely with prenatal care and higher education, and it is less likely among Black and Mixed women. The race estimates in the Abstract are correct. The error is in the reported size of two of the four quantified associations in the Abstract.

Explanations considered

Status

Suggested correction

  1. Replace the Abstract's education and prenatal-care estimates with the values from the national multivariable model, stating the education contrast explicitly. For example: "Higher maternal education (higher vs primary: aOR 1·48, 95% CI 1·39–1·59) and receipt of prenatal care (aOR 3·05, 95% CI 2·92–3·20) were associated with partner treatment".
  2. If 2·41 (2·35–2·48) and 2·64 (2·56–2·73) come from another analysis, identify that analysis (model, sample, contrast and CI method) in the Methods and Results, and explain why the Abstract reports it instead of the main model.
  3. Minor, separate: check the reference group of the Fig. 2 unadjusted OR for prenatal care "Yes" (3·33), which matches yes vs "No" and "Ignored/Unknown" pooled rather than yes vs no (3·97), and state the reference in the figure if it differs from the adjusted estimate's.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12995907 (Lancet Reg Health Am 2026; doi:10.1016/j.lana.2026.101434)
# Abstract aORs for education and prenatal care vs the paper's own Results, Fig. 2, Fig. 3 and Table 1.
# Run:  python recompute.py.txt      (needs numpy + matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) and
# five candidate figure designs (candidates/cand1..cand5 .svg/.png) into this file's folder.
import math, itertools, os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch
from matplotlib.transforms import blended_transform_factory

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964

# ---------------------------------------------------------------- inputs from the paper
# Table 1 counts: (adequate partner treatment, no adequate partner treatment)
T = {
 "race":   {"White": (14578, 39887), "Black": (4429, 16105), "Asian": (181, 584),
            "Mixed": (27822, 98612), "Indigenous": (213, 544), "Ignored": (3386, 11987)},
 "region": {"North": (4196, 15681), "Northeast": (14302, 50668), "Southeast": (20359, 71759),
            "South": (7788, 20561), "Center-west": (3964, 9050)},
 "edu":    {"Primary": (19367, 74900), "Secondary": (17804, 47544), "Higher": (1431, 2996),
            "Ignored": (12007, 42279)},
 "pnc":    {"No": (2627, 29535), "Yes": (46943, 133066), "Ignored": (1039, 5118)},
 "timing": {"During prenatal care": (38593, 83773), "At delivery/curettage": (8290, 64508),
            "After delivery": (2243, 13514), "Not performed": (217, 1286), "Ignored": (1266, 4638)},
}
N_TREATED, N_UNTREATED = 50609, 167719

# Reported estimates: (point, lo, hi)
ABS_EDU, ABS_PNC = (2.41, 2.35, 2.48), (2.64, 2.56, 2.73)          # Abstract, Findings
RES_EDU, RES_PNC = (1.48, 1.39, 1.59), (3.05, 2.92, 3.20)          # Results p5 and Fig. 2 (aOR)
RES_SEC = (1.23, 1.20, 1.27)                                       # Results p5, secondary vs primary
FIG2_UNADJ_EDU, FIG2_UNADJ_PNC = (1.85, 1.73, 1.97), (3.33, 3.22, 3.46)  # Fig. 2 unadjusted OR
FIG2_UNADJ_PNC_IGN = (2.28, 2.11, 2.47)                            # Fig. 2 unadjusted, PNC ignored vs no
ABS_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
RES_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
REGIONS = ["North", "Northeast", "Central-West", "Southeast", "South"]   # Fig. 3 order
FIG3_PNC = [(2.11, 1.86, 2.40), (2.58, 2.37, 2.80), (3.70, 3.05, 4.48), (4.50, 4.11, 4.94), (4.87, 4.21, 5.62)]
FIG3_EDU = [(1.39, 1.12, 1.72), (1.72, 1.49, 1.99), (1.48, 1.16, 1.87), (1.38, 1.23, 1.55), (1.41, 1.21, 1.64)]

def orci(a, b, c, d):
    """Crude OR of (a treated, b untreated) vs reference (c, d), Woolf 95% CI and SE(log OR)."""
    o = (a / b) / (c / d)
    se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    return o, o * math.exp(-Z * se), o * math.exp(Z * se), se

def se_ci(lo, hi):
    return math.log(hi / lo) / (2 * Z)

def se_ci_maxround(lo, hi, dp=2):
    """Largest SE consistent with CI limits printed to dp decimals."""
    h = 0.5 * 10 ** (-dp)
    return math.log((hi + h) / (lo - h)) / (2 * Z)

def f(x): return f"{x:.2f}"

print("=" * 78)
print("1. Table 1 column totals")
for v, d in T.items():
    print(f"  {v:7s}: treated {sum(x[0] for x in d.values()):6d} (paper {N_TREATED}), "
          f"untreated {sum(x[1] for x in d.values()):6d} (paper {N_UNTREATED})")

print("\n2. Crude ORs from Table 1 counts (Woolf 95% CI)")
crude = {}
pairs = {
    "Higher vs Primary":       (T["edu"]["Higher"], T["edu"]["Primary"]),
    "Secondary vs Primary":    (T["edu"]["Secondary"], T["edu"]["Primary"]),
    "Edu Ignored vs Primary":  (T["edu"]["Ignored"], T["edu"]["Primary"]),
    "Secondary+Higher vs Primary": ((T["edu"]["Secondary"][0] + T["edu"]["Higher"][0],
                                     T["edu"]["Secondary"][1] + T["edu"]["Higher"][1]), T["edu"]["Primary"]),
    "PNC Yes vs No":           (T["pnc"]["Yes"], T["pnc"]["No"]),
    "PNC Ignored vs No":       (T["pnc"]["Ignored"], T["pnc"]["No"]),
    "PNC Yes vs (No+Ignored)": (T["pnc"]["Yes"], (T["pnc"]["No"][0] + T["pnc"]["Ignored"][0],
                                                  T["pnc"]["No"][1] + T["pnc"]["Ignored"][1])),
    "Black vs White":          (T["race"]["Black"], T["race"]["White"]),
    "Mixed vs White":          (T["race"]["Mixed"], T["race"]["White"]),
}
for k, (x, r) in pairs.items():
    crude[k] = orci(*x, *r)
    o, lo, hi, se = crude[k]
    print(f"  {k:28s} OR {o:.3f} ({lo:.2f}-{hi:.2f})  SE(log OR) {se:.4f}")
print("  Fig. 2 unadjusted: higher edu 1.85 (1.73-1.97); PNC yes 3.33 (3.22-3.46); PNC ignored 2.28 (2.11-2.47)")
print("  -> Fig. 2's unadjusted 'Yes' OR equals Yes vs (No+Ignored), not Yes vs No (side observation)")

print("\n3. SE(log OR) implied by each reported 95% CI (and largest SE allowed by 2-dp rounding)")
rep = [("Abstract, higher education", ABS_EDU), ("Results/Fig. 2, higher vs primary", RES_EDU),
       ("Abstract, prenatal care", ABS_PNC), ("Results/Fig. 2, prenatal yes vs no", RES_PNC),
       ("Results, secondary vs primary", RES_SEC), ("Fig. 2 unadjusted, higher edu", FIG2_UNADJ_EDU),
       ("Fig. 2 unadjusted, prenatal yes", FIG2_UNADJ_PNC)]
for lab, (p, lo, hi) in rep:
    print(f"  {lab:36s} {p:.2f} ({lo:.2f}-{hi:.2f}): SE {se_ci(lo, hi):.4f} "
          f"(max with rounding {se_ci_maxround(lo, hi):.4f}); geometric midpoint {math.sqrt(lo*hi):.3f}")

print("\n4. Lower bounds on SE(log OR) for any logistic model fitted to these cases")
# For a logistic model that contains level indicators for the variable, the score equations make the
# fitted probabilities in each level sum to the observed events there, so the information carried by
# level A is at most n_A*pbar*(1-pbar) = 1/(1/a + 1/b). By Cauchy-Schwarz on the inverse information,
# Var(beta_A - beta_B) >= 1/I_A + 1/I_B >= (1/a + 1/b + 1/c + 1/d): the crude Woolf variance.
# Any subset of the 218,328 cases (the n = 214,259 model sample, a single region) has smaller
# counts, so the bound only grows. Weaker but more general: Var >= 1/a, a = treated cases in level.
b_edu = crude["Higher vs Primary"][3]
b_pnc = crude["PNC Yes vs No"][3]
b_edu_a = math.sqrt(1 / T["edu"]["Higher"][0])
b_pnc_a = math.sqrt(1 / T["pnc"]["No"][0])
print(f"  Higher vs primary: SE >= {b_edu:.4f} (crude Woolf); SE >= {b_edu_a:.4f} from 1431 treated alone")
print(f"  Prenatal yes vs no: SE >= {b_pnc:.4f} (crude Woolf); SE >= {b_pnc_a:.4f} from 2627 treated (No) alone")
se_ae, se_ap = se_ci(*ABS_EDU[1:]), se_ci(*ABS_PNC[1:])
se_re, se_rp = se_ci(*RES_EDU[1:]), se_ci(*RES_PNC[1:])
print(f"  Abstract education SE {se_ae:.4f} = {se_ae/b_edu:.2f} x the bound; Results {se_re:.4f} = {se_re/b_edu:.2f} x")
print(f"  Abstract prenatal SE {se_ap:.4f} = {se_ap/b_pnc:.2f} x the bound; Results {se_rp:.4f} = {se_rp/b_pnc:.2f} x")
need_a = 1 / se_ci_maxround(*ABS_EDU[1:]) ** 2
print(f"  To allow the Abstract education CI, the higher-education level would need >= {need_a:.0f} "
      f"treated cases (Table 1: 1431)")
need_p = 1 / se_ci_maxround(*ABS_PNC[1:]) ** 2
print(f"  To allow the Abstract prenatal CI, the prenatal 'No' level would need >= {need_p:.0f} "
      f"treated cases (Table 1: 2627)")
print("  Fig. 2 unadjusted prenatal SE "
      f"{se_ci(*FIG2_UNADJ_PNC[1:]):.4f} vs bound for Yes vs (No+Ignored) {crude['PNC Yes vs (No+Ignored)'][3]:.4f}")

print("\n5. Size of the discrepancies")
for lab, a, r in [("Education", ABS_EDU, RES_EDU), ("Prenatal", ABS_PNC, RES_PNC)]:
    print(f"  {lab}: Abstract/Results OR ratio {a[0]/r[0]:.3f}; log OR {math.log(a[0]):.3f} vs "
          f"{math.log(r[0]):.3f} (ratio {math.log(a[0])/math.log(r[0]):.2f}); CIs overlap: {a[1] <= r[2] and r[1] <= a[2]}")
print(f"  Abstract: prenatal/education log-OR ratio {math.log(ABS_PNC[0])/math.log(ABS_EDU[0]):.2f}; "
      f"Results: {math.log(RES_PNC[0])/math.log(RES_EDU[0]):.2f}")
for k in ABS_RACE:
    print(f"  {k}: Abstract {ABS_RACE[k]} vs Results {RES_RACE[k]} -> identical: {ABS_RACE[k] == RES_RACE[k]}")

print("\n6. Does 2.41 or 2.64 match any other estimate in the paper?")
for lab, vals in [("Fig. 3 prenatal", FIG3_PNC), ("Fig. 3 higher education", FIG3_EDU)]:
    for reg, (p, lo, hi) in zip(REGIONS, vals):
        print(f"  {lab:24s} {reg:12s} {p:.2f} ({lo:.2f}-{hi:.2f}) SE {se_ci(lo, hi):.4f}")
def search(vars_):
    hits = []
    for v in vars_:
        d = T[v]; keys = list(d)
        for r1 in range(1, len(keys)):
            for A in itertools.combinations(keys, r1):
                rest = [k for k in keys if k not in A]
                for r2 in range(1, len(rest) + 1):
                    for B in itertools.combinations(rest, r2):
                        a = sum(d[k][0] for k in A); b = sum(d[k][1] for k in A)
                        c = sum(d[k][0] for k in B); dd = sum(d[k][1] for k in B)
                        for (o, lo, hi, se), lab in ((orci(a, b, c, dd), (A, B)), (orci(c, dd, a, b), (B, A))):
                            for tgt in (2.41, 2.64):
                                if abs(o - tgt) < 0.06:
                                    hits.append((v, " + ".join(lab[0]), "vs", " + ".join(lab[1]), round(o, 3), round(se, 4)))
    return list(dict.fromkeys(hits))
h_model = search(["race", "region", "edu", "pnc"])
print("  Exhaustive crude-OR search over every merge of levels of the model variables in Table 1 (race, region,"
      f" education, prenatal care), both directions, within 0.06 of 2.41 or 2.64: {h_model if h_model else 'none'}")
h_tim = search(["timing"])
print(f"  Same search on timing of diagnosis (not a model covariate): {len(h_tim)} hits, all arbitrary merges, all with SE(log OR) >= 0.0189:")
for h in h_tim:
    print("    ", h)
o, lo, hi, se = orci(*T["timing"]["During prenatal care"],
                     sum(v[0] for k, v in T["timing"].items() if k != "During prenatal care"),
                     sum(v[1] for k, v in T["timing"].items() if k != "During prenatal care"))
print(f"  Timing, diagnosis during prenatal care vs all others: OR {o:.2f} ({lo:.2f}-{hi:.2f})")

# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW, MS = 1.5, 8
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK,
    "xtick.color": INK2, "ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})

def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg")
        fig.savefig(p + ".png", dpi=200)
    plt.close(fig)

def style_axis(ax):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(axis="y", length=0)

def fmt(t):  # 2.41 (2.35–2.48) with the paper's middle-dot decimals
    return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})".replace(".", "·")

# ---------------------------------------------------------------- candidate 1: dot-and-interval
def cand_dot_interval(paths):
    ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
    groups = [
        ("Higher vs primary education", [
            ("Abstract aOR", ABS_EDU, "abs"), ("Results text and Fig. 2 aOR", RES_EDU, "body"),
            ("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU, "body"),
            ("Crude OR from Table 1 counts", ce[:3], "rec")]),
        ("Prenatal care, yes vs no", [
            ("Abstract aOR", ABS_PNC, "abs"), ("Results text and Fig. 2 aOR", RES_PNC, "body"),
            ("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC, "body"),
            ("Crude OR from Table 1 counts", cp[:3], "rec")]),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 4.4))
    fig.subplots_adjust(left=0.31, right=0.98, top=0.87, bottom=0.12)
    y, yt, yl = 0, [], []
    for gname, rows in groups:
        ax.text(0.01, y + 0.15, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
                ha="left", va="center", fontsize=10.5, fontweight="bold")
        y -= 1
        for lab, (p, lo, hi), kind in rows:
            col = BLUE if kind == "rec" else ORANGE
            mfc = BG if kind == "body" else col
            mk = "s" if kind == "rec" else ("D" if kind == "abs" else "o")
            ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="butt")
            ax.plot(p, y, marker=mk, ms=MS, mfc=mfc, mec=col, mew=1.5, color=col)
            ax.text(hi * 1.03, y, fmt((p, lo, hi)), va="center", fontsize=10, color=INK)
            yt.append(y); yl.append(lab)
            y -= 1
        y -= 0.6
    ax.axvline(1, color=INK2, lw=0.8)
    ax.set_xscale("log")
    ax.set_xlim(0.9, 6.5)
    ticks = [1, 1.5, 2, 3, 4, 5]
    ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks])
    ax.minorticks_off()
    ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
    ax.set_ylim(y + 0.6, 0.8)
    ax.set_xlabel("Odds ratio (log scale); vertical line at 1 = no association")
    style_axis(ax)
    leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Reported in the Abstract"),
           Line2D([], [], marker="o", color=ORANGE, mfc=BG, mec=ORANGE, ms=MS, lw=LW, label="Reported in Results / Fig. 2"),
           Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
    fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False,
               fontsize=10, handlelength=2.0, columnspacing=1.0)
    save(fig, *paths)

# ---------------------------------------------------------------- candidate 2: two text panels
def cand_text_panels(paths):
    fig = plt.figure(figsize=(7.2, 4.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
    L0, L1, R0, R1 = 0.05, 2.6, 4.6, 7.15
    top, bot = 4.05, 0.35
    for x0, x1, title, sub in [(L0, L1, "Abstract", "Findings"),
                               (R0, R1, "Results text and Fig. 2", "national model, n = 214,259")]:
        ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.06",
                                    fc="#ffffff", ec=INK2, lw=1.0))
        ax.text(x0 + 0.14, top + 0.33, title, fontsize=11.5, fontweight="bold", va="center")
        ax.text(x0 + 0.14, top + 0.1, sub, fontsize=10, color=INK2, va="center")
    rows = [
        ("Higher maternal education", "aOR 2·41, 95% CI 2·35–2·48",
         "Higher vs primary education", "aOR 1·48, 1·39–1·59", "conflict"),
        ("Receipt of prenatal care", "aOR 2·64, 95% CI 2·56–2·73",
         "Prenatal care (yes vs no)", "aOR 3·05, 95% CI 2·92–3·20", "conflict"),
        ("Black women", "aOR 0·85, 95% CI 0·82–0·89",
         "Black vs White women", "aOR 0·85, 0·82–0·89", "same"),
        ("Mixed women", "aOR 0·81, 95% CI 0·79–0·84",
         "Mixed vs White women", "aOR 0·81, 0·79–0·84", "same"),
    ]
    ys = [3.55, 2.65, 1.65, 0.85]
    notes = {0: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"],
             1: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"]}
    for i, ((ll, lv, rl, rv, kind), yy) in enumerate(zip(rows, ys)):
        for x0, lab, val in [(L0, ll, lv), (R0, rl, rv)]:
            ax.text(x0 + 0.14, yy + 0.15, lab, fontsize=10, color=INK2, va="center")
            ax.text(x0 + 0.14, yy - 0.13, val, fontsize=10.5, fontweight="bold" if kind == "conflict" else "normal",
                    va="center")
        if kind == "conflict":
            ax.annotate("", xy=(R0 - 0.04, yy), xytext=(L1 + 0.04, yy),
                        arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=LW))
            ax.text((L1 + R0) / 2, yy + 0.2, "≠", ha="center", va="center", fontsize=16, color=INK)
            ax.text((L1 + R0) / 2, yy - 0.2, "\n".join(notes[i][:1]), ha="center", va="top", fontsize=10, color=INK)
        else:
            ax.plot([L1 + 0.04, R0 - 0.04], [yy, yy], color=INK2, lw=1.0, ls=(0, (3, 2)))
            ax.text((L1 + R0) / 2, yy + 0.14, "identical", ha="center", va="bottom", fontsize=10, color=INK2)
    ax.plot([L0 + 0.14, R1 - 0.14], [2.12, 2.12], color=GRID, lw=1.0)
    ax.text(3.6, 0.12, "Orange arrows: the two reported values of one estimate disagree.  "
            "Dashed gray: the values agree.", ha="center", va="center", fontsize=10, color=INK2)
    save(fig, *paths)

# ---------------------------------------------------------------- candidate 3: interval width vs bound
def cand_se_bound(paths):
    rows = [("Higher vs primary education", ABS_EDU, RES_EDU, b_edu),
            ("Prenatal care, yes vs no", ABS_PNC, RES_PNC, b_pnc)]
    fig, ax = plt.subplots(figsize=(7.2, 4.0))
    fig.subplots_adjust(left=0.33, right=0.97, top=0.86, bottom=0.13)
    yt, yl = [], []
    y = 0
    for gname, a, r, bnd in rows:
        ax.text(0.01, y + 0.75, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
                ha="left", va="center", fontsize=10.5, fontweight="bold")
        ax.fill_betweenx([y - 1.45, y + 0.35], 0, bnd, color="#eeedea", lw=0, zorder=0)
        ax.plot([bnd, bnd], [y - 1.45, y + 0.35], color=BLUE, lw=LW, zorder=2)
        ax.text(bnd, y + 0.42, f"smallest SE possible: {bnd:.4f}".replace(".", "·"), color=INK,
                fontsize=10, va="bottom", ha="center")
        for lab, t, filled, dy in [("Abstract", a, True, 0), ("Results / Fig. 2", r, False, -1)]:
            s, smax = se_ci(*t[1:]), se_ci_maxround(*t[1:])
            smin = math.log((t[2] - 0.005) / (t[1] + 0.005)) / (2 * Z)
            ax.plot([smin, smax], [y + dy, y + dy], color=ORANGE, lw=LW)
            ax.plot(s, y + dy, "D" if filled else "o", ms=MS, color=ORANGE, mfc=ORANGE if filled else BG, mew=1.5)
            ax.text(smax + 0.0006, y + dy, f"{s:.4f}".replace(".", "·"), va="center", fontsize=10)
            yt.append(y + dy); yl.append(f"{lab}, CI " + f"{t[1]:.2f}–{t[2]:.2f}".replace(".", "·"))
        ax.text(0.0005, y - 1.42, "below the bound", fontsize=10, color=INK2, va="bottom")
        y -= 2.6
    ax.set_xlim(0, 0.05)
    ax.set_ylim(y + 0.9, 1.25)
    ax.set_yticks(yt); ax.set_yticklabels(yl)
    xt = [0, 0.01, 0.02, 0.03, 0.04, 0.05]
    ax.set_xticks(xt); ax.set_xticklabels([f"{v:.2f}".replace(".", "·") for v in xt])
    ax.set_xlabel("Standard error of log(odds ratio) implied by the printed 95% CI")
    style_axis(ax)
    leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract (bar: rounding range)"),
           Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results / Fig. 2"),
           Line2D([], [], color=BLUE, lw=LW, label="Bound from Table 1 counts")]
    fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
               handlelength=1.8, columnspacing=1.0)
    save(fig, *paths)

# ---------------------------------------------------------------- candidate 4: every estimate in the paper
def cand_all_estimates(paths):
    ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
    groups = [
        ("Higher vs primary education", ABS_EDU,
         [("Results / Fig. 2 aOR", RES_EDU), ("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU)] +
         [(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_EDU)], ce[:3]),
        ("Prenatal care, yes vs no", ABS_PNC,
         [("Results / Fig. 2 aOR", RES_PNC), ("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC)] +
         [(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_PNC)], cp[:3]),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 6.6))
    fig.subplots_adjust(left=0.32, right=0.78, top=0.93, bottom=0.08)
    y, yt, yl = 0, [], []
    for gname, a, body, rc in groups:
        ax.text(0.01, y + 0.1, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
                ha="left", va="center", fontsize=10.5, fontweight="bold")
        y -= 1
        y0 = y
        rows = [("Abstract aOR", a, "abs")] + [(l, t, "body") for l, t in body] + [("Crude OR, Table 1 counts", rc, "rec")]
        for lab, (p, lo, hi), kind in rows:
            col = BLUE if kind == "rec" else ORANGE
            mk = {"abs": "D", "body": "o", "rec": "s"}[kind]
            ax.plot([lo, hi], [y, y], color=col, lw=LW)
            ax.plot(p, y, mk, ms=MS, color=col, mfc=BG if kind == "body" else col, mew=1.5)
            ax.text(1.02, y, fmt((p, lo, hi)), va="center", ha="left", fontsize=10, transform=ax.get_yaxis_transform())
            yt.append(y); yl.append(lab); y -= 1
        ax.plot([a[0], a[0]], [y + 0.6, y0 + 0.4], color=ORANGE, lw=0.8, ls=(0, (3, 2)), zorder=0)
        y -= 0.6
    ax.axvline(1, color=INK2, lw=0.8)
    ax.set_xscale("log"); ax.set_xlim(0.9, 6.0)
    ticks = [1, 1.5, 2, 3, 4, 5]
    ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks]); ax.minorticks_off()
    ax.set_yticks(yt); ax.set_yticklabels(yl)
    ax.set_ylim(y + 0.6, 0.6)
    ax.set_xlabel("Odds ratio (log scale); dashed line = Abstract value")
    style_axis(ax)
    leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract"),
           Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results, Fig. 2, Fig. 3"),
           Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
    fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10)
    save(fig, *paths)

# ---------------------------------------------------------------- candidate 5: slope chart Abstract -> Results
def cand_slope(paths):
    items = [("Prenatal care", ABS_PNC, RES_PNC), ("Higher education", ABS_EDU, RES_EDU),
             ("Black women", ABS_RACE["Black"], RES_RACE["Black"]),
             ("Mixed women", ABS_RACE["Mixed"], RES_RACE["Mixed"])]
    fig, ax = plt.subplots(figsize=(7.2, 4.6))
    fig.subplots_adjust(left=0.1, right=0.98, top=0.88, bottom=0.1)
    xa, xr = 0, 1
    dy_a = {"Prenatal care": 1.035, "Higher education": 0.965, "Black women": 1.02, "Mixed women": 0.98}
    dy_r = {"Prenatal care": 1.0, "Higher education": 1.0, "Black women": 1.02, "Mixed women": 0.98}
    for lab, a, r in items:
        changed = a != r
        col = ORANGE if changed else INK2
        ax.plot([xa, xr], [a[0], r[0]], color=col, lw=LW, ls="-" if changed else (0, (3, 2)))
        for x, t in [(xa, a), (xr, r)]:
            ax.plot([x, x], [t[1], t[2]], color=col, lw=LW + 1.5, solid_capstyle="butt")
            ax.plot(x, t[0], "o", ms=MS, color=col, mfc=col if changed else BG, mew=1.5)
        ax.text(xa - 0.06, a[0] * dy_a[lab], f"{lab}  {a[0]:.2f}".replace(".", "·"), ha="right", va="center", fontsize=10)
        ax.text(xr + 0.06, r[0] * dy_r[lab], f"{r[0]:.2f}  {lab}".replace(".", "·"), ha="left", va="center", fontsize=10)
    ax.set_yscale("log"); ax.set_ylim(0.7, 3.6)
    yt = [0.75, 1, 1.5, 2, 2.5, 3, 3.5]
    ax.set_yticks(yt); ax.set_yticklabels([str(v).replace(".", "·") for v in yt]); ax.minorticks_off()
    ax.axhline(1, color=INK2, lw=0.8)
    ax.set_xlim(-0.95, 1.95)
    ax.set_xticks([xa, xr]); ax.set_xticklabels(["Abstract", "Results text / Fig. 2"], fontsize=10.5, fontweight="bold")
    ax.set_ylabel("Adjusted odds ratio (log scale)")
    for s in ("top", "right", "bottom"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.tick_params(axis="x", length=0)
    leg = [Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported values differ"),
           Line2D([], [], color=INK2, lw=LW, ls=(0, (3, 2)), marker="o", mfc=BG, ms=MS, label="Reported values identical")]
    fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10)
    save(fig, *paths)

cand_dot_interval([os.path.join(CAND, "cand1_dot_interval")])
cand_text_panels([os.path.join(CAND, "cand2_text_panels"), os.path.join(HERE, "fig1")])
cand_se_bound([os.path.join(CAND, "cand3_se_bound"), os.path.join(HERE, "fig2")])
cand_all_estimates([os.path.join(CAND, "cand4_all_estimates")])
cand_slope([os.path.join(CAND, "cand5_slope")])
print("\nFigures written: fig1 (= candidates/cand2), fig2 (= candidates/cand3), candidates/cand1..cand5")

Abstract effect sizes for crowding, ageing and primary care coverage disagree with the paper's Table 2

Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101572 · PMC13382474

Summary

The abstract reports three effects from "the final spatial GAMLSS model", each per one percentage-point increase. It gives +3.25% expected tuberculosis (TB) mortality for household crowding, +5.92% for the population aged 60 or older, and −0.30% for primary health care (PHC) coverage. The paper's Table 2 reports that model's coefficients: 0.0211, 0.0432 and −0.0025. Converted with the formula the Results themselves use, 100·(exp(β)−1), these give +2.13%, +4.41% and −0.25%. The Results text prints exactly those three values. So the abstract states each effect larger than the paper's own estimate, by 52%, 34% and 20%. No coefficient in Table 2 or in the supplement's sensitivity models (Table S3) reproduces all three abstract figures. The direction and significance of each association are unchanged, but the abstract's headline magnitudes do not match the model they cite.

What the paper reports

The abstract:

In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.30% decrease in expected TB mortality.

— Abstract, Findings

The Results, describing the same model:

Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01.

— Results, "Final association model", paragraph 1

Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the expected number of municipal-monthly TB deaths.

— Results, "Final association model", paragraph 2 (this states how coefficients are converted to percentages)

Among the contextual variables, each one percentage-point increase in the proportion of households with more than two residents per bedroom was associated with a 2.13% increase in expected TB deaths. Similarly, each one percentage-point increase in the proportion of residents aged 60 years or older was associated with a 4.41% increase,

— Results, "Final association model", paragraph 3

Each one percentage-point increase in coverage was associated with a 0.25% decrease in expected TB deaths.

— Results, "Final association model", paragraph 4

Table 2 ("Parametric covariate estimates from the spatial GAMLSS model for monthly TB deaths in São Paulo, Brazil, 2020–2024"). The columns are variable, coded variable, estimate, standard error, exponential of the estimate and p-value:

Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041

Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001

Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002

— Table 2

I checked each quotation against the article as archived in PubMed Central (src/show.py PMC13382474). Each occurs exactly once. recompute.py.txt repeats that check.

What the paper's own numbers show

Method. The model uses a negative binomial type I distribution with a log link and an offset of log municipal population. A one-unit increase in a covariate therefore multiplies expected deaths by exp(β), a change of 100·(exp(β)−1)%. The Results state this formula for the time trend. All three covariates are measured in percent, so a one-unit increase is one percentage point. I applied the formula to the printed Table 2 coefficients. To allow for rounding, I also used the full range each coefficient could have before it was rounded to four decimals (β ± 0.00005). I inverted each abstract figure, ln(1 + p/100), to find the coefficient it would require. I formed 95% intervals as β ± 1.96·SE from the printed standard errors; the paper reports SEs and p-values but no intervals. Finally, I read every coefficient in the supplement's Table S3, which has the final model and four models that each drop one covariate. I tested whether any of them gives an abstract figure, either exponentiated or as 100·β.

Quantity Location Reported Recomputed from the paper's own numbers
Crowding (>2 residents/bedroom), % change per 1 pp Abstract +3.25% +2.13% = 100·(exp(0.0211)−1); rounding range +2.127 to +2.138%
Crowding, % change per 1 pp Results text; Table 2 +2.13%; β 0.0211, exp(β) 1.0213 +2.13% (consistent)
Coefficient needed for the abstract's +3.25% Abstract vs Table 2 β = 0.0211 ln(1.0325) = 0.0320
Population aged 60+, % change per 1 pp Abstract +5.92% +4.41% = 100·(exp(0.0432)−1); rounding range +4.409 to +4.420%
Population aged 60+, % change per 1 pp Results text; Table 2 +4.41%; β 0.0432, exp(β) 1.0442 +4.41% (+4.42% from the rounded 1.0442; consistent)
Coefficient needed for the abstract's +5.92% Abstract vs Table 2 β = 0.0432 ln(1.0592) = 0.0575
PHC coverage, % change per 1 pp Abstract −0.30% −0.25% = 100·(exp(−0.0025)−1); rounding range −0.255 to −0.245%
PHC coverage, % change per 1 pp Results text; Table 2 −0.25%; β −0.0025, exp(β) 0.9975 −0.25% (consistent)
Coefficient needed for the abstract's −0.30% Abstract vs Table 2 β = −0.0025 ln(0.9970) = −0.0030
Abstract ÷ Table 2 effect (crowding / age 60+ / PHC) Abstract vs Table 2 — 1.52 / 1.34 / 1.20
95% CI implied by Table 2, % per 1 pp (crowding / age 60+ / PHC) Table 2 (β ± 1.96 SE) not reported +0.68 to +3.60 / +2.85 to +6.00 / −0.39 to −0.11
Other Table 2 rows: time trend, TB incidence, education, Black or brown population Results text +0.48%, +2.13%, −3.18%, +1.43% +0.48%, +2.13%, −3.19%, +1.43% (consistent within rounding)
Crowding coefficient in the Table S3 sensitivity models Supplement, Table S3 0.0040 / 0.0344 / 0.0381 +0.40% / +3.50% / +3.88%; none is +3.25%
Age 60+ coefficient in the Table S3 sensitivity models Supplement, Table S3 0.0357 / 0.0313 / 0.0259 / 0.0592 +3.63% / +3.18% / +2.62% / +6.10%; none is +5.92% after conversion
PHC coefficient in the Table S3 sensitivity models Supplement, Table S3 −0.0025 / −0.0037 / −0.0024 / −0.0033 −0.25% / −0.37% / −0.24% / −0.33%; none is −0.30%

Within the body of the paper the numbers agree with each other. Table 2's coefficients, its exponentiated column and the Results text all match, as do the four other covariates in the table. The abstract is the only place where the three values differ. Each abstract figure needs a coefficient roughly 20–50% larger in absolute value than the one printed, which is far outside any rounding range.

Figure

The abstract and the Results/Table 2 give different effect sizes for the same three covariates of the same model

Figure 1. The left panel shows the three per-percentage-point effects in the abstract's Findings. The right panel shows, for the same covariates of the same model, the Table 2 coefficient and its printed exponential, the percentage given in the Results text, and 100·(exp(β)−1) recomputed from the coefficient. Orange marks values as printed in the paper; blue marks values recomputed in recompute.py.txt. Each connector gives the ratio of the abstract value to the recomputed value. The Results text and the recomputation agree to two decimals in all three rows. The abstract does not agree with either in any row.

Abstract values plotted against the Table 2 estimates and their 95% confidence intervals

Figure 2. This figure adds the uncertainty. For each covariate, the blue dot is 100·(exp(β)−1) from Table 2 and the blue line is the 95% interval from β ± 1.96·SE. The paper reports the SEs but not the intervals. The orange diamond is the value in the abstract, and the thin vertical line marks no change (0%). Each panel has its own horizontal scale because the effects differ by more than tenfold. All three abstract values lie inside the interval, so sampling uncertainty does not separate them from the estimate. Even so, none of them is the estimate the paper reports. The age 60+ value (+5.92%) sits near the upper limit (+6.00%), and the crowding value (+3.25%) sits in the upper quarter of its interval.

Why it matters

The abstract is where most readers, and most citing papers, get a study's effect sizes. Here every number in the abstract's Findings is overstated relative to the model the abstract names. Crowding is 1.52 times the Table 2 value, age 60+ is 1.34 times and PHC coverage is 1.20 times. The overstatement grows when the effects are applied to realistic differences between municipalities. Over a 10-percentage-point difference in crowding, Table 2 implies +23.5% expected TB deaths (100·(exp(10·0.0211)−1)), while the abstract's per-point figure compounds to +37.7%. For the population aged 60+ the corresponding values are +54.0% and +77.7%. The PHC effect is small either way: −2.5% versus −3.0% per 10 points. The abstract's Interpretation ends by calling to "strengthen primary health care", and that recommendation rests on this association.

The qualitative findings are not affected. Every association keeps its direction and its significance, and each abstract value lies inside the confidence interval implied by Table 2. The Interpretation, Discussion and conclusions describe the associations without these numbers, so they stand. The error is one of reported magnitude, confined to the abstract.

Explanations considered

No explanation considered accounts for the abstract's three figures. The values in the body of the paper are internally consistent and reproduce from the printed coefficients.

Status

As of 2026-10-23.

Suggested correction

In the abstract's Findings, replace the three effect sizes with the values the paper reports in its Results and Table 2. Adding the confidence intervals would also help:

In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 2.13% and 4.41% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.25% decrease in expected TB mortality.

If the authors meant the abstract to report a different model, the abstract should name that model and its coefficients should be published. As shown above, though, none of the reported models gives all three current figures. The other numbers in the Results that we checked need no change.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13382474 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2026.101572)
# Abstract vs Results / Table 2: per-percentage-point effects of household crowding,
# population aged 60+, and primary health care (PHC) coverage on expected TB mortality.
#
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# Inputs
#  * Table 2 (final spatial GAMLSS, negative binomial type I, log link, offset log population):
#    coefficients, standard errors and exp(estimate) as printed in the article archived in
#    PubMed Central (PMC13382474).
#  * Table S3 (supplement mmc1.pdf): coefficients of the final model and of the four
#    leave-one-covariate-out sensitivity models, as printed.
#  * Quotes: copied verbatim from the PMC-archived article. When run inside the
#    /home/ubuntu/papers repo, the script re-checks each quote with src/show.py (skipped elsewhere).
#  * Percent change per unit = 100 * (exp(beta) - 1), the formula the Results state
#    explicitly for the time trend: "0.48% [ (exp (0.0048) -1) * 100]".

import math, os, subprocess

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

HERE = os.path.dirname(os.path.abspath(__file__))

ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"

# ------------------------------------------------------------------ quotes
QUOTES = {
    "abstract": ("Abstract, Findings",
        "In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) "
        "and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected "
        "TB mortality for each one percentage-point increase, respectively. In contrast, each one "
        "percentage-point increase in primary health care coverage was associated with a 0.30% decrease "
        "in expected TB mortality."),
    "results_time": ("Results, Final association model, para 2",
        "Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the "
        "expected number of municipal-monthly TB deaths."),
    "results_pos": ("Results, Final association model, para 3",
        "Among the contextual variables, each one percentage-point increase in the proportion of "
        "households with more than two residents per bedroom was associated with a 2.13% increase in "
        "expected TB deaths. Similarly, each one percentage-point increase in the proportion of "
        "residents aged 60 years or older was associated with a 4.41% increase,"),
    "results_phc": ("Results, Final association model, para 4",
        "Each one percentage-point increase in coverage was associated with a 0.25% decrease in "
        "expected TB deaths."),
    "results_t2": ("Results, Final association model, para 1",
        "Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01."),
    "t2_crowd": ("Table 2",
        "Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041"),
    "t2_age": ("Table 2",
        "Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001"),
    "t2_phc": ("Table 2",
        "Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002"),
    "sens": ("Results, Diagnostic plots, para 3",
        "In all reduced models, the direction of the associations for the remaining covariates was "
        "preserved (Table S3)"),
}


def check_quotes():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.\n")
        return
    try:
        full = subprocess.run([py, "src/show.py", "PMC13382474", "--full"], cwd=repo,
                              capture_output=True, text=True, timeout=300).stdout
    except Exception as e:
        print(f"Quote check: could not load archived text ({e}); skipped.\n")
        return
    norm = lambda s: " ".join(s.replace("−", "-").split())
    full_n = norm(full)
    print("Quote check against the PMC-archived full text (occurrences; 1 = found once):")
    for key, (loc, q) in QUOTES.items():
        print(f"  {key:13s} {full_n.count(norm(q))}   ({loc})")
    print()


# ------------------------------------------------------------------ paper's numbers
# Table 2: name -> (estimate, SE, exp(estimate) as printed)
T2 = {
    "time_id":     (0.0048, 0.0009, 1.0048),
    "crowding":    (0.0211, 0.0073, 1.0213),
    "incid_tb":    (0.0211, 0.0014, 1.0213),
    "age60":       (0.0432, 0.0077, 1.0442),
    "educ":        (-0.0324, 0.0037, 0.9681),
    "black_brown": (0.0142, 0.0032, 1.0143),
    "phc":         (-0.0025, 0.0007, 0.9975),
    "ipdm":        (-3.7159, 0.5513, 0.0243),
}
# Results text, % change per unit (printed; "1,43%" in the text is read as 1.43)
TEXT = {"time_id": 0.48, "crowding": 2.13, "incid_tb": 2.13, "age60": 4.41,
        "educ": -3.18, "black_brown": 1.43, "phc": -0.25}
# Abstract, Findings (decrease written as a negative change)
ABSTRACT = {"crowding": 3.25, "age60": 5.92, "phc": -0.30}
LABEL = {"crowding": "Household crowding (>2 residents/bedroom)",
         "age60": "Population aged 60+",
         "phc": "Primary health care coverage"}
# Table S3: final model and four sensitivity models (None = covariate not included)
S3_MODELS = ["final", "without crowding", "without education", "without Black/Brown",
             "without Municipal Development Index"]
S3 = {
    "crowding": [0.0211, None, 0.0040, 0.0344, 0.0381],
    "age60":    [0.0432, 0.0357, 0.0313, 0.0259, 0.0592],
    "phc":      [-0.0025, -0.0025, -0.0037, -0.0024, -0.0033],
}

pct = lambda b: 100 * (math.exp(b) - 1)
Z = 1.959964


def main():
    check_quotes()

    print("1. Table 2: percent change per unit, 100*(exp(beta)-1), vs Results text and abstract")
    print(f"   {'variable':12s} {'beta':>8s} {'exp(b)':>8s} {'T2 exp':>7s} {'%(beta)':>8s} "
          f"{'%(T2exp)':>8s} {'text':>6s} {'abstract':>8s}")
    for k, (b, se, e) in T2.items():
        print(f"   {k:12s} {b:8.4f} {math.exp(b):8.4f} {e:7.4f} {pct(b):8.3f} "
              f"{100*(e-1):8.2f} {str(TEXT.get(k, '')):>6s} {str(ABSTRACT.get(k, '')):>8s}")
    print()

    print("2. Rounding range: beta printed to 4 decimals, so true beta lies in [b-0.00005, b+0.00005)")
    for k in ABSTRACT:
        b = T2[k][0]
        lo, hi = pct(b - 0.00005), pct(b + 0.00005)
        print(f"   {k:9s} beta {b:+.4f}: {lo:+.3f}% to {hi:+.3f}%   (abstract {ABSTRACT[k]:+.2f}%)")
    print()

    print("3. Coefficient the abstract value would require, ln(1 + p/100), and abstract / recomputed")
    for k, p in ABSTRACT.items():
        b = T2[k][0]
        print(f"   {k:9s} abstract {p:+.2f}% needs beta {math.log(1 + p/100):+.4f} "
              f"(Table 2 {b:+.4f});  abstract/recomputed = {p/pct(b):.3f} "
              f"(overstated by {100*(p/pct(b)-1):.0f}%)")
    print()

    print("4. 95% CI implied by Table 2 (beta +/- 1.96 SE), as percent change per percentage point")
    for k in ABSTRACT:
        b, se, _ = T2[k]
        lo, hi = pct(b - Z*se), pct(b + Z*se)
        inside = lo <= ABSTRACT[k] <= hi
        print(f"   {k:9s} {pct(b):+.2f}% (95% CI {lo:+.2f} to {hi:+.2f});  abstract {ABSTRACT[k]:+.2f}% "
              f"{'inside' if inside else 'OUTSIDE'} the interval")
    print()

    print("5. Table S3 sensitivity models: 100*(exp(beta)-1) and 100*beta")
    for k, row in S3.items():
        print(f"   {k}:")
        for m, b in zip(S3_MODELS, row):
            if b is None:
                print(f"      {m:38s} not included")
            else:
                print(f"      {m:38s} beta {b:+.4f}  ->  {pct(b):+.2f}%   (100*beta = {100*b:+.2f})")
    print()

    print("6. Does any reported coefficient reproduce an abstract value (within 0.005 pp)?")
    for k, p in ABSTRACT.items():
        hits = []
        for m, b in zip(S3_MODELS, S3[k]):
            if b is None:
                continue
            if abs(round(pct(b), 2) - p) < 0.005:
                hits.append(f"{m}: 100*(exp(b)-1)")
            if abs(round(100*b, 2) - p) < 0.005:
                hits.append(f"{m}: 100*beta (not exponentiated)")
        print(f"   {k:9s} {p:+.2f}%: {hits if hits else 'no match in any reported model'}")
    nb = S3["crowding"][4], S3["age60"][4], S3["phc"][4]
    print(f"   Model without Municipal Development Index, read the same way for all three "
          f"(100*(exp(b)-1)): {pct(nb[0]):+.2f}%, {pct(nb[1]):+.2f}%, {pct(nb[2]):+.2f}%;  "
          f"(100*beta): {100*nb[0]:+.2f}, {100*nb[1]:+.2f}, {100*nb[2]:+.2f}")
    print("   Abstract trio: +3.25, +5.92, -0.30")
    print()

    print("7. Scaled to a 10-percentage-point difference, 100*(exp(10*beta)-1)")
    for k, p in ABSTRACT.items():
        b = T2[k][0]
        print(f"   {k:9s} Table 2: {pct(10*b):+.1f}%   abstract value compounded: "
              f"{100*((1+p/100)**10-1):+.1f}%")
    print()

    fig1()
    fig2()
    print("Figures written to", HERE)


# ------------------------------------------------------------------ figures
def _style():
    plt.rcParams.update({
        "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
        "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
        "svg.fonttype": "none",
    })


def _save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


def fig1():
    _style()
    fig = plt.figure(figsize=(7.2, 4.8), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
    ax.set_xlim(0, 1); ax.set_ylim(0, 1)

    ax.text(0.02, 0.955, "Same model, two sets of effect sizes", fontsize=13, weight="bold", va="top")
    ax.text(0.02, 0.900, "% change in expected TB mortality per 1 percentage-point increase, final spatial GAMLSS", fontsize=10, color=INK2, va="top")

    L0, L1, R0, R1 = 0.02, 0.34, 0.49, 0.98
    top, bot = 0.83, 0.17
    for x0, x1, head in ((L0, L1, "Abstract (Findings)"), (R0, R1, "Results text / Table 2")):
        ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.015",
                                    fc="white", ec="#c9c8c3", lw=1.2))
        ax.text(x0 + 0.015, top - 0.03, head, fontsize=11, weight="bold", va="top")

    rows = [("crowding", 0.64), ("age60", 0.46), ("phc", 0.28)]
    for k, y in rows:
        b = T2[k][0]; e = T2[k][2]; p = ABSTRACT[k]; r = pct(b)
        # left panel
        ax.text(L0 + 0.015, y + 0.035, LABEL[k].replace(" (>2 residents/bedroom)", ""),
                fontsize=10, color=INK2, va="center")
        ax.text(L0 + 0.015, y - 0.025, f"{p:+.2f}%", fontsize=14, weight="bold", color=ORANGE, va="center")
        # right panel
        ax.text(R0 + 0.015, y + 0.035, f"Table 2: β = {b:+.4f}, exp(β) = {e:.4f}", fontsize=10,
                color=INK2, va="center")
        ax.text(R0 + 0.015, y - 0.025, f"text {TEXT[k]:+.2f}%", fontsize=12, weight="bold",
                color=ORANGE, va="center")
        ax.text(R0 + 0.205, y - 0.025, f"100·(e$^{{β}}$−1) = {r:+.2f}%", fontsize=12, weight="bold",
                color=BLUE, va="center")
        # connector
        ax.annotate("", xy=(R0 - 0.005, y), xytext=(L1 + 0.005, y),
                    arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
        ax.text((L1 + R0) / 2, y + 0.022, "conflict", fontsize=10, color=INK, ha="center",
                va="bottom", weight="bold")
        ax.text((L1 + R0) / 2, y - 0.022, f"abstract {p / r:.2f}×", fontsize=10, color=INK2,
                ha="center", va="top")

    ax.text(0.02, 0.125, "Orange: as printed in the paper.  Blue: recomputed from the Table 2 coefficient with",
            fontsize=10, color=INK2, va="top")
    ax.text(0.02, 0.080, "100·(exp(β)−1), the formula the Results apply to the time trend. No coefficient",
            fontsize=10, color=INK2, va="top")
    ax.text(0.02, 0.035, "reported in Table 2 or Table S3 gives all three abstract values.",
            fontsize=10, color=INK2, va="top")
    _save(fig, "fig1")


def fig2():
    _style()
    keys = ["crowding", "age60", "phc"]
    fig, axes = plt.subplots(3, 1, figsize=(7.2, 4.8), facecolor=BG)
    fig.subplots_adjust(left=0.27, right=0.96, top=0.86, bottom=0.19, hspace=1.05)
    fig.text(0.02, 0.965, "Abstract values fall inside the 95% CIs, away from the Table 2 estimates",
             fontsize=12, weight="bold", va="top")
    fig.text(0.02, 0.915, "Percent change per 1 percentage-point increase; one panel per covariate, "
             "each with its own scale", fontsize=10, color=INK2, va="top")
    lims = {"crowding": (-0.5, 4.5), "age60": (-0.5, 7.0), "phc": (-0.5, 0.1)}
    for ax, k in zip(axes, keys):
        b, se, _ = T2[k]
        est, lo, hi = pct(b), pct(b - Z*se), pct(b + Z*se)
        ax.set_facecolor(BG)
        ax.axvline(0, color=INK2, lw=0.8, zorder=1)
        ax.plot([lo, hi], [0, 0], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
        ax.plot([est], [0], "o", ms=9, color=BLUE, zorder=3,
                label="Recomputed from Table 2: 100·(exp(β)−1), 95% CI β ± 1.96 SE")
        ax.plot([ABSTRACT[k]], [0], "D", ms=9, color=ORANGE, mec="white", mew=0.8, zorder=4,
                label="Reported in the abstract")
        ax.annotate(f"{est:+.2f}", (est, 0), xytext=(0, 9), textcoords="offset points",
                    ha="center", fontsize=10, color=BLUE)
        ax.annotate(f"{ABSTRACT[k]:+.2f}", (ABSTRACT[k], 0), xytext=(0, -17), textcoords="offset points",
                    ha="center", fontsize=10, color=ORANGE)
        ax.set_xlim(*lims[k]); ax.set_ylim(-1, 1)
        ax.set_yticks([0]); ax.set_yticklabels([LABEL[k].replace(" (>2 residents/bedroom)", "").replace("Primary health care", "Primary health\ncare")], fontsize=10.5,
                                                 color=INK)
        ax.tick_params(axis="y", length=0)
        ax.tick_params(axis="x", labelsize=10)
        ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
        for s in ("top", "right", "left"):
            ax.spines[s].set_visible(False)
        ax.spines["bottom"].set_color("#bdbcb7")
        ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:g}%"))
    h, l = axes[0].get_legend_handles_labels()
    fig.legend(h, l, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=1, frameon=False, fontsize=10,
               handletextpad=0.4, columnspacing=1.2)
    _save(fig, "fig2")


if __name__ == "__main__":
    main()

Abstract reports 15.4% survival before 24 weeks; the paper's own counts give 24.1%

Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100330 · PMC9903864

Summary

The abstract gives survival to discharge or transfer for US infants born before 24 weeks' gestation in 2007–2018 (N = 77,050) as 15.4%. The Results and Discussion give the same quantity for the same group as 18,533 survivors (24.1%), and 18,533 / 77,050 = 24.05%. That count is supported three more ways inside the paper: the reported deaths, the Table 1 subgroup rows (by sex, race, payer and birth weight, each summing to about 18,533 survivors) and the yearly trend, whose values average about 24%. No count, denominator or subgroup in the paper produces 15.4%. The paper presents these rates as a national benchmark for prenatal counselling, so a reader of the abstract alone would understate survival in this group by 8.7 percentage points (36% in relative terms). None of the paper's trend or regression results depend on the abstract figure.

What the paper reports

Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively

— Abstract, Findings

In the group of infants with GA <24 weeks (N = 77,050), only 18,533 (24.1%) survived

— Results, 2nd paragraph (the same sentence goes on: "in infants born with completed 24 weeks of gestation (n = 47,295), a total of 33,872 (71.6%) survived")

mortality among all periviable infants was 71,940 (57.86%)

— Results, 1st paragraph (periviable infants: 124,345, "of those, 77,050 infants were born <24 weeks, and 47,295 were born at 24 weeks of gestation")

The study presented the national survival of 24.1% and 71.6% for preterm infants with GA <24 weeks and completed 24 weeks, respectively

— Discussion, 1st paragraph

The majority (71.6%) of infants born with GA of 24 weeks survived, whereas only 24.1% of infants <24 weeks survived

— Discussion, 2nd paragraph

In infants with GA <24 weeks, survival started at 18.4% in 2007 and was 31.9% by the year 2018

— Results, trend paragraph (also in Research in context: "survival of infants born at GA <24 weeks increased from 18.4% to 31.9%")

Female | 34,939 | 9124 (26.1) Male | 41,768 | 9409 (22.5)

— Table 1, Gender rows, columns for GA <24 weeks (Total | Alive (%)). The footer says "All numbers represent weighted data."

This study provides national benchmarks that can be used during prenatal counselling

— Conclusions

What the paper's own numbers show

Method. I took the counts the paper reports for infants born before 24 weeks (total, survivors, deaths across all periviable infants, and every Table 1 subgroup row) and recomputed the survival percentage in each way the paper's numbers allow. I then looked for any count, denominator or subgroup that would produce 15.4%. As a check against the trend, I read the yearly values off the paper's Figure 1 (upper panel, dotted line) by pixel digitization. This was calibrated on the axis labels and matches the two stated endpoints within 0.2 points. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed
Survival, GA <24 wk, 2007–2018 Abstract, Findings 15.4% 18,533 / 77,050 = 24.05%
Survival, GA <24 wk, 2007–2018 Results 2nd para; Discussion 1st and 2nd paras 18,533 (24.1%) 24.05% (agrees)
Survival, completed 24 wk Abstract; Results; Discussion 33,872 / 47,295 (71.6%) 71.62% (agrees in all places)
Group sizes Abstract; Results 1st para 77,050 + 47,295; total 124,345 77,050 + 47,295 = 124,345 (agrees)
Mortality, all periviable Results 1st para 71,940 (57.86%) 71,940 / 124,345 = 57.86%. Survivors 124,345 − 71,940 = 52,405 = 18,533 + 33,872 exactly, so 52,405 − 33,872 = 18,533 survivors <24 wk (24.05%)
Table 1 <24 wk survivors, rows summed Table 1 subgroup counts sex 18,533 / 76,707 = 24.16%; race 18,535 / 77,050 = 24.06%; payer 18,533 / 77,051 = 24.05%; birth weight 18,534 / 77,051 = 24.05%; income 18,202 / 75,830 = 24.00%
Survivors needed for 15.4% derived — 0.154 × 77,050 = 11,866 (11,827–11,904 round to 15.4%); no such count appears in the paper
Alternative denominator derived — 18,533 / 124,345 (all ≤24 wk) = 14.90%, not 15.4%; 18,533 would need a denominator of 120,344
Ratio of 24-wk to <24-wk survival Abstract vs data 71.6 / 15.4 = 4.65-fold 71.62 / 24.05 = 2.98-fold
Yearly survival, <24 wk Research in context; Results trend para; Discussion 18.4% (2007) to 31.9% (2018) Figure 1 reading (approx.): 18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0; unweighted mean 24.27%
Pooled 15.4% vs yearly values derived — Only 2008 (≈14.5%) lies below 15.4%. A pooled 15.4% would need at least (18.2 − 15.4) / (18.2 − 14.5) = 76% of the 12-year cohort to fall in 2008

Other inconsistencies noted in passing (not part of the main finding, and not needed for it):

Quantity Location Reported Recomputed
Survival by sex, completed 24 wk Table 1 vs Results 2nd para, Abstract and Figure 1 lower panel Female 17,635 / 25,480 (69.2); Male 16,213 / 21,766 (74.5); the text says "Survival among males was lower than females in infants <24 weeks (p < 0.001) and completed 24 weeks (p < 0.001)" 69.2% and 74.5%: the counts and percentages agree, but they show higher crude survival in males, the opposite of the text
Survival by birth weight, completed 24 wk Table 1 vs Results (four-group percentages 45.1% for ≤500 g and 73.8% for >500 g) ≤500 g: 43,772 / 32,283 (45.1); >500 g: 3,523 / 1,590 (73.8) 32,283 / 43,772 = 73.8% and 1,590 / 3,523 = 45.1%: each row's counts give the other row's printed percentage
Median length of stay, survivors Results vs Table 2 text 113 and 114 days; Table 2 118 and 110 days not reconcilable from the reported numbers
Median charges, survivors Results vs Table 2 (and Discussion) text $1,050,953 and $1,016,751; Table 2 (and Discussion) $926,105 and $895,137 not reconcilable from the reported numbers

Figure

Diagram: the abstract's 15.4% set against the Results, Discussion and Table 1, which give 18,533 survivors of 77,050 (24.1%), with the recomputed 24.05% below

Figure 1. The contradiction in the paper's own text. Left: the abstract sentence, with the value at issue (15.4%, orange, as reported). Right: the same quantity elsewhere in the paper: the Results count of 18,533 survivors out of 77,050 (24.1%), the two Discussion statements of 24.1%, the Table 1 survivors by sex (9,124 + 9,409 = 18,533) and the stated trend endpoints. The connector marks that both sides describe the same group (GA <24 weeks, N = 77,050), the same period (2007–2018) and the same outcome (survival to discharge or transfer). Bottom (blue, recomputed): 18,533 / 77,050 = 24.05%. For comparison, 15.4% would require 11,866 survivors, and the nearest alternative denominator (all 124,345 periviable infants) gives 14.90%, which does not round to 15.4%. Values come from the paper or from recompute.py.txt.

Dot plot of survival before 24 weeks: the abstract value at 15.4% against the 24.1% in the Results and six values of about 24% recomputed from the paper's own counts, and the alternative denominator at 14.90%

Figure 2. Survival before 24 weeks on one percentage axis starting at 0. Orange circles are values reported in the paper. Blue diamonds are values recomputed from the paper's own counts: the direct ratio, the route through the reported deaths, and the Table 1 subgroup rows summed by sex, race/ethnicity, payer and birth weight. Every route drawn lands between 24.05% and 24.16% (the income rows, which cover 75,830 of the 77,050 infants and are not drawn, give 24.00%), matching the 24.1% in the Results and Discussion. Only the abstract sits at 15.4%. The bottom row shows that the one plausible alternative denominator gives 14.90%, not 15.4%. The shaded band runs from the reported 2007 value (18.4%) to the reported 2018 value (31.9%). It marks the stated endpoints, not the full yearly range (the paper's Figure 1 shows a lower value, about 14.5%, in 2008). The yearly values themselves are not drawn.

Why it matters

Explanations considered

None of these accounts for 15.4%. The paper's counts, its tables and its trend all support 24.1%.

Status

As of 2026-10-23:

Suggested correction

In the Abstract, Findings, replace "Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively" with "Survival rates for infants <24 weeks and with completed 24 weeks were 24.1% and 71.6%, respectively", consistent with 18,533 / 77,050 in the Results, Discussion and Table 1. The same correction should be passed to PubMed and other indexes. The authors may also wish to check:

Recomputation script (python recompute.py.txt)
"""Recompute the survival figures in PMC9903864 (Lancet Reg Health Am 2022,
doi:10.1016/j.lana.2022.100330) and regenerate fig1/fig2 in this folder.

Run:  python recompute.py.txt
Needs numpy and matplotlib.

Every input below is copied from the article as archived in PubMed Central,
except YEARLY_LT24_APPROX, which is an approximate pixel reading of the
paper's Figure 1 (upper panel, dotted line). That reading is used only for the
pooling check in section 5 and is not drawn in either figure.
"""
from pathlib import Path

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = Path(__file__).resolve().parent

# ------------------------------------------------------------------ inputs (paper)
N_ALL = 124_345            # Abstract; Results, 1st paragraph: infants born at GA <=24 wk
N_LT24 = 77_050            # Abstract; Results, 1st paragraph; Table 1 header
N_24 = 47_295              # Abstract; Results, 1st paragraph; Table 1 header
ALIVE_LT24 = 18_533        # Results, 2nd paragraph: "only 18,533 (24.1%) survived"
ALIVE_24 = 33_872          # Results, 2nd paragraph: "a total of 33,872 (71.6%) survived"
DEATHS_ALL = 71_940        # Results, 1st paragraph: "mortality among all periviable infants was 71,940 (57.86%)"
ABSTRACT_LT24 = 15.4       # Abstract, Findings
ABSTRACT_24 = 71.6         # Abstract, Findings
RESULTS_LT24 = 24.1        # Results, 2nd paragraph; Discussion, 1st and 2nd paragraphs
TREND_2007, TREND_2018 = 18.4, 31.9   # Research in context; Results trend para; Discussion

# Table 1, GA <24 weeks: (total, alive) per row
T1_LT24 = {
    "sex":       [(34_939, 9_124), (41_768, 9_409)],
    "race":      [(22_834, 5_129), (21_632, 5_809), (11_801, 3_173), (20_783, 4_424)],
    "payer":     [(37_434, 10_607), (28_728, 6_326), (10_889, 1_600)],
    "income":    [(27_011, 6_706), (18_603, 4_577), (17_113, 4_017), (13_103, 2_902)],
    "birth weight": [(36_392, 2_698), (40_659, 15_836)],
}
# Table 1, completed 24 weeks: (total, alive, printed %)
T1_24_SEX = {"Female": (25_480, 17_635, 69.2), "Male": (21_766, 16_213, 74.5)}
T1_24_BW = {"<=500 g": (43_772, 32_283, 45.1), ">500 g": (3_523, 1_590, 73.8)}
# Results trend text: four GA x BW groups
FOUR_GROUPS_TEXT = {"<24 wk, <=500 g": 7.4, "<24 wk, >500 g": 38.9,
                    "24 wk, <=500 g": 45.1, "24 wk, >500 g": 73.8}
# Results text vs Table 2 (surviving infants): medians
LOS_TEXT, LOS_T2 = (113, 114), (118, 110)
CHARGES_TEXT, CHARGES_T2 = (1_050_953, 1_016_751), (926_105, 895_137)

# Approximate pixel reading of Figure 1, upper panel, dotted line (<24 wk), 2007..2018.
# Calibrated on the 0%..80% axis labels; it reproduces the stated endpoints within 0.2 pt.
YEARS = list(range(2007, 2019))
YEARLY_LT24_APPROX = [18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0]


def pct(a, b):
    return 100.0 * a / b


out = {}
print("=" * 74)
print("1. Headline survival, GA <24 weeks and completed 24 weeks")
out["lt24"] = pct(ALIVE_LT24, N_LT24)
out["w24"] = pct(ALIVE_24, N_24)
print(f"   N_LT24 + N_24 = {N_LT24 + N_24:,} (reported {N_ALL:,})")
print(f"   <24 wk : 18,533 / 77,050 = {out['lt24']:.2f}%   (Abstract {ABSTRACT_LT24}%, Results/Discussion {RESULTS_LT24}%)")
print(f"   24 wk  : 33,872 / 47,295 = {out['w24']:.2f}%   (Abstract {ABSTRACT_24}%, Results 71.6%)")
print(f"   difference Abstract vs recomputed: {out['lt24'] - ABSTRACT_LT24:.2f} percentage points; "
      f"relative understatement {100 * (1 - ABSTRACT_LT24 / out['lt24']):.0f}%")
print(f"   1 survivor per {100 / ABSTRACT_LT24:.1f} births (15.4%) vs 1 per {100 / out['lt24']:.1f} births (24.05%)")
print(f"   24 wk / <24 wk survival ratio: abstract {ABSTRACT_24 / ABSTRACT_LT24:.2f}x, data {out['w24'] / out['lt24']:.2f}x")

print("\n2. Route via the reported deaths (Results, 1st paragraph)")
surv_all = N_ALL - DEATHS_ALL
out["mort"] = pct(DEATHS_ALL, N_ALL)
out["via_deaths"] = pct(surv_all - ALIVE_24, N_LT24)
print(f"   mortality 71,940 / 124,345 = {out['mort']:.2f}% (reported 57.86%)")
print(f"   survivors 124,345 - 71,940 = {surv_all:,};  18,533 + 33,872 = {ALIVE_LT24 + ALIVE_24:,}")
print(f"   <24 wk survivors implied = {surv_all - ALIVE_24:,} -> {out['via_deaths']:.2f}% of 77,050")

print("\n3. Table 1 subgroup rows, GA <24 weeks, summed")
out["t1"] = {}
for name, rows in T1_LT24.items():
    T = sum(r[0] for r in rows)
    A = sum(r[1] for r in rows)
    out["t1"][name] = pct(A, T)
    print(f"   by {name:<13}: total {T:>6,}  alive {A:>6,}  -> {pct(A, T):.2f}%")

print("\n4. What would be needed for the abstract value")
need = ABSTRACT_LT24 / 100 * N_LT24
print(f"   survivors needed for 15.4% of 77,050: {need:,.0f} (no such count in the paper)")
lo, hi = (ABSTRACT_LT24 - 0.05) / 100 * N_LT24, (ABSTRACT_LT24 + 0.05) / 100 * N_LT24
print(f"   range that rounds to 15.4%: {lo:,.0f} to {hi:,.0f} survivors")
out["alt_den"] = pct(ALIVE_LT24, N_ALL)
print(f"   alternative denominator 18,533 / 124,345 (all periviable) = {out['alt_den']:.2f}%")
print(f"   denominator that would turn 18,533 into 15.4%: {ALIVE_LT24 / (ABSTRACT_LT24 / 100):,.0f}")
print(f"   all survivors / all periviable: {pct(ALIVE_LT24 + ALIVE_24, N_ALL):.2f}%")
print(f"   ungrouped Table 1 rows (<24 wk) that print 15.x%: "
      f"{[round(pct(a, t), 1) for rows in T1_LT24.values() for t, a in rows if 15 <= pct(a, t) < 16] or 'none'}")
print(f"   digit check: '15.4' is a permutation of '24.1'? {sorted('15.4') == sorted('24.1')}")

print("\n5. Pooling check against the yearly trend")
eq = (ABSTRACT_LT24 * 12 - TREND_2007 - TREND_2018) / 10
eq2 = (out["lt24"] * 12 - TREND_2007 - TREND_2018) / 10
print(f"   equal yearly weights, stated endpoints only: other 10 years must average {eq:.2f}% for 15.4%, "
      f"{eq2:.2f}% for 24.05%")
y = np.array(YEARLY_LT24_APPROX)
print(f"   Figure 1 reading (approx.): {', '.join(f'{v:.1f}' for v in y)}")
print(f"   reading vs stated endpoints: 2007 {y[0]:.1f} vs {TREND_2007}; 2018 {y[-1]:.1f} vs {TREND_2018}")
print(f"   unweighted mean of yearly values: {y.mean():.2f}%;  minimum {y.min():.1f}% ({YEARS[int(y.argmin())]})")
others_min = np.delete(y, y.argmin()).min()
share = (others_min - ABSTRACT_LT24) / (others_min - y.min())
print(f"   for a pooled 15.4%, the {YEARS[int(y.argmin())]} share of the cohort must be at least "
      f"({others_min:.1f} - 15.4) / ({others_min:.1f} - {y.min():.1f}) = {100 * share:.0f}%")
print(f"   average annual cohort if evenly spread: 77,050 / 12 = {N_LT24 / 12:,.0f}")

print("\n6. Side issues in Table 1 / Table 2 (not part of the main finding)")
for k, (t, a, p) in T1_24_SEX.items():
    print(f"   24 wk {k:<6}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p})")
print("   -> Table 1 shows higher crude survival for males at 24 wk; the text says males lower")
for k, (t, a, p) in T1_24_BW.items():
    print(f"   24 wk {k:<7}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p}; text gives {k} = "
          f"{FOUR_GROUPS_TEXT['24 wk, ' + k]}%)")
print("   -> each 24-wk birth-weight row's counts give the other row's printed percentage")
print(f"   24 wk sex rows sum: total {sum(v[0] for v in T1_24_SEX.values()):,}, "
      f"alive {sum(v[1] for v in T1_24_SEX.values()):,}")
print(f"   LOS median, text {LOS_TEXT} vs Table 2 {LOS_T2} days")
print(f"   charges median, text {CHARGES_TEXT} vs Table 2 {CHARGES_T2} USD")

# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(HERE / f"{stem}.svg")
    fig.savefig(HERE / f"{stem}.png", dpi=200)
    plt.close(fig)


# Figure 1: abstract vs body contradiction diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")


def panel(x, y, w, h, title, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, fontweight="bold", va="top")


# left: abstract
panel(0.2, 1.55, 2.85, 2.65, "Abstract, Findings", ORANGE)
ax.text(0.34, 3.72, "“Survival rates for infants\n<24 weeks and with completed\n24 weeks were",
        fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.34, 2.82, "15.4%", fontsize=22, fontweight="bold", color=ORANGE, va="top")
ax.text(1.48, 2.72, "and 71.6%,\nrespectively”", fontsize=10, va="top", color=INK2,
        linespacing=1.35)
ax.text(0.34, 1.95, "N = 77,050 (<24 wk)", fontsize=10, color=INK2, va="top")

# right: results / discussion / table 1
panel(4.15, 1.55, 2.85, 2.65, "Results, Discussion, Table 1", INK2)
ax.text(4.29, 3.72, "Results: “(N = 77,050), only\n18,533 (24.1%) survived”",
        fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 3.0, "Discussion (twice): 24.1%", fontsize=10, va="top", color=INK2)
ax.text(4.29, 2.6, "Table 1, <24 wk alive by sex:\n9,124 + 9,409 = 18,533",
        fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 1.95, "Trend: 18.4% (2007) to 31.9% (2018)", fontsize=10, color=INK2, va="top")

# connector
ax.add_patch(FancyArrowPatch((3.12, 2.9), (4.08, 2.9), arrowstyle="<->", mutation_scale=14,
                             lw=2, color=INK))
ax.text(3.6, 3.2, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 2.7, "same group,\nperiod and\noutcome", ha="center", va="top",
        fontsize=10, color=INK2, linespacing=1.3)

# recomputation strip
ax.add_patch(FancyBboxPatch((0.2, 0.3), 6.8, 0.95, boxstyle="round,pad=0.02,rounding_size=0.08",
                            fc="white", ec=BLUE, lw=1.5))
ax.text(0.34, 1.07, "Recomputed from the paper's own counts", fontsize=11, fontweight="bold",
        va="top")
ax.text(0.34, 0.68, f"18,533 / 77,050 = {out['lt24']:.2f}%", fontsize=13, fontweight="bold",
        color=BLUE, va="top")
ax.text(3.35, 0.73, f"15.4% would need {need:,.0f} survivors;\n"
        f"18,533 / 124,345 (all ≤24 wk) = {out['alt_den']:.2f}%",
        fontsize=10, color=INK2, va="top", linespacing=1.35)
ax.legend(handles=[Line2D([], [], marker="s", ls="", ms=9, color=ORANGE, label="value as reported (abstract)"),
                   Line2D([], [], marker="s", ls="", ms=9, color=BLUE, label="recomputed from paper's counts")],
          loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")

# Figure 2: dot comparison on one percentage axis starting at 0
rows = [
    ("Abstract, Findings", ABSTRACT_LT24, "rep"),
    ("Results; Discussion (×2)", RESULTS_LT24, "rep"),
    ("18,533 / 77,050", out["lt24"], "rec"),
    ("(124,345 − 71,940 deaths − 33,872) / 77,050", out["via_deaths"], "rec"),
    ("Table 1 rows summed: sex", out["t1"]["sex"], "rec"),
    ("Table 1 rows summed: race/ethnicity", out["t1"]["race"], "rec"),
    ("Table 1 rows summed: payer", out["t1"]["payer"], "rec"),
    ("Table 1 rows summed: birth weight", out["t1"]["birth weight"], "rec"),
    ("Alternative denominator: 18,533 / 124,345", out["alt_den"], "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.47, right=0.97, top=0.86, bottom=0.13)
n = len(rows)
ypos = np.arange(n)[::-1]
ax.axvspan(TREND_2007, TREND_2018, color="#efeeea", zorder=0)
ax.text((TREND_2007 + TREND_2018) / 2, n - 0.35, "shaded: from the reported 2007\n(18.4%) to 2018 (31.9%) value",
        ha="center", va="bottom", fontsize=10, color=INK2)
for yv, (lab, v, kind) in zip(ypos, rows):
    c = ORANGE if kind == "rep" else BLUE
    mk = "o" if kind == "rep" else "D"
    ax.plot([0, v], [yv, yv], color=GRID, lw=1, zorder=1)
    ax.plot(v, yv, marker=mk, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="")
    ax.text(v + 0.9, yv, f"{v:.2f}%" if kind == "rec" else f"{v:.1f}%", va="center",
            fontsize=10, color=INK)
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 40)
ax.set_ylim(-0.7, n + 0.6)
ax.set_xticks(range(0, 41, 5))
ax.set_xticklabels([f"{t}%" for t in range(0, 41, 5)])
ax.set_xlabel("Survival, GA <24 weeks, 2007–2018 pooled")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in the paper"),
                   Line2D([], [], marker="D", ls="", ms=9, color=BLUE, label="recomputed from the paper's counts")],
          loc="lower left", bbox_to_anchor=(-0.9, 1.08), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nfigures written:", ", ".join(sorted(p.name for p in HERE.glob("fig*.*"))))

Abstract reports low-birth-weight estimates, including a nominally significant ASD effect, as preterm-birth results

Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101247 · PMC11910110

Summary

The abstract's Findings say that, compared with term birth, preterm birth was associated with higher ASD symptoms, 3·23 (95% CI 0·37, 6·08) percentiles, and give preterm effects of 4·15 for ADHD symptoms and −5·44 for non-verbal IQ. All three values, estimates and both CI limits, are the cells of the birth weight <2500 g row of Table 4, not the gestational age <37 wk row. Table 4's preterm row gives ASD 0·60 (−2·60, 3·79), which is null (p ≈ 0·71 recomputed from the CI), ADHD 2·51 and non-verbal IQ −3·41; the Results and Discussion agree with the table and state explicitly that preterm birth was not associated with ASD symptoms. A reader of the abstract alone would therefore take away a preterm–ASD association that the paper's own analysis did not find, and preterm effects on ADHD and IQ about 1·6 times larger than estimated.

What the paper reports

Abstract, Findings (second sentence):

As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)) and externalizing problems (difference 2·31 (95% CI: 1·16, 3·46)), ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)), ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)), and lower non-verbal IQ (difference -5·44 (95% CI: -7·44, -3·44)).

Results, "Birth outcomes, emotional, behavior and cognitive outcomes", paragraph 2:

As compared to children born term, those born preterm had increased scores for ADHD symptoms in childhood (2·51 percentiles (95% CI: 1·61, 3·40)) and early adolescence (2·33 percentiles (95% CI: 1·16, 3·50)).

Not preterm birth, but low birth weight was associated with ASD symptoms (3·23 percentiles (95% CI: 0·37, 6·08)).

Preterm birth and low birth weight were also associated with a -3·41 percentiles (95% CI: -5·46, -1·36) and -5·44 percentiles (95% CI: -7·44, -3·44) lower Non-verbal IQ in childhood respectively.

Discussion, paragraph 4:

The lack of association for preterm birth might also be explained by lower numbers since ASD was only available in two cohorts, that have both a rather healthy population, without extremely premature children.

Table 4 (confounder model; childhood ADHD, ASD and non-verbal IQ columns), as printed:

<37 | 2·51 (1·61, 3·40)∗∗n = 4522 | … | 0·60 (-2·60, 3·79)n = 279 | -3·41 (-5·46, -1·36)∗∗n = 780

<2500 | 4·15 (3·15, 5·16)∗∗n = 3531 | … | 3·23 (0·37, 6·08)∗N = 353 | -5·44 (-7·44, -3·44)∗∗N = 821

Table 4 footer: "∗p value < 0·05, ∗∗p value < 0·01". Methods, Statistical analysis:

To correct for multiple testing, we used the Bonferroni correction method taking account for five outcomes and specified multiple-test corrected p-values as <0·01.

What the paper's own numbers show

Method. Each number in the abstract's "preterm birth" sentence was matched, estimate and both CI limits, against every relevant cell of Tables 3 and 4 (confounder model). For each cell a standard error was recovered from the printed 95% CI as SE = (upper − lower)/3·92 and a two-sided Wald p-value computed. The basic-model (Table S5) and complete-case (Table S9) results in the paper's Appendix A were checked the same way, to rule out that the abstract quotes a different model or analysis set. All arithmetic is in recompute.py.txt.

Quantity Location Reported Recomputed / located
ASD symptoms, "preterm birth" Abstract, Findings 3·23 (0·37, 6·08) Identical to Table 4 <2500 g ASD cell (N = 353); not in the <37 wk row
ASD symptoms, preterm <37 wk Table 4 0·60 (−2·60, 3·79), n = 279, no asterisk SE 1·630, z 0·37, p = 0·71; CI includes 0
ASD symptoms, birth weight <2500 g Table 4; Results 3·23 (0·37, 6·08)∗ SE 1·457, z 2·22, p = 0·027 (matches ∗; does not meet the paper's Bonferroni p < 0·01)
ADHD symptoms (childhood), "preterm birth" Abstract, Findings 4·15 (3·15, 5·16) Identical to Table 4 <2500 g cell; preterm cell is 2·51 (1·61, 3·40), p < 0·001; abstract/preterm ratio 1·65
Non-verbal IQ, "preterm birth" Abstract, Findings −5·44 (−7·44, −3·44) Identical to Table 4 <2500 g cell; preterm cell is −3·41 (−5·46, −1·36), p = 0·001; ratio 1·60
Internalizing (childhood), "preterm birth" Abstract, Findings 3·43 (2·52, 4·33) Identical to Table 3 <37 wk childhood cell (correct)
Externalizing, "preterm birth" Abstract, Findings 2·31 (1·16, 3·46) Table 3 <37 wk early-adolescence cell; the childhood cell is 2·28 (1·43, 3·13). Correct exposure, different age window from the other four numbers (minor)
ASD, preterm, basic model Appendix A, Table S5 1·35 (−1·89, 4·60) z 0·82, p = 0·41; not the abstract value
ASD, <2500 g, basic model Appendix A, Table S5 3·71 (0·82, 6·60)∗ not the abstract value; no basic-model cell equals any abstract value
ASD, preterm, complete-case Appendix A, Table S9 −0·50 (−4·41, 3·41) z −0·25, p = 0·80; no complete-case cell equals any abstract value
LGA vs AGA, ADHD (side finding) Abstract vs Table 5 Abstract −1·09 (−1·73, 0·45); Table 5 −1·09 (−1·73, −0·45)∗ With +0·45 the CI midpoint is −0·64, not −1·09, and the interval would include 0; Table 5's interval is symmetric about −1·09 (p = 0·0008). The abstract upper limit has lost its minus sign

Three of the five numbers in the "preterm birth" sentence (ADHD, ASD, non-verbal IQ) reproduce the confounder-model <2500 g row of Table 4 exactly, to two decimals in all three positions; none of them matches any preterm cell in the main, basic-model or complete-case tables, in any age window. In every one of these analyses the preterm ASD estimate is null. The first two numbers (internalizing, externalizing) do come from the preterm row of Table 3.

Figure

Diagram: the abstract's ASD estimate attributed to preterm birth is the Table 4 low-birth-weight cell; the Table 4 preterm cell is null

Figure 1. The abstract's ASD value (orange, as reported) is identical in estimate and both CI limits to the Table 4 row for birth weight <2500 g, not the row for gestational age <37 wk. The preterm row, 0·60 (−2·60, 3·79) with n = 279, has a recomputed p of 0·71; the low-birth-weight row has p = 0·027 (blue, recomputed from the printed 95% CIs). The Results and Discussion text (right) attribute the association to low birth weight and describe a lack of association for preterm birth.

Dot-and-interval plot of the five abstract "preterm birth" values next to the paper's preterm estimates

Figure 2. For each outcome in the abstract's "preterm birth" sentence, the value stated in the abstract (orange circles) is plotted against the preterm (<37 wk vs 37–42 wk) childhood estimate from Tables 3 and 4 (blue squares), with 95% CIs on one axis of outcome-percentile differences and a reference line at no difference. The right-hand column names the table cell each abstract value reproduces and the recomputed p-value of the preterm cell. Internalizing and externalizing agree; ADHD and non-verbal IQ are overstated by about 60%; for ASD the abstract's interval excludes 0 while the paper's preterm interval is centred near 0.

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the Abstract, Findings, replace the second sentence so that each estimate is attributed to its own exposure and age window, for example:

As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)), externalizing problems (difference 2·28 (95% CI: 1·43, 3·13)) and ADHD symptoms (difference 2·51 (95% CI: 1·61, 3·40)) in childhood, and with lower non-verbal IQ (difference −3·41 (95% CI: −5·46, −1·36)), but not with ASD symptoms (difference 0·60 (95% CI: −2·60, 3·79)). Low birth weight (<2500 g) was associated with higher ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)) and ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)) and lower non-verbal IQ (difference −5·44 (95% CI: −7·44, −3·44)).

If the early-adolescence externalizing value (2·31 (1·16, 3·46)) is preferred, label it as early adolescence. Consider noting that the low-birth-weight ASD association (p ≈ 0·03) does not meet the paper's Bonferroni threshold of p < 0·01. In the next sentence on large size for gestational age, correct the ADHD interval to (95% CI: −1·73, −0·45), as in Table 5. The same corrections apply to the abstract text repeated in the STROBE checklist (Appendix A, item 1(b)).

Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11910110
(Lancet Reg Health Eur 2025; doi:10.1016/j.lanepe.2025.101247), and redraw
fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy, scipy, matplotlib)

Every estimate below is copied from the article as archived in PubMed Central
(Abstract Findings, Results text, Tables 3, 4 and 5; confounder model), plus
the basic-model Table S5 and complete-case Table S9 from the paper's Appendix A
(mmc1.docx) as cross-checks.
Values are differences in outcome percentiles with 95% CIs.
The only assumption is the usual normal approximation used to recover a
standard error from a printed 95% CI: SE = (upper - lower) / (2 * 1.959964).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)

# ------------------------------------------------------------------ inputs
# Table cells: key -> (estimate, lower, upper, n exposed)
T = {
    # Table 3 (internalizing / externalizing), gestational age <37 wk vs 37-42 wk
    ("T3", "<37 wk", "Internalizing", "childhood"):        (3.43, 2.52, 4.33, 4780),
    ("T3", "<37 wk", "Internalizing", "early adolescence"): (1.77, 0.57, 2.96, 2873),
    ("T3", "<37 wk", "Externalizing", "childhood"):        (2.28, 1.43, 3.13, 4777),
    ("T3", "<37 wk", "Externalizing", "early adolescence"): (2.31, 1.16, 3.46, 2873),
    # Table 3, birth weight <2500 g vs 2500-4500 g
    ("T3", "<2500 g", "Internalizing", "childhood"):       (3.20, 2.20, 4.20, 3887),
    ("T3", "<2500 g", "Externalizing", "childhood"):       (3.77, 2.83, 4.71, 3885),
    # Table 4 (ADHD / ASD / non-verbal IQ), gestational age <37 wk
    ("T4", "<37 wk", "ADHD", "childhood"):                 (2.51, 1.61, 3.40, 4522),
    ("T4", "<37 wk", "ADHD", "early adolescence"):         (2.33, 1.16, 3.50, 2878),
    ("T4", "<37 wk", "ADHD", "late adolescence"):          (-1.74, -2.89, -0.58, 2668),
    ("T4", "<37 wk", "ASD", "childhood"):                  (0.60, -2.60, 3.79, 279),
    ("T4", "<37 wk", "Non-verbal IQ", "childhood"):        (-3.41, -5.46, -1.36, 780),
    # Table 4, birth weight <2500 g
    ("T4", "<2500 g", "ADHD", "childhood"):                (4.15, 3.15, 5.16, 3531),
    ("T4", "<2500 g", "ADHD", "early adolescence"):        (3.51, 2.15, 4.87, 2078),
    ("T4", "<2500 g", "ASD", "childhood"):                 (3.23, 0.37, 6.08, 353),
    ("T4", "<2500 g", "Non-verbal IQ", "childhood"):       (-5.44, -7.44, -3.44, 821),
    # Table 5, large for gestational age, ADHD
    ("T5", "LGA", "ADHD", "childhood"):                    (-1.09, -1.73, -0.45, 9718),
}

# Abstract, Findings: "As compared to term birth, preterm birth was associated
# with higher internalizing problems (...), externalizing problems (...), ADHD
# symptoms (...), ASD symptoms (...), and lower non-verbal IQ (...)."
ABSTRACT = [
    ("Internalizing", 3.43, 2.52, 4.33),
    ("Externalizing", 2.31, 1.16, 3.46),
    ("ADHD", 4.15, 3.15, 5.16),
    ("ASD", 3.23, 0.37, 6.08),
    ("Non-verbal IQ", -5.44, -7.44, -3.44),
]
# Abstract, LGA sentence: "lower ADHD symptoms (difference -1.09 (95% CI: -1.73, 0.45))"
ABSTRACT_LGA = (-1.09, -1.73, 0.45)

BONFERRONI_ALPHA = 0.01   # Methods: "specified multiple-test corrected p-values as <0.01"


def wald(est, lo, hi):
    se = (hi - lo) / (2 * Z975)
    z = est / se
    return se, z, 2 * stats.norm.sf(abs(z))


def fmt(x):
    return f"{x:+.2f}"


# ------------------------------------------------------------ 1. cell checks
print("=== 1. Table cells: SE from 95% CI, Wald z, two-sided p ===")
REC = {}
for k, (e, l, h, n) in T.items():
    se, z, p = wald(e, l, h)
    REC[k] = (se, z, p)
    print(f"  {k[0]} {k[1]:8s} {k[2]:14s} {k[3]:18s} est {fmt(e)} "
          f"({fmt(l)}, {fmt(h)}) n={n:<5d} CI midpoint {(l+h)/2:+.3f}  "
          f"SE {se:.3f}  z {z:+.2f}  p {p:.4f}")

# -------------------------------------- 2. where the abstract numbers come from
print("\n=== 2. Abstract 'preterm birth' sentence: which table cell each number matches ===")
MATCH = {}
for name, e, l, h in ABSTRACT:
    hits = [k for k, v in T.items() if v[:3] == (e, l, h)]
    MATCH[name] = hits
    pre = T[(("T3" if name in ("Internalizing", "Externalizing") else "T4"),
             "<37 wk", name, "childhood")]
    print(f"  {name:14s} abstract {fmt(e)} ({fmt(l)}, {fmt(h)})")
    print(f"      matches: {[' / '.join(k) for k in hits]}")
    print(f"      Table preterm (<37 wk) childhood cell: {fmt(pre[0])} "
          f"({fmt(pre[1])}, {fmt(pre[2])}); abstract / preterm ratio {e/pre[0]:.2f}")

# --------------------------------------------------------------- 3. ASD focus
print("\n=== 3. ASD symptoms in childhood (Table 4) ===")
se_pt, z_pt, p_pt = REC[("T4", "<37 wk", "ASD", "childhood")]
se_lb, z_lb, p_lb = REC[("T4", "<2500 g", "ASD", "childhood")]
print(f"  preterm <37 wk : 0.60 (-2.60, 3.79), SE {se_pt:.3f}, z {z_pt:.2f}, p {p_pt:.3f}; "
      f"CI contains 0: {-2.60 < 0 < 3.79}")
print(f"  LBW <2500 g    : 3.23 (0.37, 6.08),  SE {se_lb:.3f}, z {z_lb:.2f}, p {p_lb:.3f}; "
      f"passes Bonferroni p<{BONFERRONI_ALPHA}: {p_lb < BONFERRONI_ALPHA}")
print(f"  Table 4 flags the LBW cell with a single asterisk (p<0.05), consistent with p {p_lb:.3f}")

# -------------------------------------------------- 4. size of the overstatement
print("\n=== 4. Overstatement of preterm effects in the abstract (ADHD, non-verbal IQ) ===")
for name in ("ADHD", "Non-verbal IQ"):
    a = dict((x[0], x[1:]) for x in ABSTRACT)[name][0]
    pre = T[("T4", "<37 wk", name, "childhood")]
    p_pre = REC[("T4", "<37 wk", name, "childhood")][2]
    print(f"  {name:14s} abstract {fmt(a)} vs preterm {fmt(pre[0])}: "
          f"difference {a-pre[0]:+.2f}, ratio {a/pre[0]:.2f}, preterm p {p_pre:.4f}")

# ------------------------------------------------ 5. externalizing age window
print("\n=== 5. Externalizing: age window ===")
for w in ("childhood", "early adolescence"):
    e, l, h, n = T[("T3", "<37 wk", "Externalizing", w)]
    print(f"  preterm externalizing, {w:18s}: {fmt(e)} ({fmt(l)}, {fmt(h)})")
print("  abstract uses +2.31 (early adolescence) next to internalizing +3.43 (childhood)")

# --------------------------------------------- 6. LGA ADHD sign (side finding)
print("\n=== 6. LGA ADHD CI in abstract vs Table 5 ===")
e, l, h = ABSTRACT_LGA
print(f"  abstract: {fmt(e)} ({fmt(l)}, {fmt(h)}): midpoint {(l+h)/2:+.2f}, CI contains 0: {l < 0 < h}")
e5, l5, h5, _ = T[("T5", "LGA", "ADHD", "childhood")]
se5, z5, p5 = REC[("T5", "LGA", "ADHD", "childhood")]
print(f"  Table 5 : {fmt(e5)} ({fmt(l5)}, {fmt(h5)}): midpoint {(l5+h5)/2:+.2f}, p {p5:.4f}")

# ------------------------- 7. basic model (Appendix A, Table S5) as a cross-check
# Supplement Table S5 (basic model: sex, age at outcome, cohort), childhood columns.
S5 = {
    ("<37 wk", "ADHD"): (3.26, 2.36, 4.16),
    ("<37 wk", "ASD"): (1.35, -1.89, 4.60),
    ("<37 wk", "Non-verbal IQ"): (-3.58, -5.67, -1.49),
    ("<2500 g", "ADHD"): (5.15, 4.14, 6.16),
    ("<2500 g", "ASD"): (3.71, 0.82, 6.60),
    ("<2500 g", "Non-verbal IQ"): (-5.68, -7.71, -3.65),
}
print("\n=== 7. Basic model (Table S5): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S5.items():
    se, z, p = wald(e, l, h)
    a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
    print(f"  {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)})  z {z:+.2f}  p {p:.4f}  "
          f"equals abstract value: {(e, l, h) == a}")

# --------------- 8. complete-case analysis (Appendix A, Table S9) as a cross-check
S9 = {
    ("<37 wk", "ADHD"): (2.40, 1.24, 3.56),
    ("<37 wk", "ASD"): (-0.50, -4.41, 3.41),
    ("<37 wk", "Non-verbal IQ"): (-2.87, -5.28, -0.45),
    ("<2500 g", "ADHD"): (4.37, 3.02, 5.72),
    ("<2500 g", "ASD"): (2.11, -1.28, 5.50),
    ("<2500 g", "Non-verbal IQ"): (-4.53, -6.90, -2.17),
}
print("\n=== 8. Complete-case analysis (Table S9): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S9.items():
    se, z, p = wald(e, l, h)
    a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
    print(f"  {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)})  z {z:+.2f}  p {p:.4f}  "
          f"equals abstract value: {(e, l, h) == a}")

# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "path",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)
    print(f"  wrote {stem}.svg and {stem}.png")


print("\n=== Figures ===")

# ---- fig1: abstract vs Table 4 / Results diagram (ASD)
fig = plt.figure(figsize=(7.2, 5.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 80)
ax.axis("off")
renderer = fig.canvas.get_renderer()


def panel(x, y, w, h, title):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
                                fc="white", ec="#c9c7c1", lw=1.2))
    ax.text(x + 1.8, y + h - 2.6, title, fontsize=11, fontweight="bold", va="top")


def runs(x, y, parts):
    """Draw text segments one after another on a line; return each segment's extent.
    A part is (text, kwargs) or (text, kwargs, gap) where gap is extra space before it."""
    ends = []
    for part in parts:
        txt, kw = part[0], part[1]
        x += part[2] if len(part) > 2 else 0.0
        t = ax.text(x, y, txt, fontsize=10.5, va="top", **kw)
        bb = t.get_window_extent(renderer).transformed(ax.transData.inverted())
        x = bb.x1
        ends.append((bb.x0, bb.x1, bb.y0))
    return ends


# Abstract panel (top)
panel(2, 55, 96, 23, "Abstract, Findings")
runs(3.8, 71.0, [("\u201cAs compared to term birth, preterm birth was associated with higher", {})])
runs(3.8, 67.2, [("internalizing problems (\u2026) and externalizing problems (\u2026), ADHD symptoms (\u2026),", {})])
seg = runs(3.8, 63.4, [("ASD symptoms (difference", {}),
                       ("3\u00b723 (95% CI: 0\u00b737, 6\u00b708)", dict(color=ORANGE, fontweight="bold"), -0.65),
                       ("), and lower", {}, -0.7)])
runs(3.8, 59.6, [("non-verbal IQ (\u2026).\u201d", {})])
val_x0, val_x1, val_y0 = seg[1]

# Table 4 panel (bottom left)
panel(2, 8, 56, 33, "Table 4, ASD symptoms (childhood)")
runs(3.8, 33.6, [("Gestational age <37 wk (preterm)", dict(fontweight="bold"))])
runs(3.8, 29.8, [("0\u00b760 (\u22122\u00b760, 3\u00b779), n = 279, no asterisk", {})])
runs(3.8, 26.0, [(f"recomputed: z = {z_pt:.2f}, p = {p_pt:.2f}", dict(color=BLUE, fontweight="bold"))])
runs(3.8, 20.0, [("Birth weight <2500 g (low birth weight)", dict(fontweight="bold"))])
lbw = runs(3.8, 16.2, [("3\u00b723 (0\u00b737, 6\u00b708)*, N = 353", {})])
runs(3.8, 12.4, [(f"recomputed: z = {z_lb:.2f}, p = {p_lb:.3f}", dict(color=BLUE, fontweight="bold"))])

# Results / Discussion panel (bottom right)
panel(61, 8, 37, 33, "Results and Discussion")
for i, line in enumerate(["\u201cNot preterm birth, but low birth", "weight was associated with ASD",
                          "symptoms (3\u00b723 percentiles", "(95% CI: 0\u00b737, 6\u00b708)).\u201d"]):
    runs(62.8, 33.6 - 3.8 * i, [(line, {})])
runs(62.8, 16.2, [("\u201cThe lack of association for", {})])
runs(62.8, 12.4, [("preterm birth \u2026\u201d (Discussion)", {})])

# connector: abstract value -> low-birth-weight row, routed down the left panel's right margin
y_lbw = 16.2 - 1.6
vx = (val_x0 + val_x1) / 2
x_lane, y_jog = 59.5, 50.0
ax.plot([vx, vx, x_lane, x_lane], [val_y0 - 0.6, y_jog, y_jog, y_lbw], color=INK, lw=2,
        solid_joinstyle="round", solid_capstyle="round")
ax.add_patch(FancyArrowPatch((x_lane, y_lbw), (lbw[0][1] + 1.0, y_lbw), arrowstyle="-|>",
                             mutation_scale=14, lw=2, color=INK))
ax.text(x_lane + 2.0, 46.5, "identical estimate and CI:\nTable 4 low-birth-weight row",
        fontsize=10, ha="left", va="center", color=INK)
ax.text(3.0, 46.5, "Labelled \u2018preterm birth\u2019 in the abstract;\nthe preterm row is 0\u00b760, p = 0\u00b771",
        fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 4.2, "Orange: as reported in the abstract.  Blue: recomputed from the printed 95% CI "
        "(SE = width/3.92).", fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")

# ---- fig2: dot-and-interval, abstract 'preterm' values vs Table preterm cells
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.20, right=0.68, top=0.86, bottom=0.14)
names = [a[0] for a in ABSTRACT]
src = {
    "Internalizing": "= <37 wk, childhood",
    "Externalizing": "= <37 wk, early adol.",
    "ADHD": "= <2500 g row",
    "ASD": "= <2500 g row",
    "Non-verbal IQ": "= <2500 g row",
}
ypos = np.arange(len(names))[::-1] * 1.0
off = 0.17
for y, (name, e, l, h) in zip(ypos, ABSTRACT):
    tab = "T3" if name in ("Internalizing", "Externalizing") else "T4"
    pe, pl, ph, pn = T[(tab, "<37 wk", name, "childhood")]
    p_pre = REC[(tab, "<37 wk", name, "childhood")][2]
    ax.plot([l, h], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
    ax.plot(e, y + off, "o", color=ORANGE, ms=8, mec=BG, mew=1.5)
    ax.plot([pl, ph], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
    ax.plot(pe, y - off, "s", color=BLUE, ms=8, mec=BG, mew=1.5)
    ptxt = "p < 0.001" if p_pre < 0.001 else f"p = {p_pre:.3f}" if p_pre < 0.01 else f"p = {p_pre:.2f}"
    ax.annotate(f"{e:+.2f}  {src[name]}".replace("-", "\u2212", 1), xy=(1.02, y + off), xycoords=("axes fraction", "data"),
                fontsize=10, va="center", color=INK, annotation_clip=False)
    ax.annotate(f"{pe:+.2f}  {ptxt}".replace("-", "\u2212", 1), xy=(1.02, y - off), xycoords=("axes fraction", "data"),
                fontsize=10, va="center", color=INK2, annotation_clip=False)
ax.axvline(0, color=INK2, lw=0.8, zorder=0)
ax.set_yticks(ypos)
ax.set_yticklabels(names)
ax.set_ylim(ypos.min() - 0.6, ypos.max() + 0.6)
ax.set_xlim(-8.5, 7)
ax.set_xticks(np.arange(-8, 7, 2))
ax.set_xlabel("Difference vs term birth, outcome percentiles (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#c9c7c1")
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
              label="Abstract, stated as preterm birth (reported)")
h2, = ax.plot([], [], "s-", color=BLUE, ms=8, lw=2,
              label="Table 3/4 preterm <37 wk, childhood; p recomputed from CI")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.01, 0.995), frameon=False,
           fontsize=10, ncol=1)
ax.annotate("source of abstract value\n/ preterm cell p", xy=(1.02, ypos.max() + 0.55),
            xycoords=("axes fraction", "data"), fontsize=10, color=INK2, va="bottom",
            annotation_clip=False)
save(fig, "fig2")

Reported overall 90-day mortality of 63.7% conflicts with the paper's own counts of 408 deaths

Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101317 · PMC12266187

Summary

The abstract and the Results give the overall 90-day mortality of the 1109 included patients as "63.7% (n = 707)". Every breakdown of those same 1109 patients in the paper adds up to 408 deaths (36.8%) and 701 survivors (63.2%). The breakdowns are the three PROMISE risk groups, the two GRIm risk groups, and the PROMISE and GRIm death counts in the four cancer-type subgroups. The reported median overall survival of 4.78 months is longer than 90 days, which on its own requires 90-day mortality below 50%. The printed figure therefore overstates the cohort's primary-outcome event rate by 299 deaths, about 27 percentage points. It is also used in the Discussion to argue that mortality was higher than in the cohort in which PROMISE was developed.

What the paper reports

The headline figure, in the abstract:

PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was assessed in 333 patients with complete laboratory data. The 90-day mortality rate was 63.7% (n = 707).

— Abstract, Findings

The same figure opens the outcome paragraph of the Results, followed directly by the stratified counts:

The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients (97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was 18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 months (95% CI: 4.14–5.12).

— Results, overall outcome paragraph

The risk-group sizes:

According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients (19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified 511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk.

— Results, patient characteristics paragraph (Table 1 column headers give the GRIm groups as "Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)")

The cancer-type subgroups (PROMISE and GRIm sentences only; the CTI sentences refer to the 333-patient subset):

For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% (n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) in the high-risk group (p < 0.0001). [...] The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) in the high-risk group (p < 0.0001).

For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% (n = 120) in the high-risk group (p < 0.0001). [...] The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to 60.6% (n = 123) in the high-risk group (p < 0.0001).

For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% (n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) in the high-risk group (p < 0.0001). [...] The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to 54.2% (n = 26) in the high-risk group (p < 0.0001).

For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk group, and 60.8% (n = 121) in the high-risk group (p < 0.0001). [...] The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% (n = 107) in the high-risk group (p < 0.0001).

— Results, section "Subgroup analysis by cancer type" ([...] marks omitted median-OS sentences)

The separately reported subset of 333 patients with a CTI score:

In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was 35.1% (n = 117).

— Results, CTI subset paragraph

The Discussion:

According to the results of our study, the 90-day mortality rate was higher compared to the study by Mirallas et al., in which the PROMISE score was developed.

In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups were reported as 83.6%, 68.0%, and 33.4%, respectively.

— Discussion

Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC12266187). Each one occurs exactly once (passages joined by [...] were checked sentence by sentence), and recompute.py.txt reruns that check.

What the paper's own numbers show

Method. The paper states that PROMISE and GRIm were calculated for all 1109 patients, and the PROMISE group sizes (278 + 215 + 616) and the cancer-type subgroup sizes (283 + 350 + 92 + 384) each sum to 1109. Each of these breakdowns is therefore exhaustive, and summing its death counts gives the cohort's total number of 90-day deaths. For PROMISE the paper gives deaths and survivors per group. For GRIm it gives survivors per group, so deaths are group size minus survivors, using either the text's or Table 1's group sizes. For the cancer types it gives deaths per risk group under each score. As an independent check, the Kaplan–Meier median OS is compared with the 90-day time point. The 333-patient CTI subset is recomputed the same way. No number was estimated; every input is a count or percentage printed in the text or Table 1. recompute.py.txt prints every value below and redraws both figures.

Quantity Location Reported Recomputed from the paper's own numbers
Overall 90-day mortality, 1109 patients Abstract, Findings; Results, outcome paragraph 63.7% (n = 707) 408 deaths, 36.8% (every row below). Arithmetic check: 707/1109 = 63.75%
PROMISE deaths, low + intermediate + high Results, outcome paragraph 7, 39, 362 (2.5%, 18.1%, 58.8% of 278, 215, 616) 7 + 39 + 362 = 408. Group percentages reproduce exactly
PROMISE survivors Results, outcome paragraph 271, 176, 254 271 + 176 + 254 = 701; 408 + 701 = 1109
GRIm deaths (group size − survivors) Results, outcome paragraph; Table 1 survivors 449 (87.8%), 252 (42.2%); n 510/599 (Table 1) or 511/598 (text) (510 − 449) + (599 − 252) = 61 + 347 = 408; with text n, 62 + 346 = 408. Survivors 449 + 252 = 701
Deaths by cancer type, PROMISE Results, "Subgroup analysis by cancer type" lung 1+2+96, GI 5+18+120, breast 1+3+25, other 0+16+121 99 + 143 + 29 + 137 = 408 (n = 1109)
Deaths by cancer type, GRIm Results, "Subgroup analysis by cancer type" lung 9+90, GI 20+123, breast 3+26, other 30+107 99 + 143 + 29 + 137 = 408; per cancer type identical to PROMISE
Median OS vs 90-day mortality Results, outcome paragraph median OS 4.78 months ("According to the PROMISE score"); 4.63 months (stated after the GRIm results) Both exceed 90 days (2.96 months), so Kaplan–Meier survival at 90 days is above 50% and 90-day mortality is below 50%. 707 deaths within 90 days would force Kaplan–Meier mortality at 90 days to at least 63.8% and the median below 90 days
CTI subset, 333 patients Results, CTI subset paragraph 35.1% (n = 117) 117/333 = 35.1%. Also (166 − 147) + (167 − 69) = 117; PROMISE within subset 3 + 12 + 102 = 117; Combined score 0 + 1 + 116 = 117; CTI by cancer type 21 + 40 + 20 + 36 = 117. Consistent
Gap between stated and recomputed — 707 (63.7%) 707 − 408 = 299 deaths; 63.75 − 36.79 = 27.0 percentage points

Four exhaustive breakdowns, drawn from two scores and two different groupings of the same patients, agree on 408 deaths. They also agree on 701 survivors where survivors are given. The subset of 333 patients, reported the same way, is fully consistent, so the reporting of the stratified counts can be reproduced throughout. Only the single overall figure departs from them. Its percentage, 63.7%, is close to the share who survived (701/1109 = 63.2%), not the share who died.

Figure

Death counts: stated 707 versus 408 from each of the paper's four exhaustive breakdowns

Figure 1. Number of the 1109 included patients who died within 90 days. The orange bar is the overall figure printed in the abstract and Results (707, "63.7%"). Each blue bar sums the death counts the paper itself reports for one exhaustive breakdown of the same 1109 patients: the three PROMISE risk groups, the two GRIm risk groups (group size from Table 1 minus reported survivors), and the PROMISE and GRIm deaths in the four cancer-type subgroups. Segments are drawn in the order the formula on each row lists them; unlabeled thin segments are 7, 39 and 29. The dashed line marks the cohort size. All four breakdowns give 408 deaths (36.8%). Values come from the article text and Table 1 and are recomputed by recompute.py.txt.

Percentages: printed mortality, recomputed mortality and survival, CTI subset, and the 50% bound implied by median OS

Figure 2. The same comparison as percentages on one 0–100% axis. The printed overall mortality (orange, 63.7%) lies in the shaded region that the paper's own median OS of 4.78 months rules out. A Kaplan–Meier median longer than 90 days means fewer than half the patients had died by day 90. The mortality recomputed from the strata (36.8%) lies outside it. The printed value sits next to the recomputed survival share (blue diamond, 63.2%). The bottom row shows the separately reported CTI subset, where the reported value (orange ring) and the recomputed value (blue dot) coincide at 35.1%.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract (Findings) and in the first sentence of the Results outcome paragraph, replace "63.7% (n = 707)" with the figure supported by the stratified data, which on the paper's counts is 36.8% (n = 408), after confirming it against the source dataset. In the Discussion, revisit the statement that 90-day mortality was higher than in the PROMISE development study, and the explanation offered for it, in light of the corrected rate. Two further points could be checked at the same time. One is the GRIm group sizes (511/598 in the text vs 510/599 in Table 1). The other is the predictive values in Table 2, which do not correspond to the 35.1% event rate of the 333-patient subset; giving the underlying 2×2 counts would let readers verify them.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12266187 (Lancet Reg Health Eur 2025, doi:10.1016/j.lanepe.2025.101317)
# Overall 90-day mortality "63.7% (n = 707)" versus the paper's own stratified counts.
#
# Run:  python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# Inputs: every count and percentage below is copied from the article text or Table 1
# as archived in PubMed Central (PMC12266187). When run inside the /home/ubuntu/papers
# repo, the script re-checks each quoted passage against the archived full text with
# src/show.py (skipped elsewhere). Nothing is estimated or read off an image.

import os, subprocess

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ quotes
QUOTES = [
    ("Abstract, Findings",
     "PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was "
     "assessed in 333 patients with complete laboratory data. The 90-day mortality rate was "
     "63.7% (n = 707)."),
    ("Results, overall outcome paragraph",
     "The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients "
     "(97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the "
     "intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was "
     "18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day "
     "mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median "
     "overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm "
     "score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) "
     "for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 "
     "months (95% CI: 4.14–5.12)."),
    ("Results, patient characteristics paragraph",
     "According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients "
     "(19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified "
     "511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk."),
    ("Table 1, column headers",
     "Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)"),
    ("Results, Subgroup analysis by cancer type (lung)",
     "For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% "
     "(n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) "
     "in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (lung)",
     "The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) "
     "in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (gastrointestinal)",
     "For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates "
     "of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% "
     "(n = 120) in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (gastrointestinal)",
     "The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to "
     "60.6% (n = 123) in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (breast)",
     "For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% "
     "(n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) "
     "in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (breast)",
     "The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to "
     "54.2% (n = 26) in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (other cancers)",
     "For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality "
     "in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk "
     "group, and 60.8% (n = 121) in the high-risk group (p < 0.0001)."),
    ("Results, Subgroup analysis by cancer type (other cancers)",
     "The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% "
     "(n = 107) in the high-risk group (p < 0.0001)."),
    ("Results, CTI subset paragraph",
     "In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was "
     "35.1% (n = 117)."),
    ("Results, overall outcome paragraph (CTI)",
     "Similarly, in CTI score assessments, low-risk patients demonstrated an 88.6% survival rate "
     "(n = 147), while high-risk patients had a 41.3% survival rate (n = 69) (p < 0.0001)."),
    ("Results, patient characteristics paragraph (CTI)",
     "Among the 333 patients evaluated using the CTI score, 166 patients (49.8%) were categorized as "
     "low risk and 167 patients (50.2%) as high risk."),
    ("Discussion",
     "According to the results of our study, the 90-day mortality rate was higher compared to the "
     "study by Mirallas et al., in which the PROMISE score was developed."),
    ("Discussion",
     "In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups "
     "were reported as 83.6%, 68.0%, and 33.4%, respectively."),
]


def check_quotes():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.\n")
        return
    try:
        full = subprocess.run([py, "src/show.py", "PMC12266187", "--full"], cwd=repo,
                              capture_output=True, text=True, timeout=300).stdout
    except Exception as e:
        print(f"Quote check: could not load archived text ({e}); skipped.\n")
        return
    norm = lambda s: " ".join(s.split())
    full_n = norm(full)
    print("Quote check against the PMC-archived full text (occurrences):")
    for loc, q in QUOTES:
        n = full_n.count(norm(q))
        print(f"  {n}  {loc}: {q[:70]}...")
    print()


# ------------------------------------------------------------------ data (from the paper)
N = 1109                       # included patients (Abstract; Table 1)
STATED_DEATHS, STATED_PCT = 707, 63.7

# PROMISE strata: (group size, survivors, deaths, reported mortality %)
PROMISE = {"low": (278, 271, 7, 2.5), "intermediate": (215, 176, 39, 18.1),
           "high": (616, 254, 362, 58.8)}
# GRIm strata: survivors and reported survival %; group sizes as given in text and in Table 1
GRIM_SURV = {"low": (449, 87.8), "high": (252, 42.2)}
GRIM_N_TEXT = {"low": 511, "high": 598}
GRIM_N_TABLE1 = {"low": 510, "high": 599}
# Cancer-type subgroups: n, PROMISE deaths (low, int, high), GRIm deaths (low, high),
# CTI subgroup n, CTI deaths (low, high)
CANCER = {
    "Lung":             (283, (1, 2, 96),   (9, 90),   62,  (3, 18)),
    "Gastrointestinal": (350, (5, 18, 120), (20, 123), 95,  (8, 32)),
    "Breast":           (92,  (1, 3, 25),   (3, 26),   54,  (4, 16)),
    "Other":            (384, (0, 16, 121), (30, 107), 122, (4, 32)),
}
# CTI subset (333 patients)
CTI_N, CTI_DEATHS, CTI_PCT = 333, 117, 35.1
CTI_GROUPS = {"low": (166, 147), "high": (167, 69)}          # (n, survivors)
CTI_PROMISE = {"low": (114, 111, 3), "intermediate": (64, 52, 12), "high": (155, 53, 102)}
COMBINED_DEATHS = (0, 1, 116)
# Median OS for the full cohort (months)
MEDIAN_OS = {"PROMISE analysis": 4.78, "GRIm analysis": 4.63}
# Table 2 (n = 333): sensitivity, specificity, PPV, NPV, accuracy (%)
TABLE2 = {"PROMISE Score": (88.7, 64.3, 78.0, 83.0, 79.9),
          "CTI Score": (88.2, 67.5, 79.1, 81.4, 81.0),
          "PROMISE-CTI Combined score": (92.4, 81.1, 85.3, 89.6, 86.7)}
# Development study survival by PROMISE stratum, as quoted in the Discussion
MIRALLAS_SURV = (83.6, 68.0, 33.4)


def pct(a, b):
    return 100.0 * a / b


def main():
    check_quotes()

    print("1. Stated overall figure")
    print(f"   707/1109 = {pct(707, N):.2f}%  (reported 63.7%)")

    print("\n2. PROMISE strata (cover all 1109 patients)")
    nsum = sum(v[0] for v in PROMISE.values())
    for k, (n, s, d, rp) in PROMISE.items():
        print(f"   {k:12s} n={n}: survivors+deaths={s + d}; mortality {pct(d, n):.1f}% "
              f"(reported {rp}%); survival {pct(s, n):.1f}%")
    P_D = sum(v[2] for v in PROMISE.values())
    P_S = sum(v[1] for v in PROMISE.values())
    print(f"   group sizes sum to {nsum}")
    print(f"   deaths 7+39+362 = {P_D}; survivors 271+176+254 = {P_S}; total {P_D + P_S}")
    print(f"   mortality {P_D}/{N} = {pct(P_D, N):.2f}%; survival {P_S}/{N} = {pct(P_S, N):.2f}%")

    print("\n3. GRIm strata (deaths = group size - survivors)")
    G_S = sum(v[0] for v in GRIM_SURV.values())
    for label, ns in (("text", GRIM_N_TEXT), ("Table 1", GRIM_N_TABLE1)):
        dl, dh = ns["low"] - GRIM_SURV["low"][0], ns["high"] - GRIM_SURV["high"][0]
        print(f"   n from {label} {ns['low']}/{ns['high']}: survival "
              f"{pct(449, ns['low']):.1f}%/{pct(252, ns['high']):.1f}% (reported 87.8%/42.2%); "
              f"deaths {dl}+{dh} = {dl + dh}")
    print(f"   survivors 449+252 = {G_S}; deaths 1109-{G_S} = {N - G_S}")
    GT_D = (GRIM_N_TABLE1["low"] - 449, GRIM_N_TABLE1["high"] - 252)

    print("\n4. Cancer-type subgroups")
    print(f"   n: 283+350+92+384 = {sum(v[0] for v in CANCER.values())}")
    for k, (n, pd, gd, cn, cd) in CANCER.items():
        print(f"   {k:16s} n={n}: PROMISE deaths {'+'.join(map(str, pd))} = {sum(pd)}; "
              f"GRIm deaths {'+'.join(map(str, gd))} = {sum(gd)}; CTI deaths "
              f"{'+'.join(map(str, cd))} = {sum(cd)} of {cn}")
    CP = [sum(v[1]) for v in CANCER.values()]
    CG = [sum(v[2]) for v in CANCER.values()]
    print(f"   PROMISE total {'+'.join(map(str, CP))} = {sum(CP)}; "
          f"GRIm total {'+'.join(map(str, CG))} = {sum(CG)}")
    print(f"   CTI subgroup n sum = {sum(v[3] for v in CANCER.values())}; CTI deaths sum = "
          f"{sum(sum(v[4]) for v in CANCER.values())}")

    print("\n5. CTI subset (333 patients) - internally consistent")
    cd = [n - s for n, s in CTI_GROUPS.values()]
    print(f"   117/333 = {pct(117, 333):.2f}% (reported 35.1%)")
    print(f"   CTI groups: (166-147)+(167-69) = {cd[0]}+{cd[1]} = {sum(cd)}")
    print(f"   PROMISE within subset: 3+12+102 = {sum(v[2] for v in CTI_PROMISE.values())}; "
          f"n 114+64+155 = {sum(v[0] for v in CTI_PROMISE.values())}")
    print(f"   Combined score: 0+1+116 = {sum(COMBINED_DEATHS)}")

    print("\n6. Median OS versus 90-day mortality")
    m90 = 90 / 30.4375
    for k, v in MEDIAN_OS.items():
        print(f"   {k}: median OS {v} months > 90 days ({m90:.2f} months): {v > m90}")
    print("   A Kaplan-Meier median above 90 days means S(90 d) > 0.5, i.e. 90-day mortality < 50%.")
    print("   The KM estimate of mortality by day 90 is never below (deaths observed by day 90)/N,")
    print(f"   so 707 deaths would force KM mortality >= {pct(707, N):.1f}% and a median <= 90 days.")

    print("\n7. Gap between stated and recomputed")
    print(f"   707 - 408 = {707 - 408} deaths; {pct(707, N):.2f} - {pct(408, N):.2f} = "
          f"{pct(707, N) - pct(408, N):.1f} percentage points")
    print(f"   survivors 701 ({pct(701, N):.1f}%) vs stated 707 ({pct(707, N):.1f}%): differ by 6")
    print(f"   1109 - 707 = {N - 707}")

    print("\n8. Discussion comparison with the development study (stratum survival, %)")
    here = [round(pct(v[1], v[0]), 1) for v in PROMISE.values()]
    for g, a, b in zip(("low", "intermediate", "high"), here, MIRALLAS_SURV):
        print(f"   {g:12s}: this cohort {a}% vs development study {b}%")

    print("\n9. Table 2 (n = 333): prevalence implied by each reported value")
    p = CTI_DEATHS / CTI_N
    for k, (se, sp, ppv, npv, acc) in TABLE2.items():
        se, sp, ppv, npv, acc = se / 100, sp / 100, ppv / 100, npv / 100, acc / 100
        p_ppv = (1 - sp) * ppv / (se * (1 - ppv) + (1 - sp) * ppv)
        p_npv = (1 - npv) * sp / ((1 - npv) * sp + npv * (1 - se))
        p_acc = (acc - sp) / (se - sp)
        e_ppv = se * p / (se * p + (1 - sp) * (1 - p))
        e_npv = sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)
        e_acc = se * p + sp * (1 - p)
        print(f"   {k}: implied prevalence from PPV {100*p_ppv:.1f}%, NPV {100*p_npv:.1f}%, "
              f"accuracy {100*p_acc:.1f}%")
        print(f"      expected at 35.1% prevalence: PPV {100*e_ppv:.1f}, NPV {100*e_npv:.1f}, "
              f"accuracy {100*e_acc:.1f} (reported {ppv*100:.1f}, {npv*100:.1f}, {acc*100:.1f})")

    make_figures(P_D, GT_D, CP, CG)


# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
                     "axes.edgecolor": "#b9b8b3", "svg.fonttype": "none"})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
    plt.close(fig)
    print(f"   wrote {name}.svg, {name}.png")


def make_figures(P_D, GT_D, CP, CG):
    print("\nFigures")
    # ---- fig1: death counts, stated vs sums of the paper's breakdowns
    rows = [
        ("Stated overall deaths\n(Abstract; Results)", [STATED_DEATHS], ORANGE, ["707"]),
        ("PROMISE risk groups\n7 + 39 + 362 (low, int., high)",
         [v[2] for v in PROMISE.values()], BLUE, ["7", "39", "362"]),
        ("GRIm risk groups, n − survivors\n(510 − 449) + (599 − 252)", list(GT_D), BLUE,
         [str(x) for x in GT_D]),
        ("Cancer types, PROMISE deaths\n99 + 143 + 29 + 137", CP, BLUE,
         [str(x) for x in CP]),
        ("Cancer types, GRIm deaths\n99 + 143 + 29 + 137", CG, BLUE,
         [str(x) for x in CG]),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
    ax.set_facecolor(BG)
    shades = ["#2a78d6", "#5d9ae3", "#8fbbee", "#bcd6f5"]
    for i, (lab, segs, col, txt) in enumerate(rows):
        y = len(rows) - 1 - i
        left = 0
        for j, (s, t) in enumerate(zip(segs, txt)):
            c = col if col == ORANGE else shades[j % 4]
            ax.barh(y, s, left=left, height=0.56, color=c, edgecolor=BG, linewidth=2)
            if s >= 60:
                ax.text(left + s / 2, y, t, ha="center", va="center", fontsize=10,
                        color="white" if (col == ORANGE or j == 0) else INK)
            left += s
        tot = sum(segs)
        ax.text(tot + 12, y, f"= {tot}  ({pct(tot, N):.1f}% of {N})" if len(segs) > 1
                else f"{tot}  ({STATED_PCT}%)", va="center", fontsize=10.5,
                color=INK, fontweight="bold")
    ax.axvline(N, color=INK2, lw=1, ls=(0, (3, 3)))
    ax.text(N, len(rows) - 0.35, f"all {N} patients", ha="right", va="bottom",
            fontsize=10, color=INK2)
    ax.set_yticks(range(len(rows)))
    ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
    ax.set_xlim(0, 1180)
    ax.set_ylim(-0.6, len(rows) - 0.1)
    ax.set_xlabel("Patients who died within 90 days (count)")
    ax.xaxis.grid(True, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    from matplotlib.patches import Patch
    ax.legend(handles=[Patch(color=ORANGE, label="Reported in the paper"),
                       Patch(color=BLUE, label="Recomputed: sum of the paper's own subgroup counts")],
              loc="upper center", bbox_to_anchor=(0.38, -0.2), ncol=2, frameon=False, fontsize=10)
    save(fig, "fig1")

    # ---- fig2: percentages on one 0-100 axis with the median-OS bound
    fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
    ax.set_facecolor(BG)
    ax.axvspan(50, 100, color="#efeeea", zorder=0)
    ax.axvline(50, color=INK2, lw=1, zorder=1)
    ax.text(51.5, 0.25, "Ruled out by median OS 4.78 months (> 90 days):\n"
            "90-day mortality must be below 50%", fontsize=10, color=INK2, va="center")
    pts = [
        (3, "Overall 90-day mortality,\nas printed (n = 707)", [(STATED_PCT, ORANGE, "o", "63.7%")]),
        (2, "Overall 90-day mortality from\nthe paper's strata (408/1109)",
         [(pct(408, N), BLUE, "o", f"{pct(408, N):.1f}%")]),
        (1, "Share surviving 90 days from\nthe paper's strata (701/1109)",
         [(pct(701, N), BLUE, "D", f"{pct(701, N):.1f}%")]),
        (0, "CTI subset 90-day mortality\n(117/333): reported and recomputed",
         [(CTI_PCT, ORANGE, "o", None), (pct(117, 333), BLUE, "o", f"{CTI_PCT}% (both)")]),
    ]
    for y, lab, marks in pts:
        for k, (x, c, m, t) in enumerate(marks):
            ms = 14 if k == 0 else 8
            ax.plot([x], [y], marker=m, ms=ms, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
            if t:
                ax.text(x - 2.5, y + 0.02, t, ha="right", va="center", fontsize=10.5,
                        color=INK, fontweight="bold")
    ax.set_yticks([p[0] for p in pts])
    ax.set_yticklabels([p[1] for p in pts], fontsize=10)
    ax.set_xlim(0, 100)
    ax.set_ylim(-0.6, 3.7)
    ax.set_xlabel("Percent of patients")
    ax.xaxis.grid(True, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    from matplotlib.lines import Line2D
    ax.legend(handles=[Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE,
                              label="Reported in the paper"),
                       Line2D([], [], marker="o", ls="none", ms=9, color=BLUE,
                              label="Recomputed from the paper's counts")],
              loc="upper center", bbox_to_anchor=(0.3, -0.25), ncol=2, frameon=False, fontsize=10)
    save(fig, "fig2")


if __name__ == "__main__":
    main()

Abstract reports univariate hazard ratios as adjusted estimates and attaches them to the wrong treatments

Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101402 · PMC12859599

Summary

The Abstract states that "in the adjusted multivariate model" immunotherapy and targeted therapy had hazard ratios (HRs) for second primary cancer (SPC), vs chemotherapy, of 0.470 (95% CI 0.338–0.654) and 0.701 (0.574–0.855). In the Results these same two HR/CI pairs are reported as the univariate Cox estimates, with the treatments the other way round: 0.470 for targeted therapy and 0.701 for immunotherapy. The adjusted multivariate estimates in the Results, repeated in the Discussion, are 0.768 (0.622–0.947) for immunotherapy and 0.556 (0.385–0.803) for targeted therapy. The paper's own event counts and incidence rates support the Results' assignment: the width of each interval matches the number of events in the group the Results name, and the crude rate ratios rank targeted therapy as the more protective treatment. As a result, the Abstract overstates the adjusted risk reduction for immunotherapy (53% vs 23%), understates it for targeted therapy (30% vs 44%), and reverses which treatment looks more protective after adjustment. Both treatments remain associated with lower SPC risk in every version.

What the paper reports

In the adjusted multivariate model, both immunotherapy and targeted therapy were associated with a reduced risk of SPCs: HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively. Additional associated factors with SPCs included metastatic disease, HR = 0.456 (95% CI 0.377–0.552); history of tobacco use, HR = 1.599 (95% CI 1.127–2.269); and current smoking, HR = 1.319 (95% CI 0.923–1.884) p < 0.001.

Abstract, Findings

In the univariate Cox regression, the following factors were explored for their association with the development of SPCs: type of treatment received, age at diagnosis, sex, tumour stage, histological type, and smoking habits were explored. Both targeted therapy and immunotherapy, when compared to chemotherapy, were identified as protective factors with HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively.

Results, "Factors associated with second primary cancer", paragraph 1

Other factors associated with the development of SPCs included metastatic disease HR = 0.456 (95% CI 0.377–0.552), history of tobacco use HR = 1.599 (95% CI 1.127–2.269), and active tobacco use HR = 1.319 (95% CI 0.923–1.884); p < 0.001.

Results, "Factors associated with second primary cancer", paragraph 3 (between the univariate paragraph and the adjusted-model paragraph)

In the adjusted multivariate model, the effect of treatment was confirmed with immunotherapy and targeted therapy, HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively. Other associated factors such as age HR = 1.028 (95% CI 1.018–1.039), small cell histology HR = 0.704 (95% CI 0.513–0.965), and metastatic disease HR = 0.518 (95% CI 0.428–0.626); p < 0.001, were also found.

Results, "Factors associated with second primary cancer", paragraph 4

Multivariate Cox regression analysis confirmed that both immunotherapy and targeted therapy were protective for SPC compared to chemotherapy with HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively.

Discussion, paragraph 1

The inputs used for the checks below:

A total of 480 patients (2.3%) developed a SPC. The incidence rate per 100 person-years were 1.059 for the immunotherapy group, 0.661 for the targeted therapy group, and 1.791 for the chemotherapy group. […] By treatment group, 298 patients (62.1%) had received chemotherapy, 142 (29.6%) immunotherapy, and 40 (8.3%) targeted therapy. The proportion of patients who developed an SPC was lowest in the targeted therapy group (1.5%) followed by the immunotherapy group (2.1%) and the chemotherapy group (2.9%) (Table 3).

Results, "Characteristics of the population with a second primary cancer by treatment type", paragraph 1

How the Abstract uses the result:

Patients treated with immunotherapy and targeted therapy exhibited a significantly lower incidence of SPCs. Tobacco use was a key associated factor.

Abstract, Interpretation

What the paper's own numbers show

Method. The Abstract, Results and Discussion values were compared as printed. Two checks show which treatment each HR/CI pair belongs to. (1) The standard error of a log HR implied by a printed 95% CI is ln(upper/lower)/(2 × 1.96). For a Cox HR comparing two groups, this SE is approximately √(1/d₁ + 1/d₀), where d₁ and d₀ are the event counts in the two groups (142 immunotherapy, 40 targeted therapy and 298 chemotherapy events; Results and Table 3). The approximation is close for an unadjusted comparison and somewhat larger after adjustment. (2) The crude incidence-rate ratio of each group vs chemotherapy was computed from the reported rates per 100 person-years. Each CI was also checked for internal consistency: its log-scale midpoint should equal the printed HR. All numbers come from recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
Immunotherapy vs chemotherapy, "adjusted multivariate" Abstract, Findings 0.470 (0.338–0.654) Results and Discussion give the adjusted HR as 0.768 (0.622–0.947). 0.470 (0.338–0.654) is the Results' univariate value for targeted therapy
Targeted therapy vs chemotherapy, "adjusted multivariate" Abstract, Findings 0.701 (0.574–0.855) Results and Discussion give the adjusted HR as 0.556 (0.385–0.803). 0.701 (0.574–0.855) is the Results' univariate value for immunotherapy
SE(log HR) implied by 0.470 (0.338–0.654) Abstract; Results ¶1 — 0.1684. Expected from events: targeted therapy (40 vs 298) 0.1684, immunotherapy (142 vs 298) 0.1020. Ratio implied/expected 1.000 for targeted therapy, 1.651 for immunotherapy
SE(log HR) implied by 0.701 (0.574–0.855) Abstract; Results ¶1 — 0.1017. Matches immunotherapy (ratio 0.997), not targeted therapy (0.604)
SE(log HR) implied by adjusted 0.768 (0.622–0.947) Results ¶4; Discussion immunotherapy 0.1072, 1.052 × the immunotherapy event-based value. Consistent
SE(log HR) implied by adjusted 0.556 (0.385–0.803) Results ¶4; Discussion targeted therapy 0.1875, 1.114 × the targeted-therapy event-based value. Consistent
Log-scale midpoint of each CI all four pairs 0.470, 0.701, 0.768, 0.556 0.470, 0.701, 0.767, 0.556. Each pair is internally consistent, so no digit has been garbled
Crude SPC rate ratio vs chemotherapy from rates per 100 person-years (1.059, 0.661, 1.791) — Immunotherapy 0.591 (0.484–0.722), targeted therapy 0.369 (0.265–0.513). Targeted therapy is the more protective group, as in the Results' univariate assignment
12-month cumulative incidence ratio vs chemotherapy Abstract; Results (1.22%, 0.85%, 0.67%) — Immunotherapy 0.697, targeted therapy 0.549. Same ordering
Hazard reduction, immunotherapy (100 × (1 − HR)) Abstract vs Results ¶4 53.0% (abstract) 23.2% (adjusted model). The abstract's figure is 2.28 times the adjusted one
Hazard reduction, targeted therapy Abstract vs Results ¶4 29.9% (abstract) 44.4% (adjusted model)
Which treatment is more protective after adjustment Abstract vs Results ¶4 immunotherapy (0.470 < 0.701) targeted therapy (0.556 < 0.768)
Metastatic disease, listed in the abstract after the "adjusted" treatment HRs Abstract vs Results ¶3, ¶4 0.456 (0.377–0.552) 0.456 is the Results ¶3 value (between the univariate and adjusted paragraphs). The adjusted-model value is 0.518 (0.428–0.626)
Current smoking, "p < 0.001" Abstract; Results ¶3 1.319 (0.923–1.884) The CI includes 1. The Wald p implied by the CI is 0.13 (z = 1.52), so the "p < 0.001" cannot refer to this term on its own
Wald p implied by the adjusted CIs Results ¶4; Discussion ("p < 0.001") — Immunotherapy z = −2.46, p = 0.014; targeted therapy z = −3.13, p = 0.0017. These are significant at 0.05, but neither is below 0.001; the printed p may refer to a joint test of the treatment variable
Crude SPC proportions Abstract; Results; Table 3 2.9%, 2.1%, 1.5% 298/10,137 = 2.94%, 142/6,820 = 2.08%, 40/2,628 = 1.52% with the followed-up denominators of Supplementary Figure S1 (sum 19,585). These are consistent and not part of the error

So the Abstract's two "adjusted" treatment HRs are the univariate pair, with the treatment labels swapped. The event-count check fits the Results' labelling exactly (ratios 0.997 and 1.000) and misfits the Abstract's labelling by factors of 1.65 and 0.60. The crude rate ratios point the same way. The adjusted estimates in the body are consistent with the event counts and with the expected modest SE inflation after covariate adjustment.

Figure

Abstract vs Results: the same HR/CI pairs appear as adjusted estimates in the Abstract and as univariate estimates, with treatments swapped, in the Results

Figure 1. Where each treatment HR appears in the paper and how it is labelled. Orange text gives the values as printed. The left panel is the Abstract's "adjusted multivariate model". The upper right panel is the Results' univariate Cox paragraph, which carries the same two HR/CI pairs with the treatment labels reversed. The lower right panel is the Results' adjusted multivariate paragraph, repeated in the Discussion, which carries different values under the same model label as the Abstract. The blue text below summarises the checks recomputed from the paper's own event counts and incidence rates (see recompute.py.txt). Both checks agree with the treatment labels in the Results.

Standard error implied by each printed CI vs the standard error the event counts imply

Figure 2. For each of the four printed treatment HRs, the orange diamond is the standard error of the log HR implied by its 95% CI. The blue vertical lines are the approximate standard errors expected from the event counts, √(1/d₁ + 1/d₀): 0.102 for immunotherapy vs chemotherapy (142 vs 298 events, solid) and 0.168 for targeted therapy vs chemotherapy (40 vs 298 events, dashed). The interval of HR 0.470, which the Abstract assigns to immunotherapy, sits exactly on the targeted-therapy line. The interval of HR 0.701, which the Abstract assigns to targeted therapy, sits on the immunotherapy line. The two adjusted intervals from the Results lie just to the right of their own group's line, as expected after covariate adjustment. The axis starts at zero.

Why it matters

Explanations considered

Status

Suggested correction

  1. In the Abstract, Findings, replace the treatment sentence with the adjusted estimates the Results and Discussion report: immunotherapy HR 0.768 (95% CI 0.622–0.947) and targeted therapy HR 0.556 (95% CI 0.385–0.803), vs chemotherapy. If the univariate values are kept as well, label them as univariate and attach them to the right treatments: targeted therapy 0.470 (0.338–0.654), immunotherapy 0.701 (0.574–0.855).
  2. Make the Abstract's "additional associated factors" match the model it describes. For the adjusted model, give metastatic disease 0.518 (0.428–0.626), age 1.028 (1.018–1.039) and small cell histology 0.704 (0.513–0.965). Report the adjusted tobacco HRs if the model included smoking, as the Methods list it among the covariates.
  3. Give a p value for each term, or say which test "p < 0.001" refers to. As printed, it is attached to current smoking, whose CI includes 1, and to two adjusted treatment HRs whose CIs imply p = 0.014 and p = 0.0017.
  4. Consider adding a table of the univariate and multivariate SPC models (Cox and, as the Methods describe, Fine–Gray) so that each estimate's model and covariate set is explicit. If needed, revise the Interpretation sentence on tobacco use to match the adjusted results.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12859599 write-up and regenerate fig1/fig2.

Run:  python recompute.py.txt   (needs numpy, scipy, matplotlib)
All inputs are copied from the article as archived in PubMed Central (Abstract,
Results "Characteristics of the population with a second primary cancer" and
"Factors associated with second primary cancer", Discussion) and from
Supplementary Figure S1 (CONSORT diagram, mmc2.pdf).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs ----
EVENTS = {"chemo": 298, "IT": 142, "TT": 40}            # Results; Table 3
N_TREATED = {"chemo": 10650, "IT": 7182, "TT": 2742}    # Results, treatment groups
N_FOLLOWED = {"chemo": 10137, "IT": 6820, "TT": 2628}   # Supplementary Figure S1
RATE = {"chemo": 1.791, "IT": 1.059, "TT": 0.661}       # SPC per 100 person-years
CIF12 = {"chemo": 1.22, "IT": 0.85, "TT": 0.67}         # 12-month cumulative incidence, %

# (HR, lower, upper) exactly as printed
ABSTRACT_ADJ = {"IT": (0.470, 0.338, 0.654), "TT": (0.701, 0.574, 0.855)}
RESULTS_UNIV = {"TT": (0.470, 0.338, 0.654), "IT": (0.701, 0.574, 0.855)}
RESULTS_ADJ = {"IT": (0.768, 0.622, 0.947), "TT": (0.556, 0.385, 0.803)}  # also Discussion
OTHER = {
    "metastatic, abstract / Results p2": (0.456, 0.377, 0.552),
    "metastatic, Results adjusted":      (0.518, 0.428, 0.626),
    "history of tobacco use":            (1.599, 1.127, 2.269),
    "current (active) smoking":          (1.319, 0.923, 1.884),
}
NAME = {"IT": "immunotherapy", "TT": "targeted therapy", "chemo": "chemotherapy"}


def ci_stats(h, lo, hi):
    se = np.log(hi / lo) / (2 * Z)
    mid = np.exp((np.log(lo) + np.log(hi)) / 2)
    z = np.log(h) / se
    p = 2 * stats.norm.sf(abs(z))
    return se, mid, z, p


def line(s=""):
    print(s)


# ------------------------------------------------------- 1. crude data ----
line("1. Crude SPC proportions")
line(f"   total: {sum(EVENTS.values())}/{sum(N_TREATED.values())} = "
     f"{100*sum(EVENTS.values())/sum(N_TREATED.values()):.2f}%  "
     f"(followed-up denominators sum to {sum(N_FOLLOWED.values())})")
for k in ["chemo", "IT", "TT"]:
    line(f"   {NAME[k]:17s} {EVENTS[k]}/{N_TREATED[k]} = {100*EVENTS[k]/N_TREATED[k]:.2f}%   "
         f"{EVENTS[k]}/{N_FOLLOWED[k]} (Fig S1) = {100*EVENTS[k]/N_FOLLOWED[k]:.2f}%")

line("\n2. Crude incidence-rate ratios vs chemotherapy (from the reported rates)")
PY = {k: 100 * EVENTS[k] / RATE[k] for k in EVENTS}
for k in ["chemo", "IT", "TT"]:
    line(f"   person-years implied, {NAME[k]}: {PY[k]:.0f}")
CRUDE_IRR = {}
for k in ["IT", "TT"]:
    irr = RATE[k] / RATE["chemo"]
    se = np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"])
    CRUDE_IRR[k] = (irr, irr * np.exp(-Z * se), irr * np.exp(Z * se))
    line(f"   {NAME[k]:17s} IRR = {irr:.3f} (95% CI {CRUDE_IRR[k][1]:.3f}-{CRUDE_IRR[k][2]:.3f})")
line(f"   12-month cumulative incidence ratio vs chemo: IT {CIF12['IT']/CIF12['chemo']:.3f}, "
     f"TT {CIF12['TT']/CIF12['chemo']:.3f}")

# ------------------------------------------- 3. SE expected from events ----
line("\n3. SE(log HR) expected from event counts, sqrt(1/d1 + 1/d0)")
SE_EXP = {k: np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"]) for k in ["IT", "TT"]}
for k in SE_EXP:
    line(f"   {NAME[k]:17s} ({EVENTS[k]} vs {EVENTS['chemo']} events): {SE_EXP[k]:.4f}")

# ------------------------------------------- 4. SE implied by each CI ----
line("\n4. Each reported HR: SE implied by its CI, log-midpoint, Wald z and p, nearest group")
ROWS = [
    ("0.470 (0.338-0.654)", (0.470, 0.338, 0.654), "Abstract: IT 'adjusted'  |  Results: TT univariate"),
    ("0.701 (0.574-0.855)", (0.701, 0.574, 0.855), "Abstract: TT 'adjusted'  |  Results: IT univariate"),
    ("0.768 (0.622-0.947)", (0.768, 0.622, 0.947), "Results/Discussion: IT adjusted"),
    ("0.556 (0.385-0.803)", (0.556, 0.385, 0.803), "Results/Discussion: TT adjusted"),
]
IMPLIED = {}
for lab, (h, lo, hi), where in ROWS:
    se, mid, z, p = ci_stats(h, lo, hi)
    near = min(SE_EXP, key=lambda k: abs(SE_EXP[k] - se))
    IMPLIED[lab] = se
    line(f"   {lab}: SE = {se:.4f}, log-midpoint = {mid:.3f}, z = {z:.2f}, p = {p:.2g}, "
         f"nearest event-based SE: {NAME[near]}  [{where}]")
for k in SE_EXP:
    line(f"   ratio implied/expected: univariate {NAME[k]} "
         f"{ci_stats(*RESULTS_UNIV[k])[0]/SE_EXP[k]:.3f}; adjusted {ci_stats(*RESULTS_ADJ[k])[0]/SE_EXP[k]:.3f}; "
         f"abstract '{NAME[k]}' {ci_stats(*ABSTRACT_ADJ[k])[0]/SE_EXP[k]:.3f}")

line("\n5. Other factors (Wald p from the printed CI)")
for lab, v in OTHER.items():
    se, mid, z, p = ci_stats(*v)
    line(f"   {lab:36s} HR {v[0]:.3f} ({v[1]:.3f}-{v[2]:.3f}): SE {se:.4f}, z {z:.2f}, p {p:.2g}")

# ------------------------------------------- 6. comparisons and sizes ----
line("\n6. Abstract vs Results")
line(f"   abstract 'adjusted' == Results adjusted?           {ABSTRACT_ADJ == RESULTS_ADJ}")
line(f"   abstract values == Results univariate values (set)? "
     f"{set(ABSTRACT_ADJ.values()) == set(RESULTS_UNIV.values())}")
line(f"   abstract assignment == Results univariate assignment? {ABSTRACT_ADJ == RESULTS_UNIV}")
for k in ["IT", "TT"]:
    a, r = ABSTRACT_ADJ[k][0], RESULTS_ADJ[k][0]
    line(f"   {NAME[k]:17s} hazard reduction: abstract {100*(1-a):.1f}%  vs  adjusted {100*(1-r):.1f}%  "
         f"(difference {100*abs(a-r):.1f} points; ratio of reductions {(1-a)/(1-r):.2f})")
line(f"   ranking after adjustment: abstract says {'IT' if ABSTRACT_ADJ['IT'][0] < ABSTRACT_ADJ['TT'][0] else 'TT'} "
     f"more protective; Results adjusted says {'IT' if RESULTS_ADJ['IT'][0] < RESULTS_ADJ['TT'][0] else 'TT'}")
line("   all four CIs exclude 1: "
     f"{all(v[2] < 1 for d in (ABSTRACT_ADJ, RESULTS_ADJ) for v in d.values())}")

# --------------------------------------------------------------- figures ----
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
    plt.close(fig)


# Figure 1: text-panel diagram
fig = plt.figure(figsize=(7.2, 6.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 92); ax.axis("off")


def panel(x, y, w, h, title, sub, rows):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                fc="white", ec="#bdbcb7", lw=1))
    ax.text(x + 1.5, y + h - 3, title, fontsize=11, weight="bold", va="center")
    ax.text(x + 1.2, y + h - 6.6, sub, fontsize=10, color=INK2, va="center", style="italic")
    for i, (lab, val) in enumerate(rows):
        yy = y + h - 11.5 - i * 8
        ax.text(x + 1.5, yy, lab, fontsize=10.5, va="center")
        ax.text(x + 3.5, yy - 3.5, val, fontsize=10.5, va="center", color=ORANGE, weight="bold")


PW = 37
panel(2, 41, PW, 26, "Abstract, Findings", '"adjusted multivariate model"',
      [("Immunotherapy", "HR 0.470 (0.338–0.654)"), ("Targeted therapy", "HR 0.701 (0.574–0.855)")])
panel(61, 60, PW, 26, "Results, univariate", '"univariate Cox regression"',
      [("Targeted therapy", "HR 0.470 (0.338–0.654)"), ("Immunotherapy", "HR 0.701 (0.574–0.855)")])
panel(61, 26, PW, 26, "Results ¶4; Discussion", '"adjusted multivariate model"',
      [("Immunotherapy", "HR 0.768 (0.622–0.947)"), ("Targeted therapy", "HR 0.556 (0.385–0.803)")])

for (y0, y1, txt, ty) in [(63.5, 63.5, "same numbers,\ncalled\nunivariate,\ntreatments\nswapped", 72.5),
                          (46.5, 46.5, "same model\nlabel,\ndifferent\nnumbers", 38.5)]:
    ax.add_patch(FancyArrowPatch((39.9, y0), (60.1, y1), arrowstyle="<|-|>", mutation_scale=12,
                                 lw=2, color=INK2, shrinkA=0, shrinkB=0))
    ax.text(50, ty, txt, ha="center", va="center", fontsize=10, color=INK, weight="bold",
            bbox=dict(fc=BG, ec="none", pad=1.5))

ax.text(2, 86, "Orange: values as printed in the paper.", fontsize=10, color=ORANGE, weight="bold")
ax.text(2, 82, "Blue: recomputed from the paper's own numbers.", fontsize=10, color=BLUE, weight="bold")

se470, se701 = IMPLIED["0.470 (0.338-0.654)"], IMPLIED["0.701 (0.574-0.855)"]
recomp = [
    f"The CI of HR 0.470 implies SE(log HR) = {se470:.3f}. Event counts predict {SE_EXP['TT']:.3f} for targeted therapy",
    f"(40 vs 298 events) and {SE_EXP['IT']:.3f} for immunotherapy (142 vs 298). The CI of HR 0.701 implies {se701:.3f}.",
    f"Crude SPC rate ratios vs chemotherapy: targeted therapy {CRUDE_IRR['TT'][0]:.3f}, immunotherapy {CRUDE_IRR['IT'][0]:.3f}.",
    "Both checks agree with the Results' treatment labels, not the Abstract's.",
]
for i, t in enumerate(recomp):
    ax.text(2, 18 - i * 4.2, t, fontsize=10, color=BLUE, va="center")
save(fig, "fig1")

# Figure 2: SE implied by each reported CI vs SE expected from event counts
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
YLAB = ["HR 0.470 (0.338–0.654)\nAbstract: immunotherapy, 'adjusted'\nResults: targeted, univariate",
        "HR 0.701 (0.574–0.855)\nAbstract: targeted, 'adjusted'\nResults: immunotherapy, univariate",
        "HR 0.768 (0.622–0.947)\nResults, Discussion:\nimmunotherapy, adjusted",
        "HR 0.556 (0.385–0.803)\nResults, Discussion:\ntargeted, adjusted"]
ys = np.arange(len(ROWS))[::-1]
top = len(ROWS) + 0.5
for k, ls, dx, ha in [("IT", "-", -0.004, "right"), ("TT", "--", 0.004, "left")]:
    ax.axvline(SE_EXP[k], color=BLUE, lw=2, ls=ls, zorder=1)
    ax.text(SE_EXP[k] + dx, top - 0.05,
            f"expected from events,\n{NAME[k]}\n({EVENTS[k]} vs {EVENTS['chemo']}): {SE_EXP[k]:.3f}",
            color=BLUE, fontsize=10, va="top", ha=ha)
ax.scatter([IMPLIED[r[0]] for r in ROWS], ys, marker="D", s=70, color=ORANGE, zorder=3,
           label="SE(log HR) implied by the printed 95% CI (paper's values)")
for y, r in zip(ys, ROWS):
    ax.text(IMPLIED[r[0]] + 0.006, y, f"{IMPLIED[r[0]]:.3f}", color=ORANGE, fontsize=10,
            ha="left", va="center", weight="bold")
ax.plot([], [], color=BLUE, lw=2, label="SE expected from event counts, sqrt(1/d1 + 1/d0)")
ax.set_yticks(ys); ax.set_yticklabels(YLAB, fontsize=10)
ax.set_xlim(0, 0.26); ax.set_ylim(-0.5, top)
ax.set_xlabel("Standard error of log hazard ratio vs chemotherapy")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
    ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7"); ax.tick_params(axis="y", length=0)
fig.legend(loc="lower center", ncol=1, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.1, 1, 1))
save(fig, "fig2")
line("\nFigures written: fig1.svg/png, fig2.svg/png")

Abstract reports a 59.7% catalyst heat-duty reduction; the body reports 35.2% at the optimum

Nat Commun (2026) · doi:10.1038/s41467-026-69542-6 · PMC12966427

Summary

The abstract says that the optimal 1:1 iron-sulfated zirconia/silica (Fe-SZ/SiO2) catalyst "reduces relative heat duty by up to 59.7%" in the membrane vacuum regeneration (MVR) system. The Results and Discussion report something different for the same catalyst, solvent (3 M potassium glycinate, GlyK) and blank: a 35.2% reduction at 9 wt%, the loading the paper selects as optimal, and 40.3% at 12 wt%, the highest loading tested. Applying the paper's own Eq. 7 to its reported catalytic enhancements reproduces 35.2% and 40.3%. The same calculation shows that 59.7% is the relative heat duty that remains at 12 wt% (100/1.675 = 59.7%, i.e. 100 − 40.3), not a reduction. As written, the abstract's figure is 1.70 times the reduction at the stated optimum and 1.48 times the largest catalyst-only reduction in the paper. It is the abstract's only quantitative statement about the catalyst on its own.

What the paper reports

Abstract:

An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.

Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2.

Results, Composite solid acid catalyst › Catalytic support (SiO2 vs. Al2O3), Fig. 2g text (6 wt% 1:1 catalyst):

raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK

Results, Composite solid acid catalyst › Catalyst concentration and stability, paragraph 1 (the inputs):

the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement

the enhancement rising to 67.5%

Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum.

the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations.

Same section, paragraph 2 (text for Fig. 3g):

Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2.

Same section, paragraph 3 (stability at 9 wt%, Fig. S6b):

the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2

Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f text):

Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively.

Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption.

Discussion:

9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input.

Methods, MVR performance analysis (definition):

The relative heat duty, RH (%), was calculated using Eq. 7

where Eq. 7 is RH = Q_Cat / Q_Blank × 100.

All quotes were checked verbatim against the PMC-archived text (src/show.py; recompute.py.txt repeats the check). The string "59.7" occurs once in the archived article text, in the abstract.

What the paper's own numbers show

Method. The Methods define the catalytic enhancement CE = (M_cat − M_blank)/M_blank × 100 (Eq. 4), the thermal energy Q = m·cp·ΔT / (M_total_CO2·Mw) (Eq. 6) and RH = Q_cat/Q_blank × 100 (Eq. 7). In the catalyst-loading series the solvent (3 M GlyK), its mass and the temperature programme (90 °C) are the same with and without catalyst, so m, cp and ΔT cancel and RH = M_blank/M_cat × 100 = 100/(1 + CE/100). The heat-duty reduction is 100 − RH, and the thermal energy is 7.8 GJ/tCO2 (the blank) × RH/100. RH is computed from the reported CE values and, where the paper prints the desorbed amounts, also from the amounts in mmol. As a separate check, the reduction is recomputed from the reported, rounded energies, 1 − Q_cat/7.8. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed
CE, 9 wt% 1:1 Results, Catalyst concentration, para 1 54.3% (244 → 377 mmol) 377/244 − 1 = 54.5%
CE, 6 wt% 1:1 Results, Catalytic support 25.6% (244 → 307 mmol) 307/244 − 1 = 25.8%
Gain in CE, 6 → 9 wt% and 9 → 12 wt% Results, para 1 112% and 24.3% 112.1% and 24.3%
Enhancement factor CE/W Results, para 1 6.0 at 9 wt% (highest) 4.27 (6 wt%), 6.03 (9 wt%), 5.62 (12 wt%)
RH, 9 wt% (stated optimum) Results, para 3; Fig. 5f text 64.8% 100/1.543 = 64.8% (from mmol: 64.7%)
Heat-duty reduction, 9 wt% Results, para 2; Discussion 35.2% 100 − 64.8 = 35.2% (from mmol: 35.3%; from energies: 1 − 5.1/7.8 = 34.6%)
Thermal energy, 9 wt% Results, para 2 5.1 GJ/tCO2 7.8 × 0.648 = 5.06 GJ/tCO2
RH, 12 wt% (highest tested) not stated in the text — 100/1.675 = 59.7%
Heat-duty reduction, 12 wt% Results, para 2 40.3% 100 − 59.7 = 40.3% (from energies: 1 − 4.7/7.8 = 39.7%)
Thermal energy, 12 wt% Results, para 2 4.7 GJ/tCO2 7.8 × 0.597 = 4.66 GJ/tCO2
Heat-duty reduction, 6 wt% 1:1 not stated in the text — 100 − 100/1.256 = 20.4% (RH 79.6%, 6.21 GJ/tCO2)
Heat-duty reduction, "optimal" 1:1 catalyst Abstract 59.7% ("by up to") 35.2% at the optimum; 40.3% at most (12 wt%)
What a 59.7% reduction would require — — RH 40.3%, Q = 3.14 GJ/tCO2, CE = 148% (highest reported catalyst-only CE: 67.5%)
Abstract figure ÷ body figure — — 59.7/35.2 = 1.70 (optimum); 59.7/40.3 = 1.48 (12 wt%)
Hybrid solvent + catalyst Abstract; Results (Fig. 5f text) RH 33.2%, 66.8% reduction, 2.6 GJ/tCO2 100 − 33.2 = 66.8%; 7.8 × 0.332 = 2.59 GJ/tCO2

The inputs in this section of the paper agree with one another to within rounding. The Results paragraph, the stability paragraph, the Fig. 5f text and the Discussion all agree on 35.2% (RH 64.8%) at 9 wt%, and the Results give 40.3% at 12 wt%. The abstract's catalyst figure is the only value that departs from them. It equals, to the printed decimal, the RH remaining at 12 wt%. The other two headline numbers in the abstract agree with the body: 66.8% and 2.6 GJ/tCO2 for the combined system, and the 69.1% desorption gain of the hybrid solvent (69.2% in the body; 589/348 − 1 = 69.3%).

Figure

Abstract and Results/Discussion give different heat-duty reductions for the same 1:1 catalyst; recomputation shows the abstract's 59.7 is the heat duty remaining at 12 wt%

Figure 1. Left: the abstract's sentence on the catalyst, which states 59.7% as a reduction and links it to the optimal configuration. Right: the reductions the Results (Fig. 3g text) and Discussion report for the same 1:1 Fe-SZ/SiO2 catalyst in 3 M GlyK against the same blank: 35.2% at the stated optimum (9 wt%) and 40.3% at 12 wt%. Orange text marks values reported in the paper. Bottom (blue): RH recomputed from the reported catalytic enhancements using the paper's Eq. 7, RH = 100/(1 + CE/100). The recomputation reproduces the body's reductions and shows that 59.7% is the share of the blank heat duty that remains at 12 wt%. A real 59.7% reduction would mean RH 40.3% and 3.14 GJ/tCO2. The paper reports neither value for the catalyst at any loading.

Share of the blank heat duty remaining and removed at each catalyst loading, compared with the abstract's 59.7% read as a reduction

Figure 2. Each bar is the blank heat duty (3 M GlyK, no catalyst; 100%, 7.8 GJ/tCO2), split into the share that remains (light segment, RH) and the share removed (solid segment, reduction). Blue bars are recomputed from the paper's reported catalytic enhancements (25.6%, 54.3% and 67.5%) with Eq. 7. The column at the right gives the corresponding thermal energy, 7.8 × RH/100. The orange bar shows what the abstract's sentence states: a 59.7% reduction, which would leave 40.3% and imply about 3.14 GJ/tCO2 (marked *). The arrow links the abstract's 59.7 to the remaining-share segment at 12 wt%, which has the same value. The axis runs over the full 0 to 100% range.

Why it matters

The sentence is one of three quantitative claims in the abstract and the only one that isolates the catalyst. The catalyst is one of the paper's two named contributions, "catalytic solvent regeneration" and hybrid solvents. A reader of the abstract alone, or of a database or aggregator that reproduces it, would take the catalyst to cut heat duty by about 60% at its optimal configuration. The paper's own data show about 35% at that configuration (about 5.1 GJ/tCO2 instead of 7.8) and about 40% at the highest loading. A 60% reduction would mean about 3.1 GJ/tCO2. The abstract therefore overstates the catalyst's effect by a factor of 1.5 to 1.7. It also attaches the larger number to the word "optimal", although the paper selects 9 wt% as optimal on enhancement factor, which is the loading with the smaller reduction. The direction of the effect is unchanged: the catalyst still lowers the heat duty substantially. The body, figures and conclusions are not affected. These include the combined hybrid solvent plus catalyst result (RH 33.2%, a 66.8% reduction, 2.6 GJ/tCO2), which the abstract states correctly.

Explanations considered

None of these makes the abstract's sentence consistent with the body. The arithmetic shows where the number most likely comes from. It says nothing about how the discrepancy arose.

Status

As of 2026-10-23:

Suggested correction

In the abstract, replace

An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.

with a statement that matches the Results and Discussion, for example:

An optimal iron-sulfated zirconia to silica ratio of 1:1 at 9 wt% reduces the heat duty of the membrane vacuum system by 35.2% (by up to 40.3% at 12 wt%).

To keep the relative-heat-duty form instead, the sentence could say that the catalyst "reduces relative heat duty to 64.8% at the optimal 9 wt% loading (59.7% at 12 wt%)". No change to the body, figures or conclusions appears necessary.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12966427 (Nat Commun 2026, doi:10.1038/s41467-026-69542-6)
# "Catalytic hybrid solvent regeneration in membrane vacuum processes for direct air capture"
# Abstract vs body: size of the heat-duty reduction from the 1:1 Fe-SZ/SiO2 catalyst alone.
#
# Usage:   python recompute.py.txt
# Needs:   numpy, matplotlib.
# Output:  prints every recomputed number used in writeup.md; writes fig1.svg/png and
#          fig2.svg/png next to this script. If run from the papers repository (src/show.py
#          present), it also checks each quoted statement verbatim against the PMC archive.
#
# Method. Eq. 6 of the paper: Q = m * cp * dT / (M_total_CO2 * Mw). In the catalyst-loading
# series the solvent (3 M GlyK), its mass and the temperature programme (90 C) are the same with
# and without catalyst, so Q is inversely proportional to the total CO2 desorbed in 30 min.
# Eq. 7: RH = Q_cat / Q_blank * 100, hence RH = M_blank / M_cat * 100 = 100 / (1 + CE/100),
# where CE (Eq. 4) is the catalyst enhancement. "Reduction in heat duty" = 100 - RH.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- 1. statements in the paper
QUOTES = [
    ("Abstract",
     "An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system."),
    ("Abstract",
     "Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2."),
    ("Results, Catalytic support (SiO2 vs. Al2O3), Fig. 2g text",
     "raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK"),
    ("Results, Catalyst concentration and stability, para 1",
     "the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement"),
    ("Results, Catalyst concentration and stability, para 1",
     "the enhancement rising to 67.5%"),
    ("Results, Catalyst concentration and stability, para 1",
     "Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum."),
    ("Results, Catalyst concentration and stability, para 1",
     "the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations."),
    ("Results, Catalyst concentration and stability, para 2 (Fig. 3g)",
     "Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2."),
    ("Results, Catalyst concentration and stability, para 3",
     "the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2"),
    ("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
     "Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively."),
    ("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
     "Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption."),
    ("Discussion",
     "9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input."),
    ("Methods, MVR performance analysis, Eq. 7",
     "The relative heat duty, RH (%), was calculated using Eq. 7"),
]

SHOW = os.path.join(HERE, "..", "..", "..", "src", "show.py")
if os.path.exists(SHOW):
    print("== 1. Verbatim check against the PMC archive (src/show.py --full)")
    full = subprocess.run([sys.executable, SHOW, "PMC12966427", "--full"], capture_output=True,
                          text=True, cwd=os.path.join(HERE, "..", "..", "..")).stdout
    norm = lambda s: " ".join(s.split())
    fulln = norm(full)
    for loc, q in QUOTES:
        print(f"  [{'found' if norm(q) in fulln else 'NOT FOUND'}] {loc}: {q[:70]}...")
    n597 = fulln.count("59.7")
    print(f"  occurrences of '59.7' in the archived text: {n597} (abstract only)")
else:
    print("== 1. src/show.py not available; verbatim check skipped")

# ---------------------------------------------------------------- 2. the paper's own inputs
Q_BLANK = 7.8                                  # GJ/tCO2, 3 M GlyK blank (Results, Fig. 3g text)
M_BLANK = 244                                  # mmol CO2 desorbed in 30 min, blank
M_CAT = {6: 307, 9: 377}                       # mmol, 1:1 Fe-SZ/SiO2 at 6 and 9 wt%
CE_REP = {6: 25.6, 9: 54.3, 12: 67.5}          # reported catalytic enhancement (%)
RED_REP = {9: 35.2, 12: 40.3}                  # reported heat-duty reduction (%)
Q_REP = {9: 5.1, 12: 4.7}                      # reported thermal energy (GJ/tCO2)
ABSTRACT_RED = 59.7                            # abstract: "reduces relative heat duty by up to 59.7%"

print("\n== 2. Catalytic enhancement, Eq. 4, from the desorbed CO2 amounts")
print(f"  6 wt%: (307/244 - 1) = {(M_CAT[6]/M_BLANK - 1)*100:.1f}%  (reported 25.6%)")
print(f"  9 wt%: (377/244 - 1) = {(M_CAT[9]/M_BLANK - 1)*100:.1f}%  (reported 54.3%)")
print(f"  6 -> 9 wt% relative improvement: 54.3/25.6 - 1 = {(CE_REP[9]/CE_REP[6] - 1)*100:.1f}%  (reported 112%)")
print(f"  9 -> 12 wt% relative improvement: 67.5/54.3 - 1 = {(CE_REP[12]/CE_REP[9] - 1)*100:.1f}%  (reported 24.3%)")

print("\n== 3. Relative heat duty (Eq. 7) and reduction, two routes")
ROWS = {}
for w in (6, 9, 12):
    rh_ce = 100.0 / (1 + CE_REP[w] / 100)                     # from reported CE
    rh_m = 100.0 * M_BLANK / M_CAT[w] if w in M_CAT else None  # from mmol, where printed
    q = Q_BLANK * rh_ce / 100
    ROWS[w] = dict(rh=rh_ce, red=100 - rh_ce, q=q, rh_m=rh_m)
    s_m = f"; from mmol RH = {rh_m:.1f}%, reduction {100 - rh_m:.1f}%" if rh_m else ""
    print(f"  {w:>2} wt%: from CE, RH = 100/(1+{CE_REP[w]/100:.3f}) = {rh_ce:.2f}%, reduction {100 - rh_ce:.1f}%,"
          f" Q = 7.8 x {rh_ce/100:.3f} = {q:.2f} GJ/t{s_m}")
print(f"  reported: 9 wt% reduction 35.2% (RH 64.8%, Q 5.1); 12 wt% reduction 40.3% (Q 4.7)")
print(f"  check from reported energies: 1 - 5.1/7.8 = {(1 - 5.1/7.8)*100:.1f}%;  1 - 4.7/7.8 = {(1 - 4.7/7.8)*100:.1f}%")
print(f"  100 - 35.2 = {100 - 35.2:.1f} (RH at 9 wt%, also stability cycle 1 and Fig. 5f GlyK value)")
print(f"  100 - 40.3 = {100 - 40.3:.1f} (RH at 12 wt%)  <- equals the abstract's 59.7")

print("\n== 4. What the abstract's wording would imply")
rh_abs = 100 - ABSTRACT_RED
q_abs = Q_BLANK * rh_abs / 100
ce_abs = (100 / rh_abs - 1) * 100
print(f"  'reduces RH by 59.7%' -> RH = {rh_abs:.1f}%, Q = 7.8 x {rh_abs/100:.3f} = {q_abs:.2f} GJ/tCO2,"
      f" requiring CE = {ce_abs:.0f}% (largest reported catalyst-only CE: 67.5%)")
print(f"  ratio abstract / largest body reduction (12 wt%): 59.7/40.3 = {ABSTRACT_RED/RED_REP[12]:.2f}")
print(f"  ratio abstract / optimum reduction (9 wt%):       59.7/35.2 = {ABSTRACT_RED/RED_REP[9]:.2f}")

print("\n== 5. Other headline numbers (consistency)")
print(f"  hybrid + catalyst: RH 33.2% -> reduction {100 - 33.2:.1f}% (reported 66.8%);"
      f" Q = 7.8 x 0.332 = {Q_BLANK*0.332:.2f} GJ/t (reported 2.6)")
print(f"  SarK + catalyst RH 90.1% -> reduction {100-90.1:.1f}%; TauK + catalyst RH 55.8% -> reduction {100-55.8:.1f}%"
      " (neither is a GlyK catalyst-only value, and neither equals 59.7% as a reduction)")
print(f"  6 wt% (1/2): (278/244 - 1) = {(278/244 - 1)*100:.1f}% (reported 13.7%)")
print(f"  enhancement factor CE/W (Eq. 5): 6 wt% {CE_REP[6]/6:.2f}, 9 wt% {CE_REP[9]/9:.2f} (reported 6.0),"
      f" 12 wt% {CE_REP[12]/12:.2f} -> 9 wt% is the maximum, as stated")
print(f"  stability: 68.8/64.8 - 1 = {(68.8/64.8 - 1)*100:.1f}% (reported ~6.2%); 71.8/68.8 - 1 = {(71.8/68.8 - 1)*100:.1f}% (reported ~4.3%)")
print(f"  hybrid vs TauK desorption: 589/348 - 1 = {(589/348 - 1)*100:.1f}% (abstract 69.1%, body 69.2%)")

print("\n== 6. Alternative readings of 59.7 (explanations considered)")
for lab, ce in (("6 wt% 1:1", 25.6), ("6 wt% 1:2", 13.7), ("6 wt% 2:1 (~4.0%)", 4.0)):
    print(f"  heat-duty reduction, {lab}: 100 - 100/(1+{ce/100:.3f}) = {100 - 100/(1 + ce/100):.1f}%")
print(f"  reduction relative to 6 wt% instead of blank: 9 wt% {(1 - ROWS[9]['rh']/ROWS[6]['rh'])*100:.1f}%,"
      f" 12 wt% {(1 - ROWS[12]['rh']/ROWS[6]['rh'])*100:.1f}%")
print(f"  peak CO2 flux gain vs blank (19.3e-5): 9 wt% 28.2 -> {(28.2/19.3 - 1)*100:.1f}%, 12 wt% 30.2 -> {(30.2/19.3 - 1)*100:.1f}%")
lo, hi = (1 - 3.25/7.75)*100, (1 - 3.15/7.85)*100
print(f"  hybrid solvent (3 M TauK + 1 M SarK) WITHOUT catalyst vs GlyK: 1 - 3.2/7.8 = {(1 - 3.2/7.8)*100:.1f}%"
      f" (rounding range {lo:.1f}-{hi:.1f}%) -- a solvent effect, not the catalyst")
print(f"  TauK alone vs GlyK (no catalyst): 1 - 5.5/7.8 = {(1 - 5.5/7.8)*100:.1f}%")

# ---------------------------------------------------------------- figures
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_L, BLUE_L = "#f8d3c4", "#cfe0f5"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "path",
                     "text.color": FG, "axes.labelcolor": FG, "xtick.color": FG2,
                     "ytick.color": FG, "axes.edgecolor": FG2})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
    plt.close(fig)


# --- Figure 1: abstract vs body, two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")


def box(x, y, w, h, ec, lw=2, ls="-"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                fc=BG, ec=ec, lw=lw, ls=ls))


# left: abstract
box(2.5, 30, 40, 44, ORANGE)
ax.text(4.5, 71, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(4.5, 65.5, "“An optimal iron-sulfated\nzirconia to silica ratio of\n1:1 reduces relative heat\n"
        "duty by up to 59.7% in\nthe membrane vacuum\nsystem.”", fontsize=10.5, va="top", linespacing=1.35)
ax.text(4.5, 38.5, "Reduction stated:", fontsize=10, color=FG2, va="center")
ax.text(4.5, 33.8, "59.7%  (reported)", fontsize=12, color=ORANGE, weight="bold", va="center")

# right: results + discussion
box(57.5, 30, 40, 44, FG2)
ax.text(59.5, 71, "Results (Fig. 3g text)\nand Discussion", fontsize=12, weight="bold", va="top")
ax.text(59.5, 62.5, "“at 9 wt% and 12 wt%, the heat\nduty decreased by 35.2 and\n40.3% as compared to the\n"
        "blank” (7.8 → 5.1 and 4.7 GJ/tCO2)", fontsize=10, va="top", linespacing=1.3)
ax.text(59.5, 46.5, "“9 wt% ... optimal ... achieving\na 35.2% reduction”", fontsize=10, va="top",
        linespacing=1.3)
ax.text(59.5, 38.5, "Reductions stated:", fontsize=10, color=FG2, va="center")
ax.text(59.5, 33.8, "35.2% (9 wt%), 40.3% (12 wt%)", fontsize=10.5, color=ORANGE, weight="bold", va="center")

# connector
arr = FancyArrowPatch((43.5, 52), (56.5, 52), arrowstyle="<->", mutation_scale=16, lw=2, color=FG)
ax.add_patch(arr)
ax.text(50, 55, "conflict", ha="center", va="bottom", fontsize=10, color=FG, weight="bold")
ax.text(50, 49, "59.7\nvs\n35.2 / 40.3", ha="center", va="top", fontsize=10,
        color=FG2, linespacing=1.3)

# bottom: recomputation that names the conflict
box(2.5, 3, 95, 21, BLUE)
ax.text(4.5, 21.5, "Recomputed from the paper's own Eq. 7 (RH = Q_cat / Q_blank = 1 / (1 + CE))",
        fontsize=10.5, va="top", color=BLUE, weight="bold")
r9, r12 = ROWS[9], ROWS[12]
ax.text(4.5, 16.2,
        f"9 wt%  (CE 54.3%):  RH = {r9['rh']:.1f}%   →  reduction {r9['red']:.1f}%   ({r9['q']:.2f} GJ/tCO2)\n"
        f"12 wt% (CE 67.5%):  RH = {r12['rh']:.1f}%   →  reduction {r12['red']:.1f}%   ({r12['q']:.2f} GJ/tCO2)\n"
        f"59.7% is the heat duty that remains at 12 wt%, not the reduction. A 59.7% reduction\n"
        f"would mean RH = {rh_abs:.1f}% and {q_abs:.2f} GJ/tCO2; no such value appears in the paper.",
        fontsize=10, va="top", color=FG, linespacing=1.4)
save(fig, "fig1")

# --- Figure 2: heat duty remaining vs removed, by catalyst loading, plus the abstract's reading
labels = ["Blank (3 M GlyK)", "6 wt%", "9 wt% (optimum)", "12 wt%",
          "Abstract as written\n(“by up to 59.7%”)"]
rh = [100.0, ROWS[6]["rh"], ROWS[9]["rh"], ROWS[12]["rh"], rh_abs]
q = [Q_BLANK, ROWS[6]["q"], ROWS[9]["q"], ROWS[12]["q"], q_abs]
is_abs = [False, False, False, False, True]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ys = np.arange(len(labels))[::-1].astype(float)
ys[-1] -= 0.35
for y, r, qq, ab in zip(ys, rh, q, is_abs):
    c, cl = (ORANGE, ORANGE_L) if ab else (BLUE, BLUE_L)
    ax.barh(y, r, height=0.6, color=cl, edgecolor=c, lw=1.5, zorder=2)
    if r < 100:
        ax.barh(y, 100 - r, left=r, height=0.6, color=c, edgecolor=c, lw=1.5, zorder=2)
        ax.text(r + (100 - r) / 2, y, f"{100 - r:.1f}%", ha="center", va="center", color="white",
                fontsize=10.5, weight="bold", zorder=3)
    ax.text(r / 2, y, f"{r:.1f}%", ha="center", va="center", color=FG, fontsize=10.5, zorder=3)
    ax.text(102, y, f"{qq:.2f}" if not ab else f"{qq:.2f}*", ha="left", va="center",
            color=ORANGE if ab else BLUE, fontsize=10.5)
ax.text(102, ys[0] + 0.55, "GJ/tCO2", ha="left", va="bottom", color=FG2, fontsize=10)
ax.add_patch(FancyArrowPatch((rh_abs + (100 - rh_abs) / 2, ys[4] + 0.33), (ROWS[12]["rh"] / 2, ys[3] - 0.33),
                             arrowstyle="->", mutation_scale=14, lw=1.5, color=FG, zorder=4,
                             connectionstyle="arc3,rad=-0.15"))
ax.text(73, (ys[3] + ys[4]) / 2, "same number", ha="left", va="center",
        fontsize=10, color=FG, zorder=4)
ax.set_xlim(0, 100)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.85)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Share of blank heat duty (%):  light = remaining (RH),  solid = reduction")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
for s_ in ("top", "right", "left"):
    ax.spines[s_].set_visible(False)
handles = [plt.Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE,
                         label="Recomputed from the paper's enhancements (Eq. 7)"),
           plt.Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE,
                         label="Abstract's 59.7% read as a reduction (* implied energy)")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.3, -0.16), ncol=1, frameon=False,
          fontsize=10)
fig.subplots_adjust(left=0.27, right=0.88, top=0.93, bottom=0.27)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

The abstract reports a positive temperature association with H7 spread; the paper's own coefficients are negative

Nat Commun (2026) · doi:10.1038/s41467-026-72718-9 · PMC13351079

Summary

The abstract states that "temperature and poultry density show positive association" with the spread of H7 influenza viruses. The paper's Bayesian phylogeographic GLM reports the opposite sign for temperature: β = −0.484 (95% HPD −0.900 to −0.237) at origin and β = −0.227 (95% HPD −0.499 to −0.047) at destination. These are the estimates of the model without sample size (Fig. 9A). Both intervals lie entirely below zero, so by the paper's own rule both effects are significant and negative, and the Results and Discussion describe them that way ("a negative and supportive predictor"; "the negative correlation between temperature and transmission risk"). In the alternative models that add sample size as a predictor (Fig. 9B; Supplementary Fig. 24B), temperature is rarely included (activation rates of 2.7% or less) and its intervals span zero, so no model in the paper gives a supported positive temperature effect. The poultry-density half of the sentence agrees with the reported chicken-stock coefficient (+0.461); only the temperature direction is reversed. The error sits in the most-read part of the paper and concerns one of the named drivers of spread, but the underlying estimates and the body text are consistent with each other.

What the paper reports

In the abstract:

Geographic distance is negatively associated with the spread of the H7 virus, while temperature and poultry density show positive association.

— Abstract

The editor's summary printed with the abstract names temperature as a driver without giving a direction:

Authors show that spread of H7 avian influenza viruses is driven by geographic distance and temperature.

— Abstract (editor's summary)

In the Results, the GLM estimates are given as posterior mean and 95% HPD:

Geographic distance was strongly negatively associated with viral spread (coefficient = −0.873, 95% HPD: −0.995 to −0.753), with decisive support for its inclusion in the model (activation rate = 1) (Fig. 9A).

Chicken stock origin, defined as the number of annual chicken inventory at the source country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), while temperature origin (−0.484, −0.900 to −0.237) and temperature destination (−0.227, −0.499 to −0.047) was also considered as a negative and supportive predictor.

— Results, "Mechanisms of virus spatial spread"

In the Discussion:

Chicken stock origin consistently made a positive contribution to the spread of the virus, while temperature origin was considered as a supportively negative predictor.

Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may reduce viral persistence in the environment, which explains the negative correlation between temperature and transmission risk.

— Discussion, paragraph on the GLM

The concluding paragraph lists temperature without a direction:

Our findings highlight how geographic distance, temperature, GDP, and poultry density collectively shape viral dispersal across regions.

— Discussion, concluding paragraph

The Methods define the model and the significance rule:

This GLM approach was extended from the discrete diffusion model, which characterizes the logarithm of the transition rates within the CTMC matrix as a log-linear function dependent on multiple potential indicators of viral spread.

All predictors were log-transformed and standardized except bird migration before inclusion in the matrices.

HPD intervals (Highest Posterior Density intervals) from the posterior distribution were examined for each coefficient: if the HPD interval does not contain zero, the effect is considered significant; if it contains zero, the effect is uncertain.

— Methods, "Phylogeographic inference with epidemiological predictors"

Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC13351079) and occurs exactly once; recompute.py.txt reruns that check. The archived PMC text renders minus signs as ASCII hyphens; the publisher's HTML uses the Unicode minus sign (U+2212), which is what is shown above. Footnote numerals (e.g. "56") are reproduced as they appear in the archived text.

What the paper's own numbers show

Method. The claim concerns direction only, so the check uses the paper's own estimates and its own decision rule. Under the stated model, log Λij = Σp βpδpxij,p, a negative β means that a higher (log-transformed, standardized) predictor value is associated with a lower transition rate, conditional on the predictor's inclusion. For each coefficient reported in the Results text, the sign of the 95% HPD interval was classified (entirely above zero: positive; entirely below: negative; straddling zero: uncertain) and compared with the direction the abstract states. As a descriptive aid, exp(β) gives the multiplicative change in the transition rate per one standard deviation of the predictor; its interval is the one-to-one transform of the reported HPD endpoints, which is not necessarily an HPD on that scale. No posterior samples or Fig. 9 source data are published, so the model itself cannot be re-fitted; the check is confined to the numbers the paper prints. recompute.py.txt prints every value below.

Quantity Location Reported Recomputed from the paper's own numbers
Direction of temperature association Abstract "positive association" Contradicted by both temperature coefficients (rows below)
Temperature (origin), β and 95% HPD Results, "Mechanisms of virus spatial spread" −0.484 (−0.900 to −0.237); "a negative and supportive predictor" HPD entirely below 0 (upper bound −0.237): negative, significant by the paper's rule; exp(β) = 0.616 (0.407 to 0.789), about 38% lower rate per SD
Temperature (destination), β and 95% HPD Same paragraph −0.227 (−0.499 to −0.047); same wording HPD entirely below 0 (upper bound −0.047): negative; exp(β) = 0.797 (0.607 to 0.954), about 20% lower rate per SD
Direction of temperature association Discussion "supportively negative predictor"; "negative correlation between temperature and transmission risk" Agrees with the coefficients
Chicken stock (origin), the abstract's "poultry density" Results +0.461 (0.111 to 0.845); abstract: positive HPD entirely above 0: positive; exp(β) = 1.586 (1.117 to 2.328). Abstract agrees
Geographic distance Results −0.873 (−0.995 to −0.753); abstract: negative HPD entirely below 0: negative; exp(β) = 0.418 (0.370 to 0.471). Abstract agrees
Abstract direction statements checked against the reported HPDs Abstract vs Results 4 directions (distance, poultry density, and temperature at origin and destination) 2 of 4 agree (distance, poultry density); 2 of 4 contradicted (both temperature terms)

Both temperature estimates printed in the text are negative, with intervals that exclude zero, and the paper's own text describes and explains a negative relationship. The abstract's "positive" cannot be derived from these numbers.

Figure

Two text panels, "Abstract" and "Results and Discussion", holding the conflicting excerpts on the direction of the temperature effect, joined by a connector labelled "conflict: sign of temperature"

Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the clause that assigns a positive association to temperature and poultry density highlighted. Right (blue): the Results sentence that reports both temperature coefficients with their 95% HPD intervals and calls temperature "a negative and supportive predictor," and the two Discussion sentences that call it "a supportively negative predictor" and explain "the negative correlation between temperature and transmission risk." The footer notes that the abstract's directions for distance and poultry density (chicken stock origin) agree with the reported coefficients; only the temperature direction conflicts.

Dot-and-interval plot of the six GLM coefficients reported in the Results text on one axis with a reference line at zero, and two columns at right comparing the direction stated in the abstract with the sign of each reported HPD

Figure 2. Reported GLM coefficients and the direction each implies. Orange dots and lines are the posterior means and 95% HPD intervals as printed in the Results text (log scale of the transition rate; the vertical line at 0 marks no effect). At right, "Abstract says" gives the direction stated in the abstract (a dash where it states none), and the blue "Sign of reported HPD" column gives the direction implied by each interval under the paper's rule. The two temperature rows, in bold, are the only ones where the two columns disagree; both intervals lie entirely to the left of zero. The sample-size coefficients are shown for context only; according to the Fig. 9 legend they come from the model that includes sample size (Fig. 9B). The other four are from the Fig. 9A model (the distance value is cited to Fig. 9A in the text; the others match the Fig. 9A plot).

Why it matters

Explanations considered

  1. A different model reports a positive temperature effect. No model gives a supported positive effect. The paper fits a model without sample size (Fig. 9A) and one with it (Fig. 9B), and repeats both with GDP per capita added (Supplementary Fig. 24A and B, described as "similar"). In the two models without sample size, both temperature terms are negative: the text gives −0.484 and −0.227 for Fig. 9A, and Supplementary Fig. 24A plots means of about −0.45 and −0.2 with posterior draws below zero (activation rate 56.7% for Temperature (o)). In the two models with sample size, temperature has almost no inclusion support and intervals that span zero. In Fig. 9B the plotted means are about −0.33 (origin) and −0.02 (destination), with activation rates of 1.9% and 0.1%. In Supplementary Fig. 24B the Temperature (o) mean is plotted slightly above zero (about +0.15), but its median is about −0.35, its 95% HPD runs from roughly −0.8 to +2.4, and its activation rate is 2.7%; Temperature (d) sits near zero at 0.1%. Under the paper's rule, these are "uncertain" effects, not positive ones. The values for Fig. 9B and Supplementary Fig. 24 are read by eye from the publisher's figure images, as no source data are published for them. The model the text reports and the abstract summarizes is the one with significant negative temperature terms.
  2. "Positive association" means "supported for inclusion," not direction. This does not fit the sentence. The same sentence uses "negatively associated" for distance, which can only be a direction, and the Results keep the two ideas separate by calling temperature "a negative and supportive predictor." Even under this reading, the destination term would not qualify: the activation rate printed for Temperature (d) in Fig. 9A is 0.7%, which falls in the band the Methods describe as "<5% suggests potential unimportance." (Temperature (o) is printed at 68.0%.)
  3. Sign convention or transformation. The Methods specify a log-linear model of the transition rates with log-transformed, standardized predictors. Under that parameterization a negative β is a negative association, and the Discussion reads it that way. The transformation of the temperature variable does not change the sign of its association with spread.
  4. Confusion with a correlated predictor. The Results report a Spearman correlation of r = −0.65 between GDP per capita and temperature, and GDP was dropped from the final model for that reason. That is a correlation between two predictors, not the association of temperature with spread, and adding GDP does not produce a supported positive temperature effect (item 1): in Supplementary Fig. 24A both temperature terms remain negative, and in Supplementary Fig. 24B the origin term is included in 2.7% of samples with an interval spanning zero.
  5. "Poultry density" carries the positive sign and temperature was listed alongside it. The sentence does not allow that reading. "temperature and poultry density show positive association" makes the two nouns the joint subject of one plural verb, so as printed it assigns a positive direction to both. It also contrasts them ("while") with distance, which is "negatively associated". A reader cannot recover the negative temperature effect from the abstract.
  6. Rounding, a typo in the coefficients, or a text-conversion error. None applies. The sign is carried by the words "negative" and "positive" as well as by the minus signs, and the minus signs in the Results text agree with the Fig. 9A plot, where both temperature distributions lie below the zero line. The publisher's HTML and the PMC archive give the same abstract wording.

Limitations

Status

As of 2026-10-23:

Sources: PMC13351079 (PubMed Central archive, read with src/show.py); https://doi.org/10.1038/s41467-026-72718-9 (publisher's article page and Fig. 9 image); Supplementary Information PDF (41467_2026_72718_MOESM1_ESM.pdf), p. 22.

Suggested correction

In the Abstract, replace "while temperature and poultry density show positive association" with "while poultry density shows a positive association and temperature a negative association", so that the abstract matches the coefficients in the Results (temperature origin −0.484, 95% HPD −0.900 to −0.237; temperature destination −0.227, −0.499 to −0.047) and the Discussion. The authors could also state in the Results that, in the model that includes sample size (Fig. 9B), temperature has low inclusion support (activation rates 1.9% and 0.1%) and intervals that span zero.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13351079 (Nat Commun 2026, doi:10.1038/s41467-026-72718-9)
# Abstract vs Results/Discussion: direction of the temperature association in the
# Bayesian phylogeographic GLM of H7 influenza spread.
#
# Run:  python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs
#  * Quotes: copied verbatim from the article as archived in PubMed Central (PMC13351079).
#    The PMC text served by src/show.py writes minus signs as ASCII '-', while the
#    publisher's HTML uses U+2212; the check below treats the two as the same character.
#    When run inside the /home/ubuntu/papers repo, each quote is re-checked against the
#    archived full text (skipped elsewhere).
#  * GLM coefficients: the posterior means and 95% HPD bounds printed in the Results
#    paragraph "Mechanisms of virus spatial spread". No posterior samples or source data for
#    Fig. 9 are published (the Source Data file covers Figs 1 and 8 only), so the model cannot
#    be re-fitted; the check is on the paper's own reported numbers.

import math, os, subprocess

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13351079"

# ------------------------------------------------------------------ quotes
QUOTES = {
    "abstract": ("Abstract",
        "Geographic distance is negatively associated with the spread of the H7 virus, while "
        "temperature and poultry density show positive association."),
    "editor_summary": ("Abstract (editor's summary)",
        "Authors show that spread of H7 avian influenza viruses is driven by geographic distance "
        "and temperature."),
    "results_distance": ("Results, 'Mechanisms of virus spatial spread'",
        "Geographic distance was strongly negatively associated with viral spread (coefficient = "
        "-0.873, 95% HPD: -0.995 to -0.753), with decisive support for its inclusion in the model "
        "(activation rate = 1) (Fig. 9A)."),
    "results_temp": ("Results, 'Mechanisms of virus spatial spread'",
        "Chicken stock origin, defined as the number of annual chicken inventory at the source "
        "country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), "
        "while temperature origin (-0.484, -0.900 to -0.237) and temperature destination (-0.227, "
        "-0.499 to -0.047) was also considered as a negative and supportive predictor."),
    "results_gdp": ("Results, 'Mechanisms of virus spatial spread'",
        "The results incorporating GDP were similar (Supplementary Fig. 24)."),
    "disc_temp": ("Discussion, paragraph on the GLM",
        "Chicken stock origin consistently made a positive contribution to the spread of the virus, "
        "while temperature origin was considered as a supportively negative predictor."),
    "disc_mech": ("Discussion, paragraph on the GLM",
        "Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may "
        "reduce viral persistence in the environment, which explains the negative correlation "
        "between temperature and transmission risk."),
    "conclusion": ("Discussion, concluding paragraph",
        "Our findings highlight how geographic distance, temperature, GDP, and poultry density "
        "collectively shape viral dispersal across regions."),
    "disc_ews": ("Discussion, paragraph on prevention and control",
        "Integrating real-time environmental variables such as temperature monitoring into early "
        "warning systems"),
    "methods_glm": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
        "This GLM approach was extended from the discrete diffusion model, which characterizes the "
        "logarithm of the transition rates within the CTMC matrix as a log-linear function dependent "
        "on multiple potential indicators of viral spread."),
    "methods_std": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
        "All predictors were log-transformed and standardized except bird migration before "
        "inclusion in the matrices."),
    "methods_hpd": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
        "HPD intervals (Highest Posterior Density intervals) from the posterior distribution were "
        "examined for each coefficient: if the HPD interval does not contain zero, the effect is "
        "considered significant; if it contains zero, the effect is uncertain."),
    "methods_act": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
        "Activation rate (the frequency with which a predictor is included in the model) was used "
        "to gauge importance: an activation rate ≥50% indicates a highly important predictor, "
        "10–50% indicates moderate importance, and <5% suggests potential unimportance."),
}


def norm(s):
    return " ".join(s.replace("−", "-").split())


def check_quotes():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.\n")
        return
    try:
        full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
                              capture_output=True, text=True, timeout=300).stdout
    except Exception as e:  # pragma: no cover
        print(f"Quote check: could not load archived text ({e}); skipped.\n")
        return
    full_n = norm(full)
    print("Quote check against the PMC-archived full text (occurrences; >=1 means found verbatim):")
    for key, (loc, q) in QUOTES.items():
        print(f"  {key:17s} {full_n.count(norm(q))}   ({loc})")
    print()


# ------------------------------------------------------------------ reported GLM numbers
# (label, mean, HPD low, HPD high, direction the abstract states or None)
# All from the Results paragraph quoted above (Fig. 9A model, except sample size, which the
# Fig. 9 legend places in the model of Fig. 9B).
COEFS = [
    ("Geographic distance",      -0.873, -0.995, -0.753, "negative"),
    ("Chicken stock (origin)",    0.461,  0.111,  0.845, "positive"),   # "poultry density"
    ("Temperature (origin)",     -0.484, -0.900, -0.237, "positive"),
    ("Temperature (destination)", -0.227, -0.499, -0.047, "positive"),
    ("Sample size (origin)",      1.200,  0.777,  1.600, None),
    ("Sample size (destination)", 0.598,  0.402,  0.809, None),
]


def classify(lo, hi):
    if lo > 0:
        return "positive"
    if hi < 0:
        return "negative"
    return "uncertain (HPD contains 0)"


def recompute():
    print("Reported GLM coefficients (log scale) and the direction they imply under the paper's rule")
    print("('if the HPD interval does not contain zero, the effect is considered significant'):\n")
    print(f"  {'predictor':27s} {'beta':>7s}  {'95% HPD':>17s}  {'implied':>9s}  {'abstract says':>13s}  agree?")
    out = []
    for lab, m, lo, hi, abs_dir in COEFS:
        imp = classify(lo, hi)
        agree = "-" if abs_dir is None else ("yes" if abs_dir == imp else "NO")
        print(f"  {lab:27s} {m:7.3f}  [{lo:6.3f}, {hi:6.3f}]  {imp:>9s}  {abs_dir or '(not stated)':>13s}  {agree}")
        out.append((lab, m, lo, hi, abs_dir, imp, agree))
    print()
    print("Rate multipliers exp(beta) per 1 SD of the log-transformed, standardized predictor,")
    print("conditional on inclusion (endpoints transformed one-to-one; not necessarily an HPD on")
    print("the multiplier scale):")
    for lab, m, lo, hi, *_ in out:
        print(f"  {lab:27s} exp(beta) = {math.exp(m):.3f}   [{math.exp(lo):.3f}, {math.exp(hi):.3f}]")
    print()
    for lab, m, lo, hi, *_ in out:
        if lab.startswith("Temperature"):
            print(f"  {lab}: upper HPD bound {hi:+.3f} < 0 -> {hi < 0}; "
                  f"multiplier {math.exp(m):.3f} -> {100 * (1 - math.exp(m)):.1f}% lower rate per SD")
    n_temp_neg = sum(1 for r in out if r[0].startswith("Temperature") and r[5] == "negative")
    print(f"\n  Temperature coefficients whose HPD lies entirely below 0: {n_temp_neg} of 2")
    print(f"  Abstract direction statements contradicted by the reported HPDs: "
          f"{sum(1 for r in out if r[6] == 'NO')} of {sum(1 for r in out if r[4])}\n")
    return out


# ------------------------------------------------------------------ figures
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "svg.fonttype": "none",
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
                     "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT2,
                     "ytick.color": TXT})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)
    print(f"wrote {name}.svg and {name}.png")


def fig1():
    W, H = 7.2, 5.0
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")

    def panel(x, w, title, sub, body, accent):
        ax.add_patch(FancyBboxPatch((x, 0.95), w, 3.9, boxstyle="round,pad=0.02,rounding_size=0.08",
                                    fc="white", ec=accent, lw=1.5))
        ax.text(x + 0.14, 4.72, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
        ax.text(x + 0.14, 4.42, sub, fontsize=10, va="top", color=accent)
        yy = 4.08
        for line, style in body:
            ax.text(x + 0.14, yy, line, fontsize=10, va="top",
                    color={"n": TXT, "b": accent, "s": TXT2}[style],
                    fontweight="bold" if style == "b" else "normal",
                    fontstyle="italic" if style == "s" else "normal")
            yy -= 0.235

    left = [("“Geographic distance is negatively", "n"),
            ("associated with the spread of", "n"),
            ("the H7 virus, while", "n"),
            ("temperature and poultry", "b"),
            ("density show positive", "b"),
            ("association.”", "b"),
            ("", "n"), ("", "n"), ("", "n"), ("", "n"), ("", "n"),
            ("States: temperature > 0", "s"),
            ("(and poultry density > 0).", "s")]
    right = [("Results, GLM paragraph:", "s"),
             ("“…temperature origin (−0.484, −0.900", "n"),
             ("to −0.237) and temperature destination", "n"),
             ("(−0.227, −0.499 to −0.047) was also", "n"),
             ("considered as a negative and", "b"),
             ("supportive predictor.”", "b"),
             ("Discussion:", "s"),
             ("“…temperature origin was considered", "n"),
             ("as a supportively negative predictor.”", "b"),
             ("“…which explains the negative", "b"),
             ("correlation between temperature and", "b"),
             ("transmission risk.”", "b"),
             ("Both temperature HPDs lie below 0.", "s")]
    panel(0.15, 2.75, "Abstract", "reported claim", left, ORANGE)
    panel(3.62, 3.43, "Results and Discussion", "the paper's own estimates and text", right, BLUE)

    ax.add_patch(FancyArrowPatch((2.94, 3.55), (3.58, 3.55), arrowstyle="<|-|>", mutation_scale=14,
                                 lw=2, color=TXT))
    ax.text(3.26, 3.85, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
    ax.text(3.26, 3.28, "sign of\ntemper-\nature", ha="center", va="top", fontsize=10, color=TXT2)

    ax.text(0.15, 0.48,
            "Distance (β = −0.873) and chicken stock origin (β = +0.461; the abstract's “poultry\n"
            "density”) have the signs the abstract states. Only the temperature direction differs.",
            fontsize=10, color=TXT2, va="center")
    save(fig, "fig1")


def fig2(rows):
    n = len(rows)
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    fig.subplots_adjust(left=0.29, right=0.66, top=0.86, bottom=0.17)
    ax.set_xlim(-1.3, 1.8)
    ax.set_ylim(n - 0.4, -0.7)
    ax.axvline(0.0, color=TXT2, lw=1.0, zorder=1)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.set_xticks([-1.0, -0.5, 0.0, 0.5, 1.0, 1.5])
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(TXT2)
    ax.tick_params(axis="y", length=0)
    ax.set_yticks(range(n))
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_xlabel("GLM coefficient β (log scale of transition rate)", fontsize=10, color=TXT)
    for i, (lab, m, lo, hi, abs_dir, imp, agree) in enumerate(rows):
        ax.plot([lo, hi], [i, i], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
        ax.plot([m], [i], "o", ms=8, color=ORANGE, zorder=4)
        fw = "bold" if agree == "NO" else "normal"
        ax.text(1.03, i, abs_dir or "—", transform=ax.get_yaxis_transform(), va="center",
                fontsize=10, color=TXT, fontweight=fw)
        ax.text(1.36, i, imp, transform=ax.get_yaxis_transform(), va="center",
                fontsize=10, color=BLUE, fontweight=fw)
    ax.text(1.03, -0.95, "Abstract\nsays", transform=ax.get_yaxis_transform(), va="bottom",
            fontsize=10, color=TXT, fontweight="bold")
    ax.text(1.36, -0.95, "Sign of\nreported HPD", transform=ax.get_yaxis_transform(), va="bottom",
            fontsize=10, color=BLUE, fontweight="bold")
    ax.text(0.0, -0.95, "no effect", ha="center", va="bottom", fontsize=10, color=TXT2)
    ax.plot([], [], "-o", color=ORANGE, lw=2, ms=8, label="Reported mean and 95% HPD (Results)")
    ax.legend(loc="upper left", bbox_to_anchor=(-0.45, -0.14), frameon=False, fontsize=10)
    save(fig, "fig2")


if __name__ == "__main__":
    check_quotes()
    rows = recompute()
    fig1()
    fig2(rows)

Table 1 P values do not follow from its own mouse counts under the stated tests

Nat Commun (2017) · doi:10.1038/ncomms14382 · PMC5309857

Summary

Table 1 reports how many recipient mice were engrafted ("positive") after p190-B−/− LSK-SLAM cells were treated ex vivo: 9 of 16 untreated, 4 of 13 with rTGF-β1, and 13 of 20 with rTGF-β1 plus the p38MAPK inhibitor SB203580. It gives Fisher exact P values of 0.0006 and <0.0001 for the two pairwise comparisons, and a three-group χ² P<0.0001. With these counts, the stated tests give P = 0.264, P = 0.080 and χ² = 3.80 on 2 df (P = 0.149), so none of the three comparisons is significant at 0.05. All three reported values come out exactly if the rounded percentages (56%, 31%, 65%) are entered as counts out of 100 mice, as the footer's phrase "comparing per cent of negative and positive mice" suggests. Supplementary Table 3 shows the same pattern. Table 1 is the main-text in vivo test cited for the claim that rTGF-β1 "significantly reduced" long-term HSC activity through p38MAPK, so as printed that claim has no statistical support from these data.

What the paper reports

Table 1 ("Effect of TGFβ on p190-B−/− HSPC engraftment"). The columns are Chimera>1%, Chimera<1%, Total mice transplanted and P value:

p190-B-/- | 9 | 7 | 16 |

p190-B-/-+rTGFβ1 | 4 | 9 | 13 | 0.0006*

p190-B-/-+rTGFβ1+p38 inhi | 13 | 7 | 20 | < 0.0001†

Table 1

Table shows numbers of mice exhibiting more than 1% contribution to both myeloid and lymphoid lineages in PB (positive mice) and numbers of mice with less than 1% contribution in at least one lineage (negative mice) (2–3 independent experiments). P values were calculated by Fisher exact 2 × 2 contingency table by comparing per cent of negative and positive mice of the following groups: *p190-B-/- vs p190-B-/-+rTGF-β1, and †p190-B-/-+rTGF-β1 vs p190-B-/-+rTGF-β1+ SB203580. Difference between the three groups was analysed by χ2 test, P<0.0001.

Table 1, footer

How the Results use the table:

But, LT-HSC activity of pooled p190-B-/- LSK-SLAM cells cultured with rTGF-β1 was significantly reduced (30% of transplanted mice showing greater than 1% contribution to both myeloid and lymphoid lineages in PB versus 56% from non-treated cultures). This effect was prevented by addition of SB203580 (65% mice were engrafted (Table 1 and Supplementary Table 3).

Hence, rTGF-β1 can alter HSC activity via p38MAPK signalling pathway in vitro.

Results, section "p38MAPK pathway mediates TGF-β effect on HSPC activity"

Here, the authors show using both in vitro assays and in vivo transplantation that loss of the RhoGAP p190-B enhances self-renewal by inhibiting TGFβ/p38 signalling.

Abstract (editor's summary)

The statistics section names only the paired-daughter-cell and single-cell analyses ("In vitro paired daughter cell assay data were analysed by 2 × 2 Fisher exact test contingency table", Methods, Statistical analyses). It does not describe any other analysis of the transplantation counts.

Supplementary Table 3 (fetal-liver LSK-SLAM cells, from the Supplementary Information PDF) reports 8/4 (n = 12) untreated, 2/10 (n = 12) with rTGF-β1, P "<0.0001", and 4/6 (n = 10) with rTGF-β1 plus the p38 inhibitor, P "0.0005". Its footer reads: "P value was calculated by Fisher exact test ... P value by chi square test was < 0.0001 in both sets of transplant."

What the paper's own numbers show

Method. Each comparison named in the footer was set up as a 2 × 2 table of positive and negative mice and analysed with the two-sided Fisher exact test (scipy). As sensitivity checks I also ran the one-sided Fisher test, the χ² test with and without Yates correction, and Barnard's and Boschloo's unconditional exact tests. The three-group comparison is the Pearson χ² test on the 3 × 2 table. For each pairwise comparison I computed the difference in per cent positive with a Newcombe (hybrid Wilson score) 95% CI. To test the footer's wording, I then ran the same tests on the rounded percentages entered as if they were counts out of 100. Supplementary Table 3 was treated the same way. All numbers come from recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
Per cent positive, p190-B−/− (9/16) Table 1; Results 56% 56.2%
Per cent positive, +rTGF-β1 (4/13) Table 1; Results 30% 30.8%
Per cent positive, +rTGF-β1+SB203580 (13/20) Table 1; Results 65% 65.0%
Fisher, p190-B−/− vs +rTGF-β1 (9/7 vs 4/9) Table 1, * P = 0.0006 P = 0.264 two-sided (one-sided 0.160; χ² 0.170, Yates 0.319; Barnard 0.184; Boschloo 0.192)
Difference in per cent positive, same comparison derived — +25.5 points, 95% CI −9.9 to +52.9
Fisher, +rTGF-β1 vs +rTGF-β1+SB203580 (4/9 vs 13/7) Table 1, † P < 0.0001 P = 0.080 two-sided (one-sided 0.058; χ² 0.055, Yates 0.117; Barnard 0.063; Boschloo 0.069)
Difference in per cent positive, same comparison derived — −34.2 points, 95% CI −59.0 to +0.3
Three-group χ², 3 × 2 Table 1 footer P < 0.0001 χ² = 3.80, df 2, P = 0.149
Multiplier of the real counts needed to reach the reported P derived — ×7 (first comparison), ×5 (second)
Fisher on % as counts, 56/44 vs 31/69 diagnostic 0.0006 P = 5.8 × 10⁻⁴ (prints as 0.0006)
Fisher on % as counts, 31/69 vs 65/35 diagnostic < 0.0001 P = 2.4 × 10⁻⁶
3 × 2 χ² on % as counts diagnostic < 0.0001 χ² = 24.8, P = 4.1 × 10⁻⁶
Fisher, Supp. Table 3: 8/4 vs 2/10 Supp. Table 3 P < 0.0001 P = 0.036 (on % as counts, 67/33 vs 17/83: 6.2 × 10⁻¹³)
Fisher, Supp. Table 3: 2/10 vs 4/6 Supp. Table 3 P = 0.0005 P = 0.348 (on % as counts, 17/83 vs 40/60: 5.06 × 10⁻⁴, prints as 0.0005)
3 × 2 χ², Supp. Table 3 Supp. Table 3 footer P < 0.0001 χ² = 6.20, df 2, P = 0.045 (on % as counts: 6.1 × 10⁻¹²)

The cells, row totals (9 + 7 = 16, 4 + 9 = 13, 13 + 7 = 20), the percentages in the Results text and the footer all agree with each other. The discrepancy lies only in the P values. Each reported value sits exactly where the stated test lands when the per cent figures are treated as counts out of 100, which treats each group as about 5 to 8 times larger than it was (100 versus 13 to 20 mice). The match is specific. For the first comparison, 4/13 rounds to 31%, and 56/44 vs 31/69 gives 0.00058, which prints as the reported 0.0006. Using the 30% from the Results text instead gives 0.00033.

Figure

Reported P values vs P values recomputed from Table 1 and Supplementary Table 3 counts

Figure 1. P values on a log scale for the three tests in Table 1 (top) and the three in Supplementary Table 3 (bottom). The thin vertical line marks p = 0.05. Orange diamonds are the values the paper prints; an arrow means the paper gives only an upper bound ("<0.0001"). Filled blue circles are the stated test (two-sided Fisher exact for pairs, Pearson χ² for three groups) applied to the mouse counts printed in the same table. Open blue squares are the same test applied to the rounded per cent positive and negative entered as if each group had 100 mice. In every row the reported value coincides with, or is a correct bound for, the open square, not the filled circle. On the actual counts, all three Table 1 tests are above 0.05.

How the reported P = 0.0006 arises: the same Fisher test on mouse counts vs on percentages entered as counts

Figure 2. Calculation flow for the first Table 1 comparison (p190-B−/− vs +rTGF-β1). Upper path: the two-sided Fisher exact test on the printed numbers of mice (9 positive / 7 negative vs 4 / 9) gives p = 0.264, which is 440 times the reported P = 0.0006 (orange box). Lower path: the same test on the rounded per cent positive and negative (56 / 44 vs 31 / 69), entered as if each group had 100 mice, gives p = 0.00058, which prints as the reported 0.0006. The line at the bottom gives the same pair of values for the other two Table 1 tests, both printed in the paper as P < 0.0001. Blue marks values recomputed from the paper's own numbers; orange marks the paper's reported value.

Why it matters

Table 1 is the paper's main-text in vivo functional test of the step from TGF-β through p38MAPK to long-term HSC activity. The Results cite it, together with Supplementary Table 3, for the statements that rTGF-β1 "significantly reduced" LT-HSC activity, that the effect "was prevented by addition of SB203580", and hence that "rTGF-β1 can alter HSC activity via p38MAPK signalling pathway". The editor's summary relies on "in vivo transplantation" for the TGFβ/p38 conclusion.

On the printed counts, neither pairwise comparison in Table 1 is significant, and the three-group test gives P = 0.149. The intervals show how little the data pin down. The rTGF-β1 reduction (+25.5 percentage points) is compatible with anything from a 10-point increase to a 53-point decrease, and the SB203580 rescue (−34.2 points) has an upper CI limit just above zero. The direction of the effects (56% → 31% → 65%) is as described. What the table does not support is the statement that the differences are significant, let alone at P ≤ 0.0006.

Supplementary Table 3 does not supply the missing support at the stated level. Its rTGF-β1 effect is nominally significant on the counts (Fisher P = 0.036, against the printed <0.0001), its p38-inhibitor rescue is not (P = 0.348, against the printed 0.0005), and its three-group χ² gives P = 0.045 (printed <0.0001). Pooling the two tables, which the paper does not describe, gives P = 0.012, 0.027 and a three-group P = 0.014. These values are shown for scale only and are not a reanalysis the paper reports.

The abstract itself does not quote these P values, and the paper's p190-B/TGF-β/p38 conclusions also rest on other experiments (paired-daughter-cell assays, p-p38MAPK and bioactive TGF-β measurements, and TGFBRI-inhibitor experiments). The same wording, "comparing per cent", appears in the Fisher-test legends of Figures 3, 7 and 9. I have not checked whether those tests have the same problem.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Recompute the Table 1 P values on the numbers of mice (9/7, 4/9 and 13/7) rather than on percentages. With two-sided Fisher exact tests this gives P = 0.26 for p190-B−/− vs +rTGF-β1 and P = 0.080 for +rTGF-β1 vs +rTGF-β1+SB203580, and the three-group χ² gives χ² = 3.80, df 2, P = 0.15. Replace the footer's "comparing per cent of negative and positive mice" with "comparing numbers of negative and positive mice". Recompute Supplementary Table 3 the same way (P = 0.036, 0.35 and χ² P = 0.045). Revise the Results sentence "was significantly reduced" and the inference that the effect was "prevented" by SB203580 to match the recomputed values, for example by reporting the observed proportions with confidence intervals. The other Fisher tests whose legends use "per cent" (Figures 3, 7 and 9) should be checked to confirm they were run on counts.

Recomputation script (python recompute.py.txt)
# Recomputation for PMC5309857 (Nat Commun 2017, doi:10.1038/ncomms14382), Table 1
# and Supplementary Table 3. Run:  python recompute.py.txt
# Prints every number used in writeup.md and writes fig1.svg / fig1.png next to this file.
import os
import numpy as np
from scipy.stats import fisher_exact, chi2_contingency, barnard_exact, boschloo_exact, norm

HERE = os.path.dirname(os.path.abspath(__file__))

# ---- Counts as printed: (positive mice "Chimera>1%", negative mice "Chimera<1%")
T1 = {"KO": (9, 7), "KO+TGF": (4, 9), "KO+TGF+p38i": (13, 7)}         # Table 1
S3 = {"KO": (8, 4), "KO+TGF": (2, 10), "KO+TGF+p38i": (4, 6)}          # Supplementary Table 3
PAIRS = [("KO", "KO+TGF"), ("KO+TGF", "KO+TGF+p38i")]
REPORTED = {  # what the paper prints (value, is_upper_bound)
    ("T1", "KO", "KO+TGF"): (0.0006, False),
    ("T1", "KO+TGF", "KO+TGF+p38i"): (0.0001, True),
    ("T1", "chi2"): (0.0001, True),
    ("S3", "KO", "KO+TGF"): (0.0001, True),
    ("S3", "KO+TGF", "KO+TGF+p38i"): (0.0005, False),
    ("S3", "chi2"): (0.0001, True),
}

def wilson(x, n, z=1.959964):
    p = x / n
    c = (p + z*z/(2*n)) / (1 + z*z/n)
    h = z*np.sqrt(p*(1-p)/n + z*z/(4*n*n)) / (1 + z*z/n)
    return c - h, c + h

def newcombe(x1, n1, x2, n2):
    """Newcombe hybrid-score 95% CI for p1 - p2."""
    p1, p2 = x1/n1, x2/n2
    l1, u1 = wilson(x1, n1); l2, u2 = wilson(x2, n2)
    d = p1 - p2
    return d, d - np.sqrt((p1-l1)**2 + (u2-p2)**2), d + np.sqrt((u1-p1)**2 + (p2-l2)**2)

def as_pct_counts(tab):
    """Rounded per cent positive / negative, i.e. counts out of 100."""
    out = {}
    for k, (a, b) in tab.items():
        p = round(100*a/(a+b))
        out[k] = (p, 100-p)
    return out

results = {}
for name, tab in [("T1", T1), ("S3", S3)]:
    print("=" * 72)
    print({"T1": "Table 1 (1T p190-B-/- LSK-SLAM)", "S3": "Supplementary Table 3 (fetal liver)"}[name])
    for k, (a, b) in tab.items():
        print(f"  {k:12s} {a}/{a+b} positive = {100*a/(a+b):.1f}%")
    for g1, g2 in PAIRS:
        t = np.array([tab[g1], tab[g2]])
        p2 = fisher_exact(t)[1]
        p1 = min(fisher_exact(t, alternative="less")[1], fisher_exact(t, alternative="greater")[1])
        c_u, p_u, _, _ = chi2_contingency(t, correction=False)
        _, p_y, _, _ = chi2_contingency(t, correction=True)
        p_b = barnard_exact(t).pvalue
        p_bo = boschloo_exact(t).pvalue
        a1, b1 = tab[g1]; a2, b2 = tab[g2]
        d, lo, hi = newcombe(a1, a1+b1, a2, a2+b2)
        pc = as_pct_counts(tab)
        p_pct = fisher_exact(np.array([pc[g1], pc[g2]]))[1]
        # smallest integer multiplier of the real counts that reaches the reported value
        rep, _ = REPORTED[(name, g1, g2)]
        mult = next(m for m in range(1, 50) if fisher_exact(t*m)[1] <= rep)
        results[(name, g1, g2)] = dict(fisher=p2, pct=p_pct)
        print(f"\n  {g1} vs {g2}: table {t.tolist()}")
        print(f"    reported P               : {'<' if REPORTED[(name,g1,g2)][1] else ''}{rep}")
        print(f"    Fisher two-sided          : {p2:.4f}")
        print(f"    Fisher one-sided (smaller): {p1:.4f}")
        print(f"    chi2 uncorrected          : chi2={c_u:.3f}, p={p_u:.4f}")
        print(f"    chi2 Yates                : p={p_y:.4f}")
        print(f"    Barnard / Boschloo        : {p_b:.4f} / {p_bo:.4f}")
        print(f"    difference in % positive  : {100*d:+.1f} points, Newcombe 95% CI {100*lo:+.1f} to {100*hi:+.1f}")
        print(f"    percentages as counts/100 : {[pc[g1], pc[g2]]} -> Fisher two-sided p={p_pct:.2e}")
        print(f"    real counts must be multiplied by {mult} to reach p <= {rep}")
    t3 = np.array(list(tab.values()))
    c3, p3, df3, _ = chi2_contingency(t3)
    pc = as_pct_counts(tab)
    c3p, p3p, _, _ = chi2_contingency(np.array(list(pc.values())))
    results[(name, "chi2")] = dict(fisher=p3, pct=p3p)
    print(f"\n  3x2 chi-square on counts      : chi2={c3:.2f}, df={df3}, p={p3:.3f}")
    print(f"  3x2 chi-square on % as counts : {list(pc.values())} -> chi2={c3p:.1f}, p={p3p:.1e}")

# Rounding check: the Results text says 30% for 4/13 (30.8%); 30/70 instead of 31/69
print("\nRounding sensitivity, Table 1 comparison 1 with 56/44 vs 30/70:",
      f"p={fisher_exact(np.array([[56,44],[30,70]]))[1]:.2e}")

# Pooled Table 1 + Supplementary Table 3 (an analysis the paper does not describe)
pool = {k: (T1[k][0]+S3[k][0], T1[k][1]+S3[k][1]) for k in T1}
print("\nPooled Table 1 + Supp. Table 3 (not an analysis the paper describes):", pool)
for g1, g2 in PAIRS:
    print(f"  {g1} vs {g2}: Fisher two-sided p={fisher_exact(np.array([pool[g1], pool[g2]]))[1]:.3f}")
c, p, d, _ = chi2_contingency(np.array(list(pool.values())))
print(f"  3x2 chi2={c:.2f}, df={d}, p={p:.3f}")

# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
                     "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
                     "svg.fonttype": "none"})

rows = [  # (key, label)
    (("T1", "KO", "KO+TGF"), "Table 1: p190-B⁻/⁻ vs +rTGF-β1\n9/16 vs 4/13 positive mice"),
    (("T1", "KO+TGF", "KO+TGF+p38i"), "Table 1: +rTGF-β1 vs +rTGF-β1+SB203580\n4/13 vs 13/20"),
    (("T1", "chi2"), "Table 1: three groups, 3×2 χ²"),
    (("S3", "KO", "KO+TGF"), "Supp. Table 3: p190-B⁻/⁻ vs +rTGF-β1\n8/12 vs 2/12"),
    (("S3", "KO+TGF", "KO+TGF+p38i"), "Supp. Table 3: +rTGF-β1 vs +p38 inhibitor\n2/12 vs 4/10"),
    (("S3", "chi2"), "Supp. Table 3: three groups, 3×2 χ²"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
ys = [5.6, 4.6, 3.6, 2.0, 1.0, 0.0]
for (key, lab), y in zip(rows, ys):
    rep, ub = REPORTED[key]
    r = results[key]
    lo = min(rep, r["fisher"], r["pct"])
    hi = max(rep, r["fisher"], r["pct"])
    ax.plot([lo, hi], [y, y], color="#cfcdc8", lw=2, zorder=1)
    ax.plot(r["pct"], y, marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none", zorder=3)
    ax.plot(r["fisher"], y, marker="o", ms=9, color=BLUE, ls="none", zorder=4)
    ax.plot(rep, y, marker="D", ms=8.5, color=ORANGE, ls="none", zorder=5)
    if ub:  # "<" bound: arrow pointing to smaller p
        ax.annotate("", xy=(rep/12, y), xytext=(rep/1.6, y),
                    arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2), zorder=5)
    ax.text(r["fisher"], y+0.24, f"{r['fisher']:.3f}", color=BLUE, fontsize=10, ha="center", va="bottom", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6), zorder=6)
    ax.text(rep, y+0.24, ("<" if ub else "") + f"{rep:g}", color=ORANGE, fontsize=10, ha="center", va="bottom")
ax.axvline(0.05, color=SEC, lw=0.8, zorder=0)
ax.text(0.04, -0.45, "p = 0.05", color=SEC, fontsize=10, ha="right", va="center", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6))
ax.set_xscale("log")
ax.set_xlim(1e-14, 2)
ax.set_ylim(-0.6, 6.6)
ax.set_yticks(ys)
ax.set_yticklabels([lab for _, lab in rows], fontsize=10, color=TXT)
ax.axhline(2.8, color=GRID, lw=1)
ax.set_xlabel("P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [
    Line2D([], [], marker="D", ms=8.5, color=ORANGE, ls="none", label="Reported in the paper (arrow = \"<\")"),
    Line2D([], [], marker="o", ms=9, color=BLUE, ls="none", label="Recomputed: stated test on the mouse counts"),
    Line2D([], [], marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none",
           label="Recomputed: same test, rounded % entered as counts of 100"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.32, -0.13), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig1.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200, facecolor=BG)
print("\nwrote fig1.svg, fig1.png")

# ---------------------------------------------------------------- figure 2: calculation flow
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

r1 = results[("T1", "KO", "KO+TGF")]
r2 = results[("T1", "KO+TGF", "KO+TGF+p38i")]
r3 = results[("T1", "chi2")]
pc1 = as_pct_counts(T1)

fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")

def box(x, y, w, h, text, ec, bold=False, tc=TXT):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.2",
                                fc=BG, ec=ec, lw=2))
    ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=10.5, color=tc,
            fontweight="bold" if bold else "normal", linespacing=1.4)

def arrow(x0, y0, x1, y1, color, rad=0.0):
    ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14,
                                 color=color, lw=2, connectionstyle=f"arc3,rad={rad}"))

box(0.5, 26, 13, 12, "Table 1\nmouse\ncounts", SEC)
ax.text(19, 57, "Stated test on the numbers of mice", color=BLUE, fontsize=10.5, fontweight="bold", va="center")
box(19, 41, 25, 11, f"[[{T1['KO'][0]}, {T1['KO'][1]}], [{T1['KO+TGF'][0]}, {T1['KO+TGF'][1]}]]\nn = 16 and 13", BLUE)
box(48, 41, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 41, 21, 11, f"Recomputed\np = {r1['fisher']:.3f}", BLUE, bold=True, tc=BLUE)
ax.text(19, 24.5, "Same test, per cent as counts of 100", color=BLUE, fontsize=10.5,
        fontweight="bold", va="center")
box(19, 8, 25, 11, f"[[{pc1['KO'][0]}, {pc1['KO'][1]}], [{pc1['KO+TGF'][0]}, {pc1['KO+TGF'][1]}]]\nn = 100 and 100", BLUE)
box(48, 8, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 8, 21, 11, f"Recomputed\np = {r1['pct']:.5f}", BLUE, bold=True, tc=BLUE)
for y in (46.5, 13.5):
    arrow(44.6, y, 47.5, y, SEC); arrow(65.6, y, 68.5, y, SEC)
arrow(14.2, 35, 18.4, 46.5, SEC, rad=-0.3)
arrow(14.2, 29, 18.4, 13.5, SEC, rad=0.3)
box(69, 27, 21, 9, "Reported\nP = 0.0006", ORANGE, bold=True, tc=ORANGE)
ax.plot([79.5, 79.5], [36.6, 40.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.plot([79.5, 79.5], [19.6, 26.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.text(81, 38.5, f"\u2260 {r1['fisher']/0.0006:.0f}\u00d7", ha="left", va="center", fontsize=10, color=SEC)
ax.text(81, 22.6, "= rounded", ha="left", va="center", fontsize=10, color=SEC)
ax.text(50, 1.0,
        f"Other Table 1 tests, mouse counts vs per cent as counts: SB203580 comparison p = {r2['fisher']:.3f} vs {r2['pct']:.1e};"
        f"\n3×2 χ² p = {r3['fisher']:.3f} vs {r3['pct']:.1e}. Both reported as P < 0.0001.",
        ha="center", va="center", fontsize=10, color=SEC, linespacing=1.4)
fig.savefig(os.path.join(HERE, "fig2.svg"), facecolor=BG, bbox_inches="tight", pad_inches=0.1)
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200, facecolor=BG, bbox_inches="tight", pad_inches=0.1)
print(f"ratio reported/recomputed, first comparison: {r1['fisher']/0.0006:.0f}x")
print("wrote fig2.svg, fig2.png")

Abstract reports reduced PKN1 and PKN4, but the Results report increased PKN2 and no PKN4

Nat Commun (2018) · doi:10.1038/s41467-018-04411-5 · PMC5998138

Summary

For the isogenic cell-line comparison, the abstract states that PCQ "was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells". The Results and Fig. 7 instead describe a PKN1/PKN2/PKN3 cluster in which PKN2 was "three-fold more abundant in CFBE41o- cells" and confirmed by western blot, the evidence for PKN1 is "conflicting" even about its presence, and PKN4 is never mentioned outside the abstract. The paper's own data files agree with the Results: PKN2-specific peptides have a CFBE41o-/HBE41o- ratio of 3.000, the only peptide on the PKN1 node is shared with PKN2 and PKN3 (ratio 0.500), and no file contains PKN4. Because the abstract is the indexed summary and this sentence is one of its two biological examples, it gets both the proteins and the direction of the one validated kinase change wrong.

What the paper reports

Abstract:

PCQ was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed.

Results, "Analysis of individual protein clusters", paragraph 2. This paragraph is the only passage in the Results about the PKN proteins:

Finally, a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified. PKN2 (PRK2) shares 59% sequence identity with PKN1 and was three-fold more abundant in CFBE41o- cells (Fig. 7b). Western blot confirmed that PKN2 was more abundant in CFBE41o- cells when normalized to the expression levels of Na+/K+-ATPase (Fig. 7c and 7d, Supplementary Fig. 12).

A difference in PKN2 protein quantity may contribute to the altered cell morphology between HBE41o- and CFBE41o– cells in cell culture because bronchial epithelial airway cells require PKN2 to establish apical junctions32.

PKN1 is part of two incomplete protein pairs with either PKN2 or PKN3, which provide conflicting evidence for a potential presence of PKN1. While the incomplete protein pair with PKN2 shows discrepant ratio values for the peptide nodes (∆ ≥ 2-fold), the incomplete protein pair with PKN3 shows similar ratio values (∆ < 2-fold).

Fig. 7 legend, panels b and d:

b The protein cluster of serine–threonine kinase-related protein kinases PKN1, PKN2, and PKN3 is displayed.

d Western blot analysis was used to verify the difference in PKN2 expression levels between HBE41o– and CFBE41o– cells.

Data in the western blot represent independent biological replicates, CFBE41o–: n = 2, HBE41o–: n = 2

Results, "Comparison of two isogenic cell lines with PCQ", on the labelling, which fixes the direction of the ratios:

CFBE41o– and HBE41o– cells were labeled with light and heavy isobaric isotopologues, respectively, and analyzed in a single experiment (n = 1)

Methods, "Cell culture and western blot". The only antibody against a PKN protein is anti-PKN2:

PKN2 and NaK-ATPase were detected by incubation with monoclonal anti-PKN2 antibody

Discussion. The summary of the cell-line comparison mentions no kinase:

When comparing two isogenic cell lines, we found that caspase-4 and ephrin type B receptor protein fragments were differentially regulated relative to their longer proteoforms, respectively.

What the paper's own numbers show

Method. I counted every mention of PKN1 to PKN4 in the PMC-archived text, separately for the abstract and for the rest of the article (Results, Discussion, Methods, figure legends). I searched every body sentence that names a PKN protein for wording of reduction ("reduc", "decreas", "lower", "less abundant", "down"). I then opened the two supplementary data files that hold the cell-line quantification. Supplementary Data 7 (MOESM9, the 180 classified complete protein pairs) gives log2 ratios for unique and shared peptide nodes. Supplementary Data 6 (MOESM8, "Network 5" in the Results) is the XGMML peptide-to-protein network for CFBE41o- vs HBE41o-, with ion counts for each peptide node. From it I extracted the connected component that holds the three PKN accessions. I converted log2 ratios to ratios. I checked the ratio direction against the normalized light and heavy ion counts, since the Results say light = CFBE41o- and heavy = HBE41o-. I also queried UniProt for a human gene named PKN4. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own text and data
Kinases named as changed in the cell-line comparison Abstract PKN1 and PKN4, "reduced levels" in CFBE41o- Results name PKN2 as changed (increased); no reduction of any PKN protein is stated
Mentions of PKN1 / PKN2 / PKN3 / PKN4 Abstract vs rest of article 1 / 0 / 0 / 1 in the abstract 5 / 13 / 4 / 0 in the rest of the article
Body sentences naming a PKN protein that use reduction wording Results, Methods, legends — 0 of 11
Members of the PKN cluster Results; Fig. 7b legend PKN1 (Q16512), PKN2 (Q16513), PKN3 (Q6P5Z2) same three in Supplementary Data 6; no PKN4 node in the network file
PKN2-specific peptides (3 peptides), CFBE41o-/HBE41o- Results ("three-fold more abundant in CFBE41o- cells"); Supp. Data 7 log2 1.58496 ratio 3.000 (= 2^1.585; normalized light/heavy ions 4.5/1.5 in Supp. Data 6), an increase
PKN2 western blot Results; Fig. 7d (n = 2 vs n = 2); Supp. Fig. 12 "more abundant in CFBE41o- cells" an increase, the same direction as the mass-spectrometry ratio
PKN3-specific peptides (3 peptides) Supp. Data 7 only log2 −0.678 ratio 0.625 (1.60-fold lower; light/heavy 2.5/4.0); not discussed in the text
Shared peptide IADFGLCK (PKN1, PKN2, PKN3) Supp. Data 6, 7 log2 −1.000 ratio 0.500 (light/heavy 0.75/1.5)
Peptides mapped to the PKN1 node Supp. Data 6 — only the shared IADFGLCK. The node groups Q16512 with two fragment entries (H0Y429, K7EL10), and its gene-name field reads "PKN1,PKN2"
PKN1–PKN2 incomplete pair Results ("∆ ≥ 2-fold") — unique 3.000 vs shared 0.500, a 6.00-fold discrepancy
PKN1–PKN3 incomplete pair Results ("∆ < 2-fold") — unique 0.625 vs shared 0.500, a 1.25-fold discrepancy
A PKN1-specific or PKN4 ratio article and all supplementary files — none exists
Human gene named PKN4 UniProt (query 2026-10-23) — none; human PKN genes are PKN1, PKN2 (alias PRK2, PRKCL2) and PKN3

The body's account is internally consistent, and it reproduces from the data files: both the "∆ ≥ 2-fold" and the "∆ < 2-fold" statements match the network ratios. The abstract sentence is the outlier. It names a protein (PKN4) with no data behind it in the paper. It describes as an identified reduction a PKN1 inference that the body calls conflicting. It leaves out the one PKN change that was quantified with protein-specific peptides and checked by western blot, which is an increase.

Figure

The abstract's kinase sentence and what the rest of the paper reports, side by side

Figure 1. Short verbatim excerpts from the abstract (left, orange) and from the Results and Fig. 7 legend (right, blue), with the supplementary-data values printed by recompute.py.txt. The abstract names PKN1 and PKN4 and says "reduced". The body names a PKN1/PKN2/PKN3 cluster and reports PKN2 three-fold more abundant in CFBE41o- cells (ratio 3.0 in Supplementary Data 6 and 7), confirmed by western blot. It calls the evidence for PKN1's presence conflicting, and its only peptide is shared with PKN2 and PKN3 (ratio 0.5). PKN4 is mentioned nowhere outside the abstract.

Each PKN protein on one ratio axis: the abstract's claimed direction against the paper's own ratios

Figure 2. CFBE41o-/HBE41o- abundance ratios on one log2 axis; the vertical line marks no change (ratio 1). The orange hatched bands mark the abstract's claim for PKN1 and PKN4. The abstract gives a direction ("reduced") and no value, so the bands span the whole "reduced" side and carry no magnitude. Filled blue circles are ratios from protein-specific peptides in Supplementary Data 6 and 7: PKN2 at 3.0, which is the Results' "three-fold more abundant", and PKN3 at 0.625, which the text does not discuss. The hollow blue circle is the PKN1 node's only peptide, which is shared with PKN2 and PKN3 (0.5), so it is not specific to PKN1. PKN4 has no data point because the paper and its supplements contain none.

Why it matters

Explanations considered

Status

Suggested correction

  1. Revise the abstract sentence so it matches the Results. For example: "PCQ was used to identify increased levels of the serine/threonine protein kinase PKN2 in CFBE41o- cells compared to HBE41o- cells, confirmed by western blot, and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed."
  2. If a statement about PKN1 is kept, word it as the body does: the evidence for a PKN1 proteoform is inconclusive and rests on a peptide it shares with PKN2 and PKN3.
  3. Remove "PKN4". If another protein was meant, name it and give the supporting data.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5998138 (Nat Commun 2018, doi:10.1038/s41467-018-04411-5)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (SVG + PNG at 200 dpi) in the folder that holds this script:
#   fig1.*, fig2.*                      the figures used in writeup.md
#   candidates/fig_A..fig_E.*           five candidate designs for review
# Inputs: the article text as archived in PubMed Central (read through the repo's
# src/show.py when it is available) and the one PKN row of Supplementary Data 7
# (41467_2018_4411_MOESM9_ESM.xlsx), transcribed below and re-read from the
# publisher's server if it can be reached.
import io
import os
import re
import shutil
import subprocess
import textwrap
import urllib.request
import xml.etree.ElementTree as ET
import zipfile

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
SD7_URL = ("https://media.springernature.com/original/springer-static/esm/"
           "art%3A10.1038%2Fs41467-018-04411-5/MediaObjects/41467_2018_4411_MOESM9_ESM.xlsx")

# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8"          # light tints for panel fills
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})

# ---------------------------------------------------------------- 1. text
print("=== 1. Article text (PMC archive)")
txt = None
try:
    txt = subprocess.run([PY, "src/show.py", "PMC5998138", "--full"], cwd=REPO,
                         capture_output=True, text=True, timeout=120).stdout
except Exception as e:                                   # repo not present
    print("  show.py not available:", e)
if txt:
    lines = txt.splitlines()
    abstract = next(l for l in lines if l.startswith("ABSTRACT:"))
    body = "\n".join(l for l in lines if not l.startswith("ABSTRACT:"))
    claim = re.search(r"PCQ was used to identify reduced[^.]*\.", abstract).group(0)
    print("  Abstract sentence:", claim)
    counts = {}
    for k in ["PKN1", "PKN2", "PKN3", "PKN4"]:
        counts[k] = (len(re.findall(k + r"\b", abstract)), len(re.findall(k + r"\b", body)))
        print(f"  {k}: abstract {counts[k][0]}, rest of article (Results, Methods, legends) {counts[k][1]}")
    sents = list(dict.fromkeys(s.strip() for s in re.findall(r"[^.]*PKN[^.]*\.", body)))
    red = [s for s in sents if re.search(r"reduc|decreas|lower|less abundant|down", s)]
    print(f"  body sentences naming a PKN protein: {len(sents)}; of these using reduced/decreased/"
          f"lower/less abundant/down: {len(red)}")
    m = re.search(r"PKN2 \(PRK2\)[^.]*\.", body).group(0)
    print("  PKN2 sentence:", m)
    print("  cluster accessions:", re.findall(r"PKN\d \(([A-Z0-9]{6})\)", body))
else:
    counts = {"PKN1": (1, 5), "PKN2": (0, 13), "PKN3": (0, 4), "PKN4": (1, 0)}
    print("  using counts recorded on 2026-10-23:", counts)

# ---------------------------------------------------------------- 2. SD7
# Supplementary Data 7, sheet "180 classified protein pairs": the only row naming a
# PKN accession (Q16512 PKN1, Q16513 PKN2, Q6P5Z2 PKN3).  Columns: Unique Pep1 (U1),
# Ratio (U1), Protein 1, Shared Pep (S), Ratio (S), Protein 2, Unique Pep2 (U2),
# Ratio (U2), ClassificationCase.  Ratios are log2.
SD7 = {"U1": ("LLGCEQLLTAVPGR_QGAENMTHTCASGTPK_SPAAALASSPSEGWLR", -0.678071905112637, "Q6P5Z2"),
       "S": ("IADFGLCK", -1.0, "shared"),
       "U2": ("FLSTEAISIMR_LDFDLEPEPPPAPPR_LDNLLLDTEGFVK", 1.58496250072115, "Q16513"),
       "case": "CASE3"}


def read_xlsx_rows(blob):
    """Minimal stdlib reader for the first sheet of an .xlsx file."""
    ns = {"m": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
    z = zipfile.ZipFile(io.BytesIO(blob))
    shared = []
    if "xl/sharedStrings.xml" in z.namelist():
        for si in ET.fromstring(z.read("xl/sharedStrings.xml")).findall("m:si", ns):
            shared.append("".join(t.text or "" for t in si.iter("{%s}t" % ns["m"])))
    sheet = sorted(n for n in z.namelist() if n.startswith("xl/worksheets/sheet"))[0]
    rows = []
    for r in ET.fromstring(z.read(sheet)).iter("{%s}row" % ns["m"]):
        row = []
        for c in r.findall("m:c", ns):
            v = c.find("m:v", ns)
            val = "" if v is None else v.text
            if c.get("t") == "s":
                val = shared[int(val)]
            elif c.get("t") == "inlineStr":
                val = "".join(t.text or "" for t in c.iter("{%s}t" % ns["m"]))
            row.append(val)
        rows.append(row)
    return rows


print("\n=== 2. Supplementary Data 7 (complete protein pairs)")
try:
    req = urllib.request.Request(SD7_URL, headers={"User-Agent": "Mozilla/5.0"})
    rows = read_xlsx_rows(urllib.request.urlopen(req, timeout=60).read())
    hits = [r for r in rows if any(a in " ".join(r) for a in ("Q16512", "Q16513", "Q6P5Z2", "PKN"))]
    print(f"  downloaded; {len(rows) - 1} protein-pair rows; rows naming a PKN accession: {len(hits)}")
    print("  PKN1 (Q16512) present in any row:", any("Q16512" in " ".join(r) for r in rows))
    for h in hits:
        print("  ", h)
    h = hits[0]
    assert abs(float(h[1]) - SD7["U1"][1]) < 1e-9 and abs(float(h[7]) - SD7["U2"][1]) < 1e-9
    print("  matches the transcribed values")
except Exception as e:
    print("  could not re-read the file (", type(e).__name__, "); using transcribed values")

u1, s, u2 = SD7["U1"][1], SD7["S"][1], SD7["U2"][1]
r_pkn3, r_sh, r_pkn2 = 2 ** u1, 2 ** s, 2 ** u2
print(f"  PKN2 (Q16513) unique peptides: log2 {u2:.3f} -> ratio {r_pkn2:.3f} "
      f"(log2(3) = {__import__('math').log2(3):.5f})")
print(f"  PKN3 (Q6P5Z2) unique peptides: log2 {u1:.3f} -> ratio {r_pkn3:.3f} (= {1 / r_pkn3:.2f}-fold lower)")
print(f"  shared peptide IADFGLCK:        log2 {s:.3f} -> ratio {r_sh:.3f}")
print(f"  PKN2-unique vs shared: {2 ** (u2 - s):.2f}-fold apart; PKN3-unique vs shared: {2 ** (u1 - s):.2f}-fold apart")
print("  ratio direction: the PKN2 value equals the Results' 'three-fold more abundant in "
      "CFBE41o- cells', so the ratios are CFBE41o-/HBE41o-")

# ---------------------------------------------------------------- 3. network file
# Supplementary Data 6 (41467_2018_4411_MOESM8_ESM.zip, XGMML, "Network 5" in the
# Results; label CFBEvsHBE_protpepCollapsed_3ionsPerNode_Rc_IDs). The connected
# component holding the PKN proteins, transcribed (log2 finalRatio; normalized ion
# counts, light = CFBE41o-, heavy = HBE41o- per the Results):
NET = {
    "proteins": {"Q16513 (PKN2)": ["U_PKN2", "IADFGLCK"],
                 "Q6P5Z2 (PKN3)": ["IADFGLCK", "U_PKN3"],
                 "H0Y429 K7EL10 Q16512 (PKN1 node; genes PKN1,PKN2)": ["IADFGLCK"]},
    "peptides": {"U_PKN2": (1.5849625007211563, 4.5, 1.5, 0),     # log2, normLight, normHeavy, significant
                 "IADFGLCK": (-1.0, 0.75, 1.5, 0),
                 "U_PKN3": (-0.6780719051126377, 2.5, 4.0, 0)},
}
SD6_URL = SD7_URL.replace("MOESM9_ESM.xlsx", "MOESM8_ESM.zip")
print("\n=== 3. Supplementary Data 6 (peptide-to-protein network 5, CFBE41o- vs HBE41o-)")
try:
    req = urllib.request.Request(SD6_URL, headers={"User-Agent": "Mozilla/5.0"})
    z = zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(req, timeout=120).read()))
    root = ET.fromstring(z.read(z.namelist()[0]))
    tag = lambda e: e.tag.split("}")[-1]
    nodes = {n.get("id"): n for n in root if tag(n) == "node"}
    adj = {}
    for e in root:
        if tag(e) == "edge":
            adj.setdefault(e.get("source"), set()).add(e.get("target"))
            adj.setdefault(e.get("target"), set()).add(e.get("source"))
    seed = [k for k in nodes if any(a in k for a in ("Q16512", "Q16513", "Q6P5Z2"))]
    comp, fr = set(seed), list(seed)
    while fr:
        x = fr.pop()
        for y in adj.get(x, ()):
            if y not in comp:
                comp.add(y); fr.append(y)
    print(f"  downloaded; PKN component has {len(comp)} nodes; any node naming PKN4:",
          any("PKN4" in ET.tostring(nodes[k], encoding="unicode") for k in nodes))
    for k in sorted(comp):
        a = {x.get("name"): x.get("value") for x in nodes[k].iter() if tag(x) == "att"}
        if a.get("isProtein") == "1":
            print(f"   protein {k} | genes {a.get('GeneName')} | peptide nodes {sorted(adj[k])}")
        else:
            print(f"   peptide {k[:44]} | log2 {float(a['finalRatio']):.3f} | normLight {a['normLightIons']}"
                  f" | normHeavy {a['normHeavyIons']} | significant {a['significant']}")
except Exception as e:
    print("  could not re-read the file (", type(e).__name__, "); using transcribed values")
for k, (l2, li, hv, sig) in NET["peptides"].items():
    print(f"  {k:9s} normLight/normHeavy = {li}/{hv} = {li / hv:.3f}; 2^log2 = {2 ** l2:.3f}; significant = {sig}")
print("  -> light/heavy reproduces each ratio, and light = CFBE41o-, so ratios are CFBE41o-/HBE41o-")
print("  PKN1 node: its only peptide is the shared IADFGLCK (ratio 0.500), which also maps to PKN2 and PKN3;")
print(f"    pair with PKN2: unique {r_pkn2:.3f} vs shared {r_sh:.3f} -> {2 ** (u2 - s):.2f}-fold (>= 2, 'discrepant')")
print(f"    pair with PKN3: unique {r_pkn3:.3f} vs shared {r_sh:.3f} -> {2 ** (u1 - s):.2f}-fold (< 2, 'similar')")
print("  No PKN1-specific or PKN4 ratio exists in the article or its supplements.")


# ---------------------------------------------------------------- helpers
def save(fig, path_noext):
    os.makedirs(os.path.dirname(path_noext), exist_ok=True)
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)
    print("  wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")


ABS_Q = ("PCQ was used to identify reduced levels of serine/threonine protein kinases "
         "PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells")
RES_Q = [("Results", "a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), "
          "PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified"),
         ("Results", "PKN2 (PRK2) ... was three-fold more abundant in CFBE41o- cells"),
         ("Results", "Western blot confirmed that PKN2 was more abundant in CFBE41o- cells"),
         ("Results", "conflicting evidence for a potential presence of PKN1")]


# ---------------------------------------------------------------- A: text panels
def quote_block(ax, x, y, q, wrap, step=0.19):
    for ln in textwrap.wrap("\u201c" + q + "\u201d", wrap):
        ax.text(x, y, ln, fontsize=10, color=INK, va="top")
        y -= step
    return y


def fig_A(path):
    H = 5.0
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
    # left: abstract
    lx, lw_, ltop, lh = 0.12, 2.85, H - 0.12, 2.25
    ax.add_patch(FancyBboxPatch((lx, ltop - lh), lw_, lh, boxstyle="round,pad=0,rounding_size=0.08",
                                fc=ORANGE_T, ec=ORANGE, lw=1.5))
    ax.text(lx + 0.14, ltop - 0.16, "Abstract", fontsize=11, weight="bold", va="top")
    ax.text(lx + 0.14, ltop - 0.40, "the claim as reported", fontsize=10, color=INK2, va="top")
    quote_block(ax, lx + 0.14, ltop - 0.72, ABS_Q, 34)
    ax.text(lx + 0.14, ltop - lh + 0.14, "Named: PKN1, PKN4\nDirection: reduced", fontsize=10,
            weight="bold", color=INK, va="bottom", linespacing=1.4)
    # right: body
    rx, rw, rtop = 3.95, 3.13, H - 0.12
    rh = H - 0.24
    ax.add_patch(FancyBboxPatch((rx, rtop - rh), rw, rh, boxstyle="round,pad=0,rounding_size=0.08",
                                fc=BLUE_T, ec=BLUE, lw=1.5))
    ax.text(rx + 0.14, rtop - 0.16, "Rest of the paper", fontsize=11, weight="bold", va="top")
    ax.text(rx + 0.14, rtop - 0.40, "Results, Fig. 7, Supp. Data 6 and 7", fontsize=10, color=INK2,
            va="top")
    yy = rtop - 0.72
    for _, q in RES_Q:
        yy = quote_block(ax, rx + 0.14, yy, q, 40) - 0.10
    ax.plot([rx + 0.14, rx + rw - 0.14], [yy + 0.02, yy + 0.02], color=BLUE, lw=0.8)
    yy -= 0.10
    ax.text(rx + 0.14, yy, f"Supp. Data 6, 7: PKN2 ratio {r_pkn2:.1f};\n"
            f"PKN1's only peptide is shared with PKN2\nand PKN3 (ratio {r_sh:.1f})", fontsize=10,
            color=INK, va="top", linespacing=1.3)
    yy -= 0.72
    ax.text(rx + 0.14, yy, f"PKN4: {counts['PKN4'][1]} mentions outside the abstract", fontsize=10,
            color=INK, va="top")
    ax.text(rx + 0.14, rtop - rh + 0.14, "Quantified and blotted: PKN2\nDirection: increased", fontsize=10,
            weight="bold", color=INK, va="bottom", linespacing=1.4)
    # connector
    yc = ltop - 0.95
    ax.add_patch(FancyArrowPatch((lx + lw_ + 0.04, yc), (rx - 0.04, yc), arrowstyle="<->",
                                 mutation_scale=14, lw=2, color=INK2))
    ax.text((lx + lw_ + rx) / 2, yc + 0.14, "conflict", ha="center", fontsize=10, weight="bold")
    # mismatch summary under the abstract panel
    sy = ltop - lh - 0.35
    ax.text(lx + 0.02, sy, "The two statements differ in", fontsize=10, weight="bold", va="top")
    ax.text(lx + 0.02, sy - 0.30, "1. direction: \u201creduced\u201d vs \u201cthree-fold\n"
            "    more abundant\u201d in CFBE41o- cells", fontsize=10, va="top")
    ax.text(lx + 0.02, sy - 0.86, "2. proteins: PKN1 and PKN4 vs PKN2;\n"
            "    PKN4 appears nowhere else", fontsize=10, va="top")
    save(fig, path)


# ---------------------------------------------------------------- B: evidence matrix
def fig_B(path):
    cols = ["Abstract", "Results text", "Supp. Data 6, 7\n(CFBE/HBE ratio)", "Western blot\n(Fig. 7d)"]
    rows = ["PKN1", "PKN2", "PKN3", "PKN4"]
    cells = {
        "PKN1": ["reduced", "presence\nuncertain", f"shared peptide only\n{r_sh:.1f}\u00d7 (also PKN2, 3)", "not tested"],
        "PKN2": ["not named", "three-fold more\nabundant in CFBE", f"{r_pkn2:.1f}\u00d7\n(log2 {u2:.3f})", "more abundant\nin CFBE (n = 2 vs 2)"],
        "PKN3": ["not named", "cluster member;\nno level stated", f"{r_pkn3:.3f}\u00d7\n(log2 {u1:.3f})", "not tested"],
        "PKN4": ["reduced", "0 mentions", "not listed", "not tested"],
    }
    fig = plt.figure(figsize=(W, 3.7))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, 3.7); ax.axis("off")
    x0, cw, rh, ytop = 0.85, 1.55, 0.62, 2.95
    for j, c in enumerate(cols):
        ax.text(x0 + j * cw + cw / 2, ytop + 0.30, c, ha="center", va="center", weight="bold", fontsize=10)
    for i, r in enumerate(rows):
        y = ytop - (i + 1) * rh
        ax.text(x0 - 0.12, y + rh / 2, r, ha="right", va="center", weight="bold", fontsize=11)
        for j, v in enumerate(cells[r]):
            empty = v in ("not named", "not listed", "not tested", "0 mentions")
            if j == 0:
                fc, ec = (BG, GRID) if empty else (ORANGE_T, ORANGE)
            else:
                fc, ec = (BG, GRID) if empty else (BLUE_T, BLUE)
            ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
                                   fc=fc, ec=ec, lw=1.5))
            ax.text(x0 + j * cw + cw / 2, y + rh / 2, v, ha="center", va="center", fontsize=10,
                    color=INK2 if empty else INK, weight="bold" if (j == 0 and not empty) else "normal")
    # legend
    ly = 0.18
    ax.add_patch(Rectangle((0.2, ly), 0.25, 0.18, fc=ORANGE_T, ec=ORANGE, lw=1.5))
    ax.text(0.52, ly + 0.09, "claim as reported in the abstract", va="center", fontsize=10)
    ax.add_patch(Rectangle((3.35, ly), 0.25, 0.18, fc=BLUE_T, ec=BLUE, lw=1.5))
    ax.text(3.67, ly + 0.09, "what the body and supplements show", va="center", fontsize=10)
    save(fig, path)


# ---------------------------------------------------------------- C: direction plot
def fig_C(path):
    fig, ax = plt.subplots(figsize=(W, 3.4))
    fig.subplots_adjust(left=0.10, right=0.97, top=0.80, bottom=0.17)
    ys = {"PKN1": 3, "PKN2": 2, "PKN3": 1, "PKN4": 0}
    ax.set_xscale("log", base=2)
    ax.set_xlim(1 / 8, 8)
    ax.set_ylim(-0.6, 3.75)
    ticks = [1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8]
    ax.set_xticks(ticks); ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
    ax.set_yticks(list(ys.values())); ax.set_yticklabels(list(ys.keys()), fontsize=11, color=INK)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ["top", "right", "left"]:
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.axvline(1, color=INK2, lw=1)
    ax.text(1, 3.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
    ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
    # abstract claim: direction only, no value -> shaded half-row on the 'reduced' side
    for p in ("PKN1", "PKN4"):
        y = ys[p] - (0.12 if p == "PKN1" else 0)
        ax.add_patch(Rectangle((1 / 8, y - 0.22), 1 - 1 / 8, 0.44, fc=ORANGE_T, ec=ORANGE, lw=1.5,
                               hatch="///", alpha=1))
        ax.text(0.2, y, "abstract: \u201creduced\u201d", ha="center", va="center",
                fontsize=10, color=INK, bbox=dict(fc=BG, ec="none", pad=1.5))
    ax.text(2.8, ys["PKN1"], "body: presence\nuncertain", ha="center", va="center", fontsize=10, color=INK2)
    ax.plot([r_sh], [ys["PKN1"]], "o", ms=10, mfc=BG, mec=BLUE, mew=2, zorder=5)
    ax.text(r_sh, ys["PKN1"] + 0.26, f"{r_sh:.1f}  shared peptide only (also PKN2, PKN3)", ha="center",
            va="bottom", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
    ax.text(2.8, ys["PKN4"], "body, supplements:\nnot mentioned", ha="center", va="center", fontsize=10, color=INK2)
    # recomputed / reported in body
    ax.plot([r_pkn2], [ys["PKN2"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
    ax.text(r_pkn2, ys["PKN2"] + 0.2, f"{r_pkn2:.1f}  (Results; Supp. Data 6, 7)", va="bottom",
            ha="center", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
    ax.plot([r_pkn3], [ys["PKN3"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
    ax.text(r_pkn3, ys["PKN3"] + 0.2, f"{r_pkn3:.3f}  (Supp. Data 6, 7 only)", va="bottom", ha="center",
            fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
    # legend
    from matplotlib.lines import Line2D
    h = [Rectangle((0, 0), 1, 1, fc=ORANGE_T, ec=ORANGE, hatch="///", lw=1.5),
         Line2D([], [], marker="o", ls="", ms=10, color=BLUE, mec=BG, mew=2),
         Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=BLUE, mew=2)]
    ax.legend(h, ["Abstract: direction only", "Ratio, protein-specific peptides",
                  "Ratio, shared peptide"],
              loc="lower center", bbox_to_anchor=(0.5, 1.07), ncol=3, frameon=False, fontsize=10,
              handlelength=1.4, columnspacing=1.2)
    save(fig, path)


# ---------------------------------------------------------------- D: mention counts
def fig_D(path):
    fig, ax = plt.subplots(figsize=(W, 3.0))
    fig.subplots_adjust(left=0.10, right=0.97, top=0.82, bottom=0.17)
    names = ["PKN1", "PKN2", "PKN3", "PKN4"]
    for i, n in enumerate(names):
        y = len(names) - 1 - i
        a, b = counts[n]
        ax.barh(y + 0.19, a, height=0.34, color=ORANGE, edgecolor=BG, lw=2)
        ax.barh(y - 0.19, b, height=0.34, color=BLUE, edgecolor=BG, lw=2)
        ax.text(a + 0.2, y + 0.19, str(a), va="center", fontsize=10)
        ax.text(b + 0.2, y - 0.19, str(b) + ("   (named as reduced in abstract)" if n == "PKN4" else ""),
                va="center", fontsize=10, color=INK)
    ax.set_yticks(range(4)); ax.set_yticklabels(names[::-1], fontsize=11, color=INK)
    ax.set_xlim(0, 15); ax.set_xlabel("Number of mentions in the archived article")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ["top", "right", "left"]:
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h = [Rectangle((0, 0), 1, 1, fc=ORANGE), Rectangle((0, 0), 1, 1, fc=BLUE)]
    ax.legend(h, ["Abstract (where the claim is made)", "Rest of article (Results, legends, Methods)"],
              loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, frameon=False, fontsize=10)
    save(fig, path)


# ---------------------------------------------------------------- E: SD7 peptide ratios
def fig_E(path):
    fig, ax = plt.subplots(figsize=(W, 3.0))
    fig.subplots_adjust(left=0.31, right=0.97, top=0.80, bottom=0.18)
    rows = [("PKN2 (Q16513)\nunique peptides (3)", r_pkn2, u2),
            ("shared peptide IADFGLCK\n(PKN1 node's only peptide)", r_sh, s),
            ("PKN3 (Q6P5Z2)\nunique peptides (3)", r_pkn3, u1)]
    ax.set_xscale("log", base=2); ax.set_xlim(1 / 8, 8); ax.set_ylim(-0.4, 2.75)
    ax.set_xticks([1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8])
    ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ["top", "right", "left"]:
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.axvline(1, color=INK2, lw=1)
    ax.text(1, 2.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
    for i, (lab, r, l2) in enumerate(rows):
        y = 2 - i
        ax.plot([1, r], [y, y], color=BLUE, lw=2)
        ax.plot([r], [y], "o", ms=10, color=BLUE, mec=BG, mew=2)
        ax.text(r, y + 0.2, f"{r:.3f}  (log2 {l2:.3f})", va="bottom", ha="center", fontsize=10,
                bbox=dict(fc=BG, ec="none", pad=1))
    ax.set_yticks([2, 1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
    ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
    ax.set_title("Every PKN peptide ratio in Supplementary Data 6 and 7",
                 fontsize=10, color=INK2, loc="left", pad=20)
    save(fig, path)


print("\n=== 4. Figures")
fig_A(os.path.join(CAND, "fig_A_text_panels"))
fig_B(os.path.join(CAND, "fig_B_evidence_matrix"))
fig_C(os.path.join(CAND, "fig_C_direction_plot"))
fig_D(os.path.join(CAND, "fig_D_mention_counts"))
fig_E(os.path.join(CAND, "fig_E_peptide_ratios"))
# The figures used in writeup.md: fig1 = candidate A, fig2 = candidate C (until a design is chosen)
for src, dst in [("fig_A_text_panels", "fig1"), ("fig_C_direction_plot", "fig2")]:
    for ext in (".svg", ".png"):
        shutil.copyfile(os.path.join(CAND, src + ext), os.path.join(HERE, dst + ext))
print("  copied candidates A -> fig1, C -> fig2")

Abstract says host genetics explains almost half of meningitis susceptibility; the paper estimates 29%

Nat Commun (2019) · doi:10.1038/s41467-019-09976-3 · PMC6520353

Summary

The abstract states that human genetic variation "explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity". Table 4 gives 0.25 (GCTA) and 0.29 (LDAK) for susceptibility and 0.29 (GCTA) and 0.49 (LDAK) for severity; the Results and the Discussion quote only the LDAK values, as 29% ± 7% for susceptibility and 49% ± 14% for severity (the Discussion gives the severity error as ± 9%). No susceptibility estimate in the paper is near one half: 0.5 lies outside both recomputed susceptibility 95% confidence intervals. The abstract's two magnitudes match the body's headline values only if the phenotypes are swapped. The abstract also implies that host genetics matters more for susceptibility than for severity, while the paper's headline estimates rank them the other way. The significance of either estimate is unaffected, and so are the paper's other findings.

What the paper reports

Abstract:

In a genome-wide association study of human and pathogen we show that human variation explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity, identifying variants in CCDC33 associated with susceptibility.

Results, "Human genetics associates with susceptibility and severity", paragraph 3:

We then used two standard methods to calculate SNP heritability, which both showed that variation in host genetics explains 29% ± 7% of the observed variation in pneumococcal meningitis susceptibility and 49% ± 14% of the variation in meningitis severity (Table 4).

Discussion, paragraph 2:

Host genetics explained 29% ± 7% of variation in susceptibility to meningitis.

We found no evidence for pathogen genetics affecting the severity of disease, whereas human genetics explained 49% ± 9% of this variation.

Table 4, "Human SNP heritability (h2SNP) of meningitis in the MeninGene cohort" (columns: Phenotype | Method | Heritability | Error | p-value):

Susceptibility (pneumococcal) | GCTA | 0.25 | 0.05 | 2.4 × 10–6

Susceptibility (pneumococcal) | LDAK | 0.29 | 0.07 | 3.9 × 10–6

Severity (any species) | GCTA | 0.29 | 0.11 | 2.8 × 10–5

Severity (any species) | LDAK | 0.49 | 0.14 | 1.4 × 10–4

Table 4 footer:

Heritabilities are shown on the liability scale (adjusted for population prevalence and case ascertainment ratio).

What the paper's own numbers show

Method. The abstract gives words, not numbers, so I read "almost half" as about 0.50 and "one-third" as 0.333. I compared each with both Table 4 estimates for the same phenotype. I built 95% confidence intervals as estimate ± 1.96 × the Table 4 "Error" column, treating that column as a standard error, and checked whether each interval contains 0.5 or 1/3. I also checked that the Results and Discussion percentages match the Table 4 rows. Finally I tested whether the two LDAK estimates differ from each other, with a z-test that treats them as independent. That is an approximation: the two phenotypes share control samples, so the estimates are not strictly independent. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Host h² for susceptibility Abstract "almost half" (≈ 0.50) Table 4 gives 0.25 (GCTA) and 0.29 (LDAK). The nearest is 0.21 below 0.50
Host h² for severity Abstract "one-third" (≈ 0.33) Table 4 gives 0.29 (GCTA, 0.04 away) and 0.49 (LDAK, 0.16 away). The text's headline value is 0.49
Susceptibility, GCTA Table 4 0.25 ± 0.05 95% CI 0.152–0.348, which excludes 0.5
Susceptibility, LDAK Table 4; Results p3; Discussion p2 0.29 ± 0.07; "29% ± 7%" Text matches the table. 95% CI 0.153–0.427, which excludes 0.5
Severity, GCTA Table 4 0.29 ± 0.11 95% CI 0.074–0.506
Severity, LDAK Table 4; Results p3 0.49 ± 0.14; "49% ± 14%" Text matches the table. 95% CI 0.216–0.764
Abstract reading vs headline value, susceptibility Abstract vs Results p3 ≈ 0.50 vs 0.29 21.0 percentage points higher, 1.72 times the paper's estimate
Abstract reading vs headline value, severity Abstract vs Results p3 ≈ 0.33 vs 0.49 15.7 percentage points lower
Value nearest to "almost half" anywhere in Table 4 Table 4 — Severity, LDAK, 0.49 (0.01 from 0.50)
Severity minus susceptibility, LDAK Table 4 0.49 − 0.29 Difference 0.20, SE 0.157, z = 1.28, two-sided p = 0.20 (GCTA: 0.04, z = 0.33, p = 0.74)
Severity error Discussion p2 "49% ± 9%" Table 4 and the Results give ± 14%. With ± 9% the interval would be 0.314–0.666

The arithmetic in the body is internally consistent: the Results and Discussion quote the LDAK rows of Table 4. The inconsistency is between the abstract and everything else. For susceptibility no estimator in the paper gives a value near one half. For severity, "one-third" is close to the GCTA estimate (0.29), but the paper presents the LDAK estimate (0.49) as its headline severity figure in both the Results and the Discussion. Read together, the abstract's two magnitudes match the body's headline values (29% and 49%) with the phenotypes swapped.

Figure

The abstract's wording and the Results/Table 4 headline heritabilities, side by side

Figure 1. On the left is the abstract sentence. On the right is the Results sentence that cites Table 4 (verbatim excerpts). Below each are the values each passage gives for the same two phenotypes. Orange diamonds mark the abstract's words read as fractions. Blue dots mark the Table 4 LDAK estimates quoted in the Results. For susceptibility, the abstract's "almost half" (≈ 0.50) conflicts with 0.29 ± 0.07, and 0.5 is outside both recomputed susceptibility 95% confidence intervals. For severity, the abstract's "one-third" (≈ 0.33) sits below the headline 0.49 ± 0.14, although it is close to the GCTA estimate of 0.29. All values are host SNP heritability on the liability scale.

Table 4 heritability estimates with recomputed 95% confidence intervals, and the abstract's wording, on one axis

Figure 2. All four Table 4 estimates (blue dots) with 95% confidence intervals recomputed as ± 1.96 × the printed error, on one heritability axis that starts at zero (no heritability). The orange vertical marks show where the abstract's words fall for each phenotype. The susceptibility mark at 0.5 lies to the right of both susceptibility intervals. The severity mark at 1/3 lies inside both severity intervals. The intervals are wide, so the severity wording is not ruled out statistically, but it does not match the value the paper reports as its headline severity estimate.

Why it matters

The abstract is the part of the paper most readers see, and the host-heritability sentence is its main quantitative claim about human genetics. As written it overstates the host contribution to susceptibility by about 20 percentage points (≈ 0.50 against 0.29) and understates the headline severity estimate (≈ 0.33 against 0.49). It also reverses the order of the two phenotypes. The abstract suggests that host genetics matters more for susceptibility, while the paper's headline estimates suggest the opposite. Because the two estimates are not significantly different (z = 1.28, p = 0.20 under an independence approximation), the reversal matters less for the substance than for how the result is described. Anyone who cites the abstract's figures, for example in a review comparing heritability across infectious diseases, would carry forward a susceptibility value the paper does not support.

What the discrepancy does not affect: both host estimates remain significantly above zero (Table 4 p-values 3.9 × 10–6 and 1.4 × 10–4 for LDAK). The pathogen heritability results (70% for invasiveness, 0% for severity) are reported separately and correctly. The CCDC33, UBE2U, pspC and zmpD findings and the vaccine-target discussion are not affected.

Explanations considered

Status

As of 2026-10-23:

Sources: PMC6520353; publisher version; Crossref record; Europe PMC; bioRxiv preprint.

Suggested correction

Bring the abstract into line with Table 4 and the Results, for example:

… we show that human variation explains around 30% of variation in susceptibility to pneumococcal meningitis and almost half of variation in severity, …

If the authors prefer to describe severity by the GCTA estimate (0.29), the abstract should say so and give the same order of magnitude for both phenotypes ("around 30%"). In that case the Results and Discussion, which present 49% as the severity figure, would also need to change. Separately, the Discussion's "49% ± 9%" should read "49% ± 14%" to match Table 4 and the Results.

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC6520353 and regenerate the figures.

Run:  python recompute.py.txt
All inputs are copied from the article as archived in PubMed Central (PMC6520353):
Abstract, Results ("Human genetics associates with susceptibility and severity", paragraph 3),
Discussion paragraph 2 and Table 4 (paragraphs counted from 1). Outputs are written next to this file:
fig1.svg/png, fig2.svg/png and candidates/design_1..5.svg/png (five figure designs).
"""
import math
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from scipy.stats import norm

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs
# Table 4: Human SNP heritability, liability scale (estimate, error, printed p)
T4 = {
    ("Susceptibility", "GCTA"): (0.25, 0.05, "2.4e-6"),
    ("Susceptibility", "LDAK"): (0.29, 0.07, "3.9e-6"),
    ("Severity", "GCTA"): (0.29, 0.11, "2.8e-5"),
    ("Severity", "LDAK"): (0.49, 0.14, "1.4e-4"),
}
# Abstract wording, read as the fractions the words denote
ABSTRACT = {"Susceptibility": ("almost half", 0.5), "Severity": ("one-third", 1 / 3)}
# Results paragraph 3 and Discussion paragraph 2 (percent, +/- percent)
RESULTS = {"Susceptibility": (29, 7), "Severity": (49, 14)}
DISCUSSION = {"Susceptibility": (29, 7), "Severity": (49, 9)}
Z = norm.ppf(0.975)

# ---------------------------------------------------------------- numbers
print("== Table 4 estimates with 95% CI (estimate +/- 1.96 x error)")
ci = {}
for (ph, m), (h, se, p) in T4.items():
    lo, hi = h - Z * se, h + Z * se
    ci[(ph, m)] = (lo, hi)
    print(f"{ph:15s} {m}: {h:.2f} +/- {se:.2f}  95% CI {lo:.3f} to {hi:.3f}  "
          f"contains 0.5: {lo <= 0.5 <= hi}  contains 1/3: {lo <= 1/3 <= hi}")

print("\n== Results and Discussion text vs Table 4 LDAK rows")
for ph in ("Susceptibility", "Severity"):
    h, se, _ = T4[(ph, "LDAK")]
    r, d = RESULTS[ph], DISCUSSION[ph]
    print(f"{ph}: Results {r[0]}% +/- {r[1]}%  Discussion {d[0]}% +/- {d[1]}%  "
          f"Table 4 LDAK {h*100:.0f}% +/- {se*100:.0f}%  "
          f"Results match: {r == (round(h*100), round(se*100))}  "
          f"Discussion match: {d == (round(h*100), round(se*100))}")
h, _, _ = T4[("Severity", "LDAK")]
print(f"Severity CI if the Discussion error 0.09 were used: {h-Z*0.09:.3f} to {h+Z*0.09:.3f}")

print("\n== Distance from each abstract word to each Table 4 estimate")
for ph, (word, val) in ABSTRACT.items():
    print(f"Abstract '{word}' ({val:.3f}) for {ph}:")
    for (ph2, m), (h, se, _) in T4.items():
        print(f"    {ph2:15s} {m}: |{h:.2f} - {val:.3f}| = {abs(h-val):.3f}"
              f"{'   <- same phenotype' if ph2 == ph else ''}")
best = {ph: min((abs(T4[(ph, m)][0] - v), m) for m in ("GCTA", "LDAK"))
        for ph, (_, v) in ABSTRACT.items()}
print("Closest same-phenotype estimate:", {k: (m, round(dist, 3)) for k, (dist, m) in best.items()})

print("\n== Size of the discrepancy (abstract reading vs headline LDAK value)")
for ph, (word, val) in ABSTRACT.items():
    h = T4[(ph, "LDAK")][0]
    print(f"{ph}: abstract {val:.3f} vs {h:.2f}: difference {100*(val-h):+.1f} percentage points, "
          f"ratio {val/h:.2f}")

print("\n== Is severity h2 different from susceptibility h2? (LDAK, treated as independent)")
(h1, s1, _), (h2, s2, _) = T4[("Susceptibility", "LDAK")], T4[("Severity", "LDAK")]
zd = (h2 - h1) / math.hypot(s1, s2)
print(f"difference {h2-h1:.2f}, SE {math.hypot(s1, s2):.4f}, z {zd:.2f}, two-sided p {2*norm.sf(abs(zd)):.2f}")
(g1, gs1, _), (g2, gs2, _) = T4[("Susceptibility", "GCTA")], T4[("Severity", "GCTA")]
zg = (g2 - g1) / math.hypot(gs1, gs2)
print(f"GCTA: difference {g2-g1:.2f}, z {zg:.2f}, two-sided p {2*norm.sf(abs(zg)):.2f}")

# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2


def save(fig, stem):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(f"{stem}.{ext}", bbox_inches="tight", pad_inches=0.15, **kw)
    plt.close(fig)


def axis_clean(ax):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)


# ---- design 1: two text panels joined by annotated connectors
def design_1(stem):
    fig = plt.figure(figsize=(W, 4.4))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100); ax.set_ylim(0, 62); ax.axis("off")
    panels = [
        (1, "Abstract", ORANGE,
         "\u201chuman variation explains almost\nhalf of variation in susceptibility\n"
         "to pneumococcal meningitis and\none-third of variation in severity\u201d",
         [("Susceptibility", "almost half (\u2248 0.50)"), ("Severity", "one-third (\u2248 0.33)")]),
        (61, "Results and Table 4", BLUE,
         "\u201cvariation in host genetics explains\n29% \u00b1 7% of the observed variation\n"
         "in pneumococcal meningitis\nsusceptibility and 49% \u00b1 14% of the\n"
         "variation in meningitis severity\u201d",
         [("Susceptibility", "0.29 \u00b1 0.07"), ("Severity", "0.49 \u00b1 0.14")]),
    ]
    rows_y = (18, 7.5)
    for x0, title, col, quote, rows in panels:
        ax.add_patch(FancyBboxPatch((x0, 2), 38, 57, boxstyle="round,pad=0.3,rounding_size=1.5",
                                    fc="white", ec=col, lw=2))
        ax.text(x0 + 2, 56, title, fontsize=11.5, weight="bold", va="top", color=INK)
        ax.text(x0 + 2, 49, quote, fontsize=10, va="top", color=INK2, linespacing=1.35)
        ax.plot([x0 + 2, x0 + 36], [23, 23], color=GRID, lw=1)
        for (ph, val), y in zip(rows, rows_y):
            ax.text(x0 + 4, y + 2.6, ph, fontsize=10.5, va="center", color=INK2)
            ax.text(x0 + 4, y - 1.6, val, fontsize=11.5, va="center", weight="bold", color=INK)
            ax.plot(x0 + 2.3, y - 1.6, "D" if col == ORANGE else "o", ms=8, color=col)
    labels = ("0.50 vs 0.29\n(0.5 is outside\nboth 95% CIs)",
              "0.33 vs 0.49\n(Table 4 GCTA: 0.29)")
    for y, lab in zip(rows_y, labels):
        ax.add_patch(FancyArrowPatch((40.5, y - 1.6), (60, y - 1.6), arrowstyle="<->", mutation_scale=12,
                                     color=INK2, lw=2))
        ax.text(50.2, y + 0.2, lab, ha="center", va="bottom", fontsize=10, color=INK, linespacing=1.2)
    ax.text(50.2, 37, "Same\nphenotypes,\nmagnitudes\nin reverse\norder", ha="center", va="center",
            fontsize=10.5, color=INK, weight="bold")
    ax.text(50.2, -2.5, "Host SNP heritability, liability scale. The abstract gives words, read here as fractions. "
            "LDAK values shown on the right.", ha="center", fontsize=10, color=INK2)
    save(fig, stem)


# ---- design 2: dot-and-interval on one h2 axis
def design_2(stem):
    fig, ax = plt.subplots(figsize=(W, 3.6))
    order = [("Susceptibility", "GCTA"), ("Susceptibility", "LDAK"),
             ("Severity", "GCTA"), ("Severity", "LDAK")]
    ys = [4.6, 3.8, 1.8, 1.0]
    for (ph, m), y in zip(order, ys):
        h, se, _ = T4[(ph, m)]
        lo, hi = ci[(ph, m)]
        ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
        ax.plot(h, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
        ax.text(0.81, y, f"{m} {h:.2f} ({lo:.2f}\u2013{hi:.2f})", va="center",
                fontsize=10, color=INK2)
    for ph, (yb, yt) in (("Susceptibility", (3.4, 5.0)), ("Severity", (0.6, 2.2))):
        word, v = ABSTRACT[ph]
        ax.plot([v, v], [yb, yt], color=ORANGE, lw=2.5)
        ax.plot(v, yt, "D", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.text(v, yt + 0.22, f"Abstract: \u201c{word}\u201d", ha="center", fontsize=10,
                color=INK, weight="bold")
    ax.axvline(0, color=INK2, lw=0.8)
    ax.set_xlim(-0.02, 1.12); ax.set_ylim(0.2, 5.8)
    ax.set_xticks([0, 0.2, 0.4, 0.6, 0.8])
    ax.set_yticks([4.2, 1.4]); ax.set_yticklabels(["Susceptibility\n(pneumococcal)", "Severity\n(any species)"],
                                                  fontsize=10.5, color=INK)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Host SNP heritability, liability scale (h\u00b2 = 0 is no heritability)", x=0.36)
    axis_clean(ax)
    h1 = [plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
                     label="Table 4 estimate, 95% CI recomputed as \u00b11.96 \u00d7 error"),
          plt.Line2D([], [], color=ORANGE, marker="D", ms=8, lw=2.5,
                     label="Abstract wording, read as a fraction")]
    ax.legend(handles=h1, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=1,
              frameon=False, fontsize=10)
    save(fig, stem)


# ---- design 3: dumbbell, abstract vs headline LDAK value per phenotype
def design_3(stem):
    fig, ax = plt.subplots(figsize=(W, 2.9))
    for ph, y in (("Susceptibility", 2), ("Severity", 1)):
        word, v = ABSTRACT[ph]
        h, se, _ = T4[(ph, "LDAK")]
        lo, hi = ci[(ph, "LDAK")]
        ax.plot([lo, hi], [y - 0.12, y - 0.12], color=BLUE, lw=2, alpha=0.45)
        ax.add_patch(FancyArrowPatch((v, y), (h, y), arrowstyle="-|>", mutation_scale=14,
                                     color=INK2, lw=2, shrinkA=6, shrinkB=6))
        ax.plot(v, y, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.plot(h, y, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
        ax.text(v, y + 0.2, f"“{word}”", ha="center", fontsize=10, color=INK)
        ax.text(h, y + 0.2, f"{h:.2f} ± {se:.2f}", ha="center", fontsize=10, color=INK)
        ax.text(0.985, y, f"{100*(v-h):+.0f} pp".replace("-", "\u2212"), ha="right", va="center", fontsize=10.5,
                weight="bold", color=INK)
    ax.set_xlim(0, 1.0); ax.set_ylim(0.5, 2.6)
    ax.set_yticks([2, 1]); ax.set_yticklabels(["Susceptibility", "Severity"], fontsize=10.5, color=INK)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Host SNP heritability, liability scale")
    axis_clean(ax)
    hs = [plt.Line2D([], [], color=ORANGE, marker="D", ms=9, lw=0, label="Abstract (word read as fraction)"),
          plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0, label="Results / Table 4, LDAK"),
          plt.Line2D([], [], color=BLUE, lw=2, alpha=0.45, label="its recomputed 95% CI")]
    ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=3,
              frameon=False, fontsize=10)
    save(fig, stem)


# ---- design 4: slope chart, abstract -> body, lines cross
def design_4(stem):
    fig, ax = plt.subplots(figsize=(W, 3.8))
    for ph, ls in (("Susceptibility", "-"), ("Severity", "--")):
        word, v = ABSTRACT[ph]
        h, se, _ = T4[(ph, "LDAK")]
        ax.plot([0, 1], [v, h], color=INK2, lw=2, ls=ls)
        ax.plot(0, v, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
        ax.plot(1, h, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
        ax.text(-0.04, v, f"{ph}: “{word}”", ha="right", va="center", fontsize=10.5)
        ax.text(1.04, h, f"{ph}: {h:.2f} ± {se:.2f}", ha="left", va="center", fontsize=10.5)
    ax.set_xlim(-1.15, 1.85); ax.set_ylim(0, 0.6)
    ax.set_xticks([0, 1]); ax.set_xticklabels(["Abstract\n(orange: words read as fractions)",
                                               "Results / Table 4, LDAK\n(blue)"], fontsize=10.5, color=INK)
    ax.set_ylabel("Host SNP heritability, liability scale")
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.text(0.5, 0.56, "Lines cross: the abstract ranks the phenotypes\nthe opposite way to the paper's own estimates",
            ha="center", fontsize=10, color=INK2)
    save(fig, stem)


# ---- design 5: one number line, distance from each abstract word to its own estimates
def design_5(stem):
    fig, ax = plt.subplots(figsize=(W, 3.6))
    ax.axhline(0, color=INK2, lw=1, zorder=1)
    for h, lab, ha in ((0.25, "0.25\nSusc. GCTA", "right"),
                       (0.29, "0.29\nSusc. LDAK,\nSev. GCTA", "left"),
                       (0.49, "0.49\nSev. LDAK", "left")):
        ax.plot(h, 0, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=5)  # drawn over the diamonds
        ax.text(h + (0.006 if ha == "left" else -0.006), -0.12, lab, ha=ha, va="top",
                fontsize=10, color=INK, linespacing=1.15)
    for ph, (word, v) in ABSTRACT.items():
        ax.plot(v, 0, "D", ms=13, color=ORANGE, mec=BG, mew=1.5, zorder=4)
        ax.plot([v, v], [0.12, 0.42], color=ORANGE, lw=2)
        ax.text(v + (0.006 if v > 0.4 else -0.006), 0.48, f"Abstract, {ph.lower()}:\n\u201c{word}\u201d",
                ha="left" if v > 0.4 else "right", va="bottom",
                fontsize=10, color=INK)
    for ph, y in (("Susceptibility", -1.45), ("Severity", -2.35)):
        word, v = ABSTRACT[ph]
        dist, m = best[ph]
        h = T4[(ph, m)][0]
        ax.annotate("", xy=(h, y), xytext=(v, y),
                    arrowprops=dict(arrowstyle="|-|", color=INK2, lw=2, mutation_scale=4))
        ax.text(min(v, h), y + 0.13, f"{ph}: \u201c{word}\u201d is {dist:.2f} from the\nnearest "
                f"{ph.lower()} estimate ({m} {h:.2f})", va="bottom", fontsize=10, color=INK2)
    ax.set_xlim(0, 0.72); ax.set_ylim(-2.5, 1.15)
    ax.set_xticks([0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6])
    ax.set_yticks([])
    for sp in ("top", "right", "left"):
        ax.spines[sp].set_visible(False)
    ax.set_xlabel("Host SNP heritability, liability scale  (blue dots: Table 4; orange diamonds: abstract)",
                  x=0.5)
    save(fig, stem)


DESIGNS = [design_1, design_2, design_3, design_4, design_5]
for i, f in enumerate(DESIGNS, 1):
    f(os.path.join(CAND, f"design_{i}"))
design_1(os.path.join(HERE, "fig1"))
design_2(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)

Headline protection p-value cannot be reproduced from the paper's counts using its stated test

Nat Commun (2023) · doi:10.1038/s41467-023-36907-0 · PMC9990026

Summary

The paper reports that after nine repeated low-dose intrarectal SIVmac251 challenges, 4 of 7 vaccinated macaques and 6 of 6 controls were infected, giving a vaccine efficacy (VE) of 43%, and attaches "P < 0.0001" to that result. The paper says this comparison used a one-tailed Fisher's exact test. Applied to the paper's own counts (3/7 vs 0/6 uninfected), that test gives p = 35/286 = 0.122 (two-tailed 0.192), which is above the paper's significance threshold of 0.05. With 7 vs 6 animals, no Fisher (conditional) or permutation test can give a p-value below 1/1716 = 5.8 × 10⁻⁴. Unconditional exact tests (Barnard, Boschloo) cannot go below (7/13)⁷(6/13)⁶ = 1.3 × 10⁻⁴, and a large-sample two-sided log-rank test on any 7-vs-6 outcome cannot go below 1.7 × 10⁻⁴. Each of these floors is above 0.0001, so none of these tests can produce "P < 0.0001" with these group sizes, whatever the outcome. The 43% point estimate is arithmetically correct. The statistical significance claimed for the paper's primary protection result is not supported.

What the paper reports

Results, section "Tonsil vaccination with MVA prevented acquisition in 3/7 animals after 9 repeated low dose IR challenges with SIVmac251", first paragraph:

All 6/6 control animals were infected by the 9th low dose IR challenge whereas only 4/7 vaccinated animals were infected by the 9th challenge demonstrating a VE = 43% (Fig. 2a; P < 0.0001).

Methods, "Data analysis":

Rate of SIV acquisition after each challenge was determined using Kaplan–Meier plots and vaccine efficacy (VE) after the 9th IR challenge was determined using the formula VE = 1 – [(infection rate among vaccinated animals)/(infection rate among control animals)] × 100 and followed by Fisher’s exact test as described by Klasse et al.101.

A P < 0.05 was considered significant.

Figure 2 legend, panel (a):

(a) Study design and percentage of uninfected animals after repeated low dose intrarectal challenge with SIVmac251 in tonsil vaccinated (n = 6) and control (n = 6) groups. Statistical analysis was performed using one-tailed Fishers Exact Test as described in ref. 101 and a P < 0.05 (*) was considered significant.

Figure 2a, Kaplan–Meier panel (text printed inside the plot, read from the published figure image):

VE = 43% P = <0.0001

Methods, "Animals, vaccination, infection, and samples" (group sizes):

(unvaccinated control; n = 6 and vaccinated; n = 7)

Abstract:

43% (3/7) of vaccinated macaques remained uninfected after 9 challenges as compared to the unvaccinated control (0/6) animals.

Introduction, last paragraph:

Our results demonstrated a significant level of protection from acquisition

What the paper's own numbers show

Method. At the 9th challenge the paper's counts form a 2×2 table: vaccinated, 3 uninfected and 4 infected (n = 7); control, 0 uninfected and 6 infected (n = 6). Fisher's exact test conditions on the margins: 3 uninfected animals out of 13, 7 of them vaccinated. The one-tailed p-value in the direction of protection is the probability that all 3 uninfected animals fall in the vaccinated group if vaccination had no effect, which is C(7,3)/C(13,3) = 35/286. The full null distribution of the number of uninfected animals in the vaccinated group (0, 1, 2 or 3 of 3) is 20/286, 105/286, 126/286 and 35/286. The two-tailed p adds the 0-of-3 tail, giving (35 + 20)/286. For the lower bound, the most extreme possible result (all 7 vaccinated uninfected, all 6 controls infected) has one-tailed p = 1/C(13,7) = 1/1716. Its two-tailed p is also 1/1716, because the only opposite-tail table (just 1 of the 7 uninfected animals in the vaccinated group) has probability 7/1716 and is therefore not added. A search over every 7-vs-6 table gives the same minimum two-tailed Fisher p. Unconditional exact tests do not condition on the margins and have a lower floor here: for the most extreme table, Barnard's and Boschloo's tests both give (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴. The Kaplan–Meier curves in Fig. 2a show when each animal became infected, which allows a log-rank (Mantel–Cox) test. Read from the published figure, approximately: controls step from 100% to 83%, 67%, 33%, 17% and 0% at challenges 1, 2, 4 (two animals), 7 and 9. Vaccinated animals step from 100% to 57% at challenge 2 (three animals) and to 43% at challenge 4, and three remain uninfected. The Fisher values were computed with scipy.stats.fisher_exact and checked against the closed-form binomial ratios, the unconditional values with scipy.stats.barnard_exact and boschloo_exact, and the log-rank values with the standard Mantel–Cox formula, all in recompute.py.txt. To find the log-rank floor, the script also enumerates all 57,257,200 possible outcomes, with each animal infected at one of challenges 1 to 9 or uninfected after the 9th.

Quantity Location Reported Recomputed
Vaccine efficacy, 1 − (4/7)/(6/6) Results; Abstract 43% 42.9% (consistent)
p-value for the 9th-challenge acquisition comparison Results, first acquisition paragraph P < 0.0001 0.122 with the stated test
One-tailed Fisher exact, 3/7 vs 0/6 uninfected Test named in Methods and Fig. 2a legend (not given separately) 35/286 = 0.1224
Two-tailed Fisher exact, 3/7 vs 0/6 Same table — 55/286 = 0.1923
One-tailed Fisher exact if n = 6 vaccinated, 3/6 vs 0/6 Fig. 2a legend states n = 6 — 20/220 = 0.0909 (two-tailed 0.1818)
Smallest attainable Fisher or permutation p, 7 vs 6 animals Lower bound for conditional exact tests of this design — one-tailed 1/1716 = 5.83 × 10⁻⁴; two-tailed also 1/1716 = 5.83 × 10⁻⁴
Smallest attainable Barnard or Boschloo p, 7 vs 6 animals Lower bound for unconditional exact tests of this design — (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴ (still above 1 × 10⁻⁴)
Barnard exact, 3/7 vs 0/6 Same table — one-sided 0.106 (two-sided 0.106)
Boschloo exact, 3/7 vs 0/6 Same table — one-sided 0.054 (two-sided 0.108)
Log-rank (Mantel–Cox), Fig. 2a curves as read Fig. 2a Kaplan–Meier panel (approximate readings) — χ² = 1.55, p = 0.21 (one-sided 0.11); χ² = 1.07, p = 0.30 if the first vaccinated step is read at challenge 1
Largest log-rank χ² over every possible 7-vs-6 outcome Lower bound for the large-sample log-rank test — χ² = 14.14, two-sided p = 1.7 × 10⁻⁴
Fewest uninfected vaccinated animals (of 7, vs 0/6) needed for one-tailed p < 0.05 Same test — 4 of 7 (p = 0.0490)
Ratio of recomputed p to reported bound — — 0.1224 / 0.0001 ≈ 1,224

Figure

Reported p-value vs p-values recomputed from the paper's counts, on one log axis

Figure 1. The p-value reported for the 9th-challenge acquisition comparison in the Results and the Fig. 2a panel (orange, "P < 0.0001", drawn as an upper bound with an arrow) and p-values recomputed from the paper's own counts and curves (blue), all on a single log-scaled axis. The dashed line marks the paper's significance threshold, α = 0.05. The one-tailed Fisher exact test named in the Methods and the Fig. 2a legend gives 0.122 for 3/7 vs 0/6 uninfected. The two-tailed version gives 0.192. If the vaccinated group had the n = 6 stated in the Fig. 2a legend, the one-tailed value is 0.091. The unconditional exact tests give one-sided 0.054 (Boschloo) and 0.106 (Barnard). A log-rank test on the Fig. 2a curves, as read approximately, gives 0.21. The shaded band marks values below 1/1716 = 5.8 × 10⁻⁴, the smallest p a Fisher or permutation test can give with 7 vs 6 animals. The dotted line marks 1.3 × 10⁻⁴, the smallest p a Barnard or Boschloo test can give. The reported bound lies below both.

One-tailed Fisher p for every possible number of uninfected vaccinated animals

Figure 2. One-tailed Fisher exact p-values for every possible outcome of the 9th-challenge comparison with 0/6 controls uninfected, as the number of uninfected vaccinated animals runs from 1/7 to 7/7 (gray). The observed 3/7 is shown in blue (p = 0.122). Even the most extreme outcome, all seven vaccinated animals uninfected, gives p = 1/1716 = 5.8 × 10⁻⁴. That value is above the reported "P < 0.0001" (orange line). The design would have needed at least 4 of 7 vaccinated animals uninfected to reach p < 0.05.

Why it matters

The 9th-challenge acquisition comparison is the paper's primary efficacy result. The paper's title says the vaccine "protects macaques", the Introduction describes "a significant level of protection from acquisition", and the Abstract presents the 3/7 vs 0/6 comparison as its main finding. "P < 0.0001" is the only p-value the paper gives for this comparison (Results text and Fig. 2a panel). Recomputed with the stated test, the comparison does not reach the paper's own α of 0.05 (one-tailed p = 0.122). A reader would therefore take a result in 13 animals that does not reach significance to be highly significant.

Some parts of the paper are not affected. The VE point estimate (42.9%, reported as 43%) and the descriptive counts are internally consistent, and so is the report that one vaccinated animal stayed uninfected through 22 challenges. The separate analyses of acute viremia, CD4 counts, cytokines and immune correlates use other tests and other data and are not addressed here. The point concerns only the inferential support for protection from acquisition.

Explanations considered

Status

Suggested correction

The authors should report the p-value for the 9th-challenge acquisition comparison that the stated method gives for the stated counts: a one-tailed Fisher's exact test of 3/7 vs 0/6 uninfected, p = 0.12 (two-tailed p = 0.19). They should state that this comparison is not significant at the paper's α of 0.05. If a different analysis produced the reported value, they should name that test, report the per-animal infection times it used, and confirm that the value is valid for 13 animals. Wording that rests on this comparison should be revised to match, specifically "significant level of protection from acquisition" in the Introduction and the claim that the vaccine "protects" in the title and Abstract. The group size in the Fig. 2a legend (n = 6 vaccinated) should be reconciled with n = 7 in the Methods, Results and Abstract. The post-infection panels use n = 6, which matches the six vaccinated animals eventually infected, but Fig. 2a covers acquisition in all 7.

Recomputation script (python recompute.py.txt)
# Recomputation for PMC9990026 (Nat Commun 2023, doi:10.1038/s41467-023-36907-0)
# Run:  python recompute.py.txt
# Prints every recomputed number in writeup.md and regenerates fig1/fig2 (.svg, .png)
# plus five candidate figure designs (A-E) in ./candidates/ for design review.
# candidates/ is review material only and is not part of the deliverable.
import os, itertools
from math import comb
import numpy as np
from scipy.stats import fisher_exact, hypergeom, barnard_exact, boschloo_exact, chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------- numbers from the paper ----------------
N_VAC, N_CTL = 7, 6            # Methods: control n = 6, vaccinated n = 7
INF_VAC, INF_CTL = 4, 6        # Results: 4/7 vaccinated, 6/6 control infected by 9th challenge
UNINF_VAC, UNINF_CTL = N_VAC - INF_VAC, N_CTL - INF_CTL   # 3 and 0
REPORTED_P = 1e-4              # Results and Fig. 2a panel: "P < 0.0001" (an upper bound)
ALPHA = 0.05                   # Methods and Fig. 2a legend

# Approximate readings of the Fig. 2a Kaplan-Meier steps (challenge number at which
# each animal became infected; 10 = still uninfected after the 9th challenge).
# Controls: 100 -> 83 (ch 1) -> 67 (ch 2) -> 33 (ch 4, two animals) -> 17 (ch 7) -> 0 (ch 9).
# Vaccinated: 100 -> 57 (ch 2, three animals) -> 43 (ch 4); three animals uninfected.
KM_CTL = [1, 2, 4, 4, 7, 9]
KM_VAC = [2, 2, 2, 4, 10, 10, 10]
KM_VAC_ALT = [1, 1, 1, 4, 10, 10, 10]   # alternative reading: first vaccinated step at challenge 1

# ---------------- recomputation ----------------
ve = 1 - (INF_VAC / N_VAC) / (INF_CTL / N_CTL)
print(f"VE = 1 - (4/7)/(6/6) = {ve:.4f} = {100*ve:.1f}%   (paper: 43%)")

table = [[UNINF_VAC, INF_VAC], [UNINF_CTL, INF_CTL]]   # rows vaccinated, control; cols uninfected, infected
_, p1 = fisher_exact(table, alternative="greater")
_, p2 = fisher_exact(table, alternative="two-sided")
p1_exact = comb(7, 3) / comb(13, 3)
print(f"2x2 table (uninfected, infected): vaccinated {table[0]}, control {table[1]}")
print(f"One-tailed Fisher exact p = C(7,3)/C(13,3) = {comb(7,3)}/{comb(13,3)} = {p1_exact:.4f}  (scipy {p1:.4f})")
print(f"Two-tailed Fisher exact p = {p2:.4f}  (= {round(p2*286)}/286)")

# Null distribution: number of the 3 uninfected animals that fall in the vaccinated group
null = {k: hypergeom.pmf(k, 13, 3, 7) for k in range(4)}
for k, v in null.items():
    print(f"  P(k={k} of 3 uninfected in vaccinated group) = {v:.4f} = {round(v*286)}/286")

# Fig. 2a legend gives n = 6 vaccinated: 3/6 vs 0/6
_, p1_6 = fisher_exact([[3, 3], [0, 6]], alternative="greater")
_, p2_6 = fisher_exact([[3, 3], [0, 6]], alternative="two-sided")
print(f"If n=6 vaccinated (3/6 vs 0/6): one-tailed = C(6,3)/C(12,3) = {comb(6,3)}/{comb(12,3)} = {p1_6:.4f}; two-tailed = {p2_6:.4f}")

# Floors of conditional (Fisher) / permutation tests: most extreme table [[7,0],[0,6]]
floor = 1 / comb(13, 7)
_, pf = fisher_exact([[7, 0], [0, 6]], alternative="greater")
_, pf2 = fisher_exact([[7, 0], [0, 6]], alternative="two-sided")
min2 = min(fisher_exact([[a, 7 - a], [b, 6 - b]], alternative="two-sided")[1]
           for a in range(8) for b in range(7))
print(f"Smallest attainable one-tailed Fisher/permutation p, 7 vs 6 = 1/C(13,7) = 1/{comb(13,7)} = {floor:.3e} (scipy {pf:.3e})")
print(f"Two-tailed Fisher p for [[7,0],[0,6]] = {pf2:.3e} (opposite-tail table k=1 has probability "
      f"{hypergeom.pmf(1, 13, 7, 7)*comb(13,7):.0f}/1716 > 1/1716, so not added); "
      f"minimum two-tailed Fisher p over all 7-vs-6 tables = {min2:.3e}")

# Unconditional exact tests (Barnard, Boschloo)
bar_floor = barnard_exact([[7, 0], [0, 6]], alternative="greater").pvalue
bos_floor = boschloo_exact([[7, 0], [0, 6]], alternative="greater").pvalue
closed = (7 / 13) ** 7 * (6 / 13) ** 6
print(f"Barnard floor, one-sided, [[7,0],[0,6]] = {bar_floor:.3e}; Boschloo floor = {bos_floor:.3e}; "
      f"(7/13)^7 (6/13)^6 = {closed:.3e}; above 1e-4? {closed > REPORTED_P}")
bar1 = barnard_exact(table, alternative="greater").pvalue
bar2 = barnard_exact(table, alternative="two-sided").pvalue
bos1 = boschloo_exact(table, alternative="greater").pvalue
bos2 = boschloo_exact(table, alternative="two-sided").pvalue
print(f"Observed 3/7 vs 0/6: Barnard one-sided {bar1:.4f} (two-sided {bar2:.4f}); "
      f"Boschloo one-sided {bos1:.4f} (two-sided {bos2:.4f})")

# Log-rank (Mantel-Cox) test on the Fig. 2a readings
def logrank(tv, tc, n_times=9):
    tv, tc = np.asarray(tv), np.asarray(tc)
    oe = var = 0.0
    for t in range(1, n_times + 1):
        nv, nc = (tv >= t).sum(), (tc >= t).sum(); n = nv + nc
        dv = (tv == t).sum(); d = dv + (tc == t).sum()
        if d == 0 or n < 2:
            continue
        oe += dv - d * nv / n
        var += d * (nv / n) * (nc / n) * (n - d) / (n - 1)
    return oe ** 2 / var, oe / np.sqrt(var)

lr_c, lr_z = logrank(KM_VAC, KM_CTL)
lr_p = chi2.sf(lr_c, 1)
lr_c_alt, _ = logrank(KM_VAC_ALT, KM_CTL)
lr_p_alt = chi2.sf(lr_c_alt, 1)
print(f"Log-rank on Fig. 2a readings: chi2 = {lr_c:.2f}, p = {lr_p:.3f} (one-sided {norm.cdf(lr_z):.3f}); "
      f"alternative reading (first vaccinated step at challenge 1): chi2 = {lr_c_alt:.2f}, p = {lr_p_alt:.3f}")

# Largest asymptotic log-rank chi2 over every possible 7-vs-6 outcome
# (each animal infected at challenge 1..9 or uninfected after 9).
def compositions(n, k):
    rows = []
    for c in itertools.combinations(range(n + k - 1), k - 1):
        prev, row = -1, []
        for x in c:
            row.append(x - prev - 1); prev = x
        row.append(n + k - 2 - prev); rows.append(row)
    return np.array(rows)

SLOTS = 10
CV, CC = compositions(7, SLOTS), compositions(6, SLOTS)
riskV = np.cumsum(CV[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
riskC = np.cumsum(CC[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
Z1E4 = norm.isf(REPORTED_P)          # one-sided normal z for p = 1e-4
first_vac = np.array([next((t for t in range(9) if r[t] > 0), 9) for r in CV])   # slot of first vaccinated infection
lr_max, n_below, n_below_other = 0.0, 0, 0
with np.errstate(divide="ignore", invalid="ignore"):
    for i in range(len(CC)):
        nv, nc = riskV, riskC[i][None, :]; n = nv + nc
        dv = CV[:, :9]; d = dv + CC[i, :9][None, :]
        e = np.where(n > 0, d * nv / n, 0.0)
        v = np.where(n > 1, d * (nv / n) * (nc / n) * (n - d) / (n - 1), 0.0)
        vv = v.sum(1)
        oe = (dv - e).sum(1)
        c2 = np.where(vv > 0, oe ** 2 / vv, 0.0)
        lr_max = max(lr_max, c2.max())
        z = np.where(vv > 0, oe / np.sqrt(vv), 0.0)
        below = z < -Z1E4                    # one-sided p < 1e-4 in the direction of protection
        ctl_before = np.cumsum(CC[i])[np.maximum(first_vac - 1, 0)] * (first_vac > 0)
        all_ctl_first = ctl_before == 6      # every control infected before any vaccinated animal
        n_below += below.sum()
        n_below_other += (below & ~all_ctl_first).sum()
lr_floor = chi2.sf(lr_max, 1)
print(f"Over all {len(CV)*len(CC):,} outcomes: max log-rank chi2 = {lr_max:.2f}, two-sided p = {lr_floor:.2e}, "
      f"one-sided normal p = {norm.sf(np.sqrt(lr_max)):.2e}")
print(f"  outcomes with one-sided log-rank p < 1e-4: {n_below:,}; of these, outcomes in which any vaccinated "
      f"animal was infected before all 6 controls: {n_below_other}")

print(f"Reported bound 1e-4 below Fisher floor? {REPORTED_P < floor}; floor / 1e-4 = {floor/REPORTED_P:.1f}")
print(f"Recomputed one-tailed p / reported bound = {p1/REPORTED_P:.0f}x")

# Ladder: one-tailed p if k of 7 vaccinated stayed uninfected and 0/6 controls did
ladder = {}
for k in range(1, 8):
    _, pk = fisher_exact([[k, 7 - k], [0, 6]], alternative="greater")
    ladder[k] = pk
    print(f"  k={k}/7 vaccinated uninfected vs 0/6: one-tailed p = C(7,{k})/C(13,{k}) = {pk:.4g}")
k_needed = min(k for k, v in ladder.items() if v < ALPHA)
print(f"Fewest uninfected vaccinated animals (of 7, vs 0/6) giving one-tailed p < 0.05: {k_needed} (p = {ladder[k_needed]:.4f})")

# ---------------- style ----------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
    "axes.edgecolor": "#b9b8b3", "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})

def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)

def clean(ax):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)

# ---------- Candidate A: p-values on one log axis ----------
def design_A(path):
    import matplotlib.transforms as mtrans
    fig, ax = plt.subplots(figsize=(7.2, 5.0))
    rows = [
        ("Reported (Results; Fig. 2a)", REPORTED_P, ORANGE, "P < 0.0001"),
        ("Fisher one-tailed, 3/7 vs 0/6\n(test named in Methods, legend)", p1, BLUE, f"{p1:.3f}"),
        ("Fisher two-tailed, 3/7 vs 0/6", p2, BLUE, f"{p2:.3f}"),
        ("Fisher one-tailed, 3/6 vs 0/6\n(n = 6 as in Fig. 2a legend)", p1_6, BLUE, f"{p1_6:.3f}"),
        ("Boschloo one-sided, 3/7 vs 0/6", bos1, BLUE, f"{bos1:.3f}"),
        ("Barnard one-sided, 3/7 vs 0/6", bar1, BLUE, f"{bar1:.3f}"),
        ("Log-rank on Fig. 2a curves\n(approximate readings)", lr_p, BLUE, f"{lr_p:.2f}"),
    ]
    ys = list(range(len(rows)))[::-1]
    ax.axvspan(3e-5, floor, color="#efeeea", zorder=0)
    ax.axvline(bar_floor, color=SEC, lw=1.2, ls=":", zorder=1)
    ax.axvline(ALPHA, color=SEC, lw=1, ls="--", zorder=1)
    ax.text(ALPHA / 1.15, len(rows) - 0.45, "α = 0.05", ha="right", va="center", fontsize=10, color=SEC)
    tr = mtrans.blended_transform_factory(ax.transAxes, ax.transData)
    ax.text(1.03, len(rows) - 0.45, "p", transform=tr, fontsize=10, color=SEC, ha="left", va="center")
    for (lab, v, c, ann), y in zip(rows, ys):
        if c == ORANGE:
            ax.annotate("", xy=(4e-5, y), xytext=(v, y),
                        arrowprops=dict(arrowstyle="-|>", color=c, lw=2, mutation_scale=14))
            ax.plot(v, y, marker="|", ms=14, mew=2.5, color=c)
        else:
            ax.plot(v, y, "o", ms=9, color=c, zorder=3)
        ax.text(1.03, y, ann, transform=tr, fontsize=10, color=c, ha="left", va="center",
                weight="bold" if c == ORANGE else "normal")
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    for t, r in zip(ax.get_yticklabels(), rows):
        t.set_color(r[2] if r[2] == ORANGE else TXT)
    ax.set_xscale("log")
    ax.set_xlim(3e-5, 1)
    ax.set_ylim(-0.6, len(rows) - 0.1)
    ax.set_xlabel("p-value (log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8, which="major")
    clean(ax)
    ax.tick_params(axis="y", length=0)
    fig.tight_layout(rect=(0, 0.13, 0.93, 0.94))
    fig.suptitle("9th-challenge acquisition p-value: reported (orange) vs recomputed (blue)",
                 x=0.015, ha="left", fontsize=11, color=TXT)
    fig.text(0.015, 0.075, f"Shaded band: below 1/1716 = {floor:.1e}, the smallest p a Fisher or permutation test can give.",
             fontsize=9.5, color=SEC)
    fig.text(0.015, 0.03, f"Dotted line: {bar_floor:.1e}, the smallest p a Barnard or Boschloo test can give (7 vs 6 animals).",
             fontsize=9.5, color=SEC)
    save(fig, path)

# ---------- Candidate B: null distribution of the Fisher test ----------
def design_B(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    ks = list(null.keys())
    vals = [null[k] for k in ks]
    cols = [BLUE if k == 3 else "#c9c8c3" for k in ks]
    ax.bar(ks, vals, color=cols, width=0.6, zorder=2)
    for k, v in zip(ks, vals):
        ax.text(k, v + 0.012, f"{round(v*286)}/286\n= {v:.3f}", ha="center", fontsize=10,
                color=BLUE if k == 3 else SEC)
    ax.annotate(f"Observed (3/7 vs 0/6)\none-tailed p = {p1:.3f}", xy=(3, null[3] + 0.115),
                xytext=(2.55, 0.31), fontsize=10, color=BLUE,
                arrowprops=dict(arrowstyle="-", color=BLUE, lw=1))
    # The bars are point probabilities, not p-values, so the reported p-value is not drawn on this
    # axis. It is stated as text: the one-tailed p is the sum of the bars at least as extreme as
    # the observed result, which here is the 3-of-3 bar alone.
    ax.text(-0.42, 0.70, "Reported: P < 0.0001", fontsize=10, color=ORANGE, va="top", weight="bold")
    ax.text(-0.42, 0.655, "The one-tailed p is the sum of bars at least as\nextreme as observed: here the 3-of-3 bar alone,\n"
            f"35/286 = {p1:.3f}, about 1,200 times the reported bound.",
            fontsize=9.5, color=SEC, va="top")
    ax.set_xticks(ks)
    ax.set_xticklabels([f"{k} of 3" for k in ks])
    ax.set_xlabel("Uninfected animals (3 in total) that fall in the vaccinated group of 7")
    ax.set_ylabel("Probability if vaccine had no effect")
    ax.set_ylim(0, 0.75)
    ax.set_yticks([0, 0.1, 0.2, 0.3, 0.4, 0.5])
    ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.set_title("Fisher's exact test on the 9th-challenge counts", fontsize=11, loc="left")
    fig.tight_layout()
    save(fig, path)

# ---------- helpers for text diagrams ----------
def box(ax, x, y, w, h, title, body, edge, tcolor, size=10.5):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.02",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.02, y + h - 0.05, title, fontsize=10, color=SEC, va="top", weight="bold")
    ax.text(x + 0.02, y + h - 0.17, body, fontsize=size, color=tcolor, va="top", linespacing=1.45)

# ---------- Candidate C: text panels, test named vs value reported ----------
def design_C(path):
    fig, ax = plt.subplots(figsize=(7.2, 4.0))
    fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
    ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    box(ax, 0.01, 0.56, 0.46, 0.42, "Methods and Fig. 2a legend",
        "VE after the 9th challenge\n\"followed by Fisher\u2019s exact test\"\n\"one-tailed Fishers Exact Test\"",
        "#b9b8b3", TXT, 10)
    box(ax, 0.53, 0.56, 0.46, 0.42, "Results",
        "4/7 vaccinated vs 6/6 control\ninfected by the 9th challenge\n\"VE = 43% (Fig. 2a; P < 0.0001)\"",
        ORANGE, ORANGE, 10)
    box(ax, 0.12, 0.02, 0.76, 0.34, "Recomputed: the stated test on the stated counts",
        f"One-tailed Fisher, 3/7 vs 0/6 uninfected:\np = C(7,3)/C(13,3) = 35/286 = {p1:.3f}  (two-tailed {p2:.3f})",
        BLUE, BLUE, 10)
    for x0, x1 in ((0.24, 0.36), (0.76, 0.64)):
        ax.add_patch(FancyArrowPatch((x0, 0.555), (x1, 0.365), arrowstyle="-|>",
                                     mutation_scale=14, color=SEC, lw=1.5))
    ax.text(0.5, 0.46, "Orange (reported) vs blue (recomputed):\nmore than 1,000-fold apart",
            ha="center", va="center", fontsize=9.5, color=SEC)
    save(fig, path)

# ---------- Candidate D: calculation flow ----------
def design_D(path):
    fig, ax = plt.subplots(figsize=(7.2, 2.7))
    fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
    ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    steps = [
        ("Counts", "uninfected after\n9 challenges\n(Results):\nvaccinated 3/7\ncontrol 0/6", TXT, "#b9b8b3"),
        ("Stated test", "one-tailed\nFisher exact\n(Methods and\nFig. 2a legend)", TXT, "#b9b8b3"),
        ("Recomputed", f"C(7,3)/C(13,3)\n= 35/286\n= {p1:.3f}", BLUE, BLUE),
        ("Reported", "P < 0.0001", ORANGE, ORANGE),
    ]
    w = 0.215; gap = (0.98 - 4 * w) / 3
    for i, (t, b, c, e) in enumerate(steps):
        x = 0.01 + i * (w + gap)
        box(ax, x, 0.24, w, 0.72, t, b, e, c, 10)
        if i < 2:
            ax.add_patch(FancyArrowPatch((x + w, 0.62), (x + w + gap, 0.62), arrowstyle="-|>",
                                         mutation_scale=12, color=SEC, lw=1.5))
    xn = 0.01 + 3 * w + 2.5 * gap
    ax.text(xn, 0.61, "\u2260", ha="center", va="center", fontsize=20, color=ORANGE)
    ax.text(0.5, 0.08, f"Recomputed {p1:.3f} vs reported < 0.0001: more than 1,000-fold apart", ha="center",
            fontsize=10, color=SEC)
    save(fig, path)

# ---------- Candidate E: ladder of attainable p-values ----------
def design_E(path):
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ks = list(ladder.keys())
    ax.axhspan(3e-5, floor, color="#efeeea", zorder=0)
    ax.axhline(ALPHA, color=SEC, lw=1, ls="--")
    ax.text(7.85, ALPHA * 1.25, "α = 0.05", ha="right", va="bottom", fontsize=10, color=SEC)
    ax.plot(ks, [ladder[k] for k in ks], "-", color="#c9c8c3", lw=2, zorder=1)
    ax.plot(ks, [ladder[k] for k in ks], "o", ms=8, color="#9a9994", zorder=2,
            label="Possible outcomes: k of 7 vaccinated uninfected")
    ax.plot(3, p1, "o", ms=11, color=BLUE, zorder=3, label=f"Observed 3/7, recomputed p = {p1:.3f}")
    ax.axhline(REPORTED_P, color=ORANGE, lw=2, label="Reported: P < 0.0001")
    ax.text(0.8, 2.2e-4, f"Shaded: below the smallest p a Fisher or permutation\ntest can give (all 7 uninfected: 1/1716 = {floor:.1e})",
            fontsize=9.5, color=SEC, va="center")
    ax.set_yscale("log")
    ax.set_ylim(3e-5, 1)
    ax.set_xlim(0.7, 7.9)
    ax.set_xticks(ks)
    ax.set_xticklabels([f"{k}/7" for k in ks])
    ax.set_xlabel("Vaccinated animals uninfected after 9 challenges (controls 0/6)")
    ax.set_ylabel("One-tailed Fisher p (log scale)")
    ax.grid(axis="y", color=GRID, lw=0.8)
    for s_ in ("top", "right"):
        ax.spines[s_].set_visible(False)
    ax.legend(loc="upper right", frameon=False, fontsize=9.5)
    ax.set_title("Every possible outcome of a 7-vs-6 study gives p above 0.0001", fontsize=11, loc="left")
    fig.tight_layout()
    save(fig, path)

design_A(os.path.join(CAND, "A_pvalue_axis"))
design_B(os.path.join(CAND, "B_null_distribution"))
design_C(os.path.join(CAND, "C_text_panels"))
design_D(os.path.join(CAND, "D_calculation_flow"))
design_E(os.path.join(CAND, "E_attainable_ladder"))
# Provisional choice, pending the user's review of candidates A-E: fig1 = A, fig2 = E.
design_A(os.path.join(HERE, "fig1"))
design_E(os.path.join(HERE, "fig2"))
print("Figures written to", HERE)

Abstract's verified-vaccination percentages and effect do not match its own counts or the Results

Nat Med (2023) · doi:10.1038/s41591-023-02670-4 · PMC10719106

Summary

The abstract of this cluster-randomized trial in rural Ghana reports that 36.6% (355 of 1,058) of verified participants in the cash-incentive arms had at least one COVID-19 vaccine dose, compared with 30.3% (439 of 1,544) in the placebo arm, a difference of 6.3% (95% CI 2.4, 10.2; P = 0.001). The counts printed in that sentence give 33.6% and 28.4%. Those are the rates the Results text and Table 2 report for the same verified sample, where the difference is 5.2% (95% CI 1.5, 8.7; P = 0.005). The abstract's 36.6%, 6.3% and 2.4–10.2 match, figure for figure, a spillover estimate that the Results give for a different subgroup. Verified vaccination is the trial's only registry-confirmed outcome, so the abstract overstates the size and precision of its headline objective result, though the effect's direction and statistical significance are unchanged.

What the paper reports

Abstract:

We also verified vaccination status of participants: in the cash treatment arm, 36.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 30.3% (439 of 1,544) for those in the placebo—a difference of 6.3% (95% CI: 2.4, 10.2; P = 0.001).

Results, "Combined low and high cash treatment effect", third paragraph:

In the cash treatment arm (both low and high cash), we verified that 33.6% of individuals had at least one dose of the COVID-19 vaccine as of April 2023 compared to 28.4% of those in the placebo arm (difference from placebo: 5.2%; 95% CI: 1.5, 8.7; P = 0.005).

In relative terms, this represents about an 18% increase over the non-treated placebo vaccination rates.

Results, "Verified vaccination status":

For our six-district sample, we verified that 29.1% of the 3,075 participants received a COVID-19 vaccination

We verified that 28.4% of individuals in the placebo arm were vaccinated (at least one dose)

Table 2, phase IV panel ("Phase IV: verified vaccination status of treated participants (30 April 2022)"; columns: total sample | placebo | health message | low cash | high cash):

Number of participants | 3075 | 1544 | 473 | 548 | 510

Actual vaccination (%) | 29.1 (45.4) | 28.4 (45.1) | 21.6 (41.2) | 40.7 (49.2) | 25.9 (43.8)

Results, "Spillover", second paragraph, which describes Extended Data Table 5 (participants who saw the placebo video, grouped by the treatment assigned to their village):

Participants in placebo-treated village clusters had an average vaccination rate of 26.8%.

Individuals receiving the placebo video in low cash treatment villages had a verified vaccination rate of 38.3% (difference from placebo: 11.5%; 95% CI: 5.2, 17.8; P < 0.001).

Placebo participants in the combined cash treatment also had a verified vaccination rate, 36.6%, that is significantly larger than the placebo treatment village participants (difference from placebo: 6.3%; 95% CI: 2.4, 10.2; P = 0.002).

What the paper's own numbers show

Method. I divided the abstract's numerators by its denominators. I then rebuilt each arm's vaccinated count from the phase IV panel of Table 2 (n × printed %) and checked that the cash-arm count and size equal the abstract's 355 and 1,058. I computed the cash-minus-placebo difference with a naive two-proportion Wald 95% interval and normal-approximation P value. This interval ignores village clustering and serves only to check the printed arithmetic. Finally I set the abstract's figures beside the spillover paragraph's figures. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Cash-arm size Abstract; Table 2 phase IV 1,058; low cash 548, high cash 510 548 + 510 = 1,058, the same sample
Cash-arm vaccinated Abstract; Table 2 phase IV 355; 40.7% of 548, 25.9% of 510 223.0 + 132.1 → 223 + 132 = 355, the same count
Placebo size and vaccinated Abstract; Table 2 phase IV 439 of 1,544; 28.4% of 1,544 1,544 × 28.4% = 438.5 (438 or 439 after rounding), consistent
Total verified sample Table 2 phase IV 3,075; 29.1% 895/3,075 = 29.1% from the rebuilt counts, consistent
Cash-arm rate Abstract 36.6% (355 of 1,058) 355/1,058 = 33.55% → 33.6%
Cash-arm rate Results, combined cash effect 33.6% 33.6%, agrees
Placebo rate Abstract 30.3% (439 of 1,544) 439/1,544 = 28.43% → 28.4%
Placebo rate Results; Table 2 28.4% 28.4%, agrees
Counts the abstract's percentages would need Abstract 36.6% of 1,058; 30.3% of 1,544 387 (not 355); 468 (not 439)
Cash minus placebo Abstract 6.3% (95% CI 2.4, 10.2; P = 0.001) 5.12 pp, naive 95% CI (1.5, 8.7), P = 0.0057
Cash minus placebo Results, combined cash effect 5.2% (95% CI 1.5, 8.7; P = 0.005) 5.12 pp (33.6 − 28.4 = 5.2 from the rounded rates); same interval
Relative increase Results "about 18%" 5.12/28.43 = 18.0%; the abstract's figures imply 6.3/30.3 = 20.8%
Placebo-video participants in cash villages Results, Spillover 36.6%; difference 6.3 (95% CI 2.4, 10.2; P = 0.002) Identical to the abstract's 36.6%, 6.3 and 2.4–10.2; the P values differ (0.002 vs 0.001)
Spillover reference rate Results, Spillover placebo villages 26.8% 38.3 − 11.5 = 26.8 agrees for the low-cash line; 36.6 − 6.3 = 30.3, not 26.8 (36.6 − 26.8 = 9.8)

The abstract's counts and denominators agree with every other part of the paper. Its percentages, difference, interval and P value do not agree with those counts. The abstract's placebo rate, 30.3%, is exactly 36.6 − 6.3, the spillover paragraph's rate minus its difference.

Figure

Abstract versus Results/Table 2 for verified vaccination, with values recomputed from the abstract's counts

Figure 1. The left panel gives the abstract's verified-vaccination figures and the right panel the Results text and Table 2 (phase IV) figures, both as printed (orange). Both panels describe the same 1,058 cash-arm and 1,544 placebo participants. The blue strip shows the rates and difference recomputed from the abstract's own counts, 355/1,058 and 439/1,544. These agree with the Results and Table 2, not with the abstract. The note under the strip records where the abstract's 36.6%, 6.3% and 2.4–10.2 appear in the paper: in the spillover paragraph, for a different subgroup.

Cash-minus-placebo difference in verified vaccination with 95% CIs, as printed and as recomputed

Figure 2. Cash-minus-placebo difference in verified vaccination (percentage points, 95% CI) on one axis, with zero as a thin reference line. Orange marks the values printed in the abstract and in the Results. Blue marks the value recomputed from the abstract's counts with a naive, unclustered two-proportion interval. Gray marks the spillover-subgroup estimate printed in the Results. The recomputed value and interval match the Results. The abstract's estimate and interval are the same as the spillover subgroup's.

Why it matters

A side note for a statistician: the Results interval (1.5, 8.7) is the same as a naive interval that ignores clustering, although the Results report an intracluster correlation of 0.64 for verified vaccination. This write-up compares like with like and makes no claim about which interval is appropriate. The inconsistency described here does not depend on that question.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the abstract, change the verified-vaccination sentence so that it matches its counts and the Results, for example: "in the cash treatment arm, 33.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 28.4% (439 of 1,544) for those in the placebo—a difference of 5.2% (95% CI: 1.5, 8.7; P = 0.005)." The authors may also wish to:

  1. Check the spillover paragraph's combined-cash figures (36.6%; difference 6.3%; 95% CI 2.4, 10.2; P = 0.002) against Extended Data Table 5 and the replication data, since 36.6 − 6.3 does not equal the stated reference rate of 26.8%.
  2. Change "as of April 2023" in the combined-cash sentence to the 30 April 2022 cut-off used in Table 2, if that is the intended date.
Recomputation script (python recompute.py.txt)
"""Recompute the verified-vaccination figures of PMC10719106
(Nat Med 2023, doi:10.1038/s41591-023-02670-4).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

All inputs are copied from the article as archived in PubMed Central:
  Abstract: verified-vaccination sentence (percentages, counts, difference, CI, P)
  Table 2, phase IV (verified vaccination status of treated participants, 30 April 2022):
      number of participants and "Actual vaccination (%)" by arm
  Results > Combined low and high cash treatment effect (33.6% vs 28.4%, 5.2, CI 1.5-8.7)
  Results > Spillover, paragraph on Extended Data Table 5 (26.8%, 38.3%/11.5, 36.6%/6.3)
The naive confidence interval below is an unadjusted two-proportion Wald interval; it
ignores village clustering and is used only to check the arithmetic of the printed values.
"""
import math
import os
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402
from scipy.stats import norm  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
# Abstract
ABS = dict(cash_pct=36.6, cash_k=355, cash_n=1058, plac_pct=30.3, plac_k=439, plac_n=1544,
           diff=6.3, lo=2.4, hi=10.2, p="0.001")
# Results > Combined low and high cash treatment effect
RES = dict(cash_pct=33.6, plac_pct=28.4, diff=5.2, lo=1.5, hi=8.7, p="0.005", rel="about 18%")
# Table 2, phase IV
T2_N = dict(placebo=1544, health=473, low=548, high=510)
T2_PCT = dict(placebo=28.4, health=21.6, low=40.7, high=25.9)
T2_TOTAL_N, T2_TOTAL_PCT = 3075, 29.1
# Results > Spillover (Extended Data Table 5, panel B, as described in the text)
SPILL = dict(placebo_village=26.8, lowcash_placebo=38.3, lowcash_diff=11.5,
             highcash_placebo=26.3, highcash_diff=0.5,
             cash_placebo=36.6, cash_diff=6.3, lo=2.4, hi=10.2, p="0.002")
# Results > Combined cash effect on reported (self-reported) vaccination
SELF_DIFF = 3.7


def pct(k, n):
    return 100.0 * k / n


def wald(k1, n1, k0, n0):
    p1, p0 = k1 / n1, k0 / n0
    d = p1 - p0
    se = math.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
    z = norm.ppf(0.975)
    return 100 * d, 100 * (d - z * se), 100 * (d + z * se), 2 * norm.sf(abs(d) / se)


def main():
    print("== 1. Table 2 phase IV: rebuild the vaccinated counts from n and %")
    k = {a: round(T2_N[a] * T2_PCT[a] / 100) for a in T2_N}
    for a in T2_N:
        print(f"  {a:8s} n={T2_N[a]:5d}  {T2_PCT[a]:.1f}% -> {T2_N[a] * T2_PCT[a] / 100:.1f} "
              f"-> {k[a]} vaccinated")
    tot_n, tot_k = sum(T2_N.values()), sum(k.values())
    print(f"  total    n={tot_n} (Table 2: {T2_TOTAL_N}); vaccinated {tot_k} -> "
          f"{pct(tot_k, tot_n):.1f}% (Table 2: {T2_TOTAL_PCT}%)")
    cash_n = T2_N["low"] + T2_N["high"]
    cash_k = k["low"] + k["high"]
    print(f"  cash arms combined: n = {T2_N['low']} + {T2_N['high']} = {cash_n} "
          f"(abstract: {ABS['cash_n']}); vaccinated = {k['low']} + {k['high']} = {cash_k} "
          f"(abstract: {ABS['cash_k']})")
    print(f"  placebo: n = {T2_N['placebo']} (abstract: {ABS['plac_n']}); "
          f"vaccinated rebuilt = {k['placebo']} (abstract: {ABS['plac_k']}; "
          f"{T2_N['placebo']}*28.35% = {T2_N['placebo'] * .2835:.1f} to "
          f"{T2_N['placebo']}*28.45% = {T2_N['placebo'] * .2845:.1f}, so 438 or 439)")

    print("\n== 2. Abstract: percentages implied by the abstract's own counts")
    c = pct(ABS["cash_k"], ABS["cash_n"])
    p = pct(ABS["plac_k"], ABS["plac_n"])
    print(f"  cash    {ABS['cash_k']}/{ABS['cash_n']} = {c:.2f}% -> {c:.1f}%  "
          f"(abstract prints {ABS['cash_pct']}%; Results prints {RES['cash_pct']}%)")
    print(f"  placebo {ABS['plac_k']}/{ABS['plac_n']} = {p:.2f}% -> {p:.1f}%  "
          f"(abstract prints {ABS['plac_pct']}%; Results/Table 2 print {RES['plac_pct']}%)")
    print(f"  counts the printed percentages would need: {ABS['cash_pct']}% of {ABS['cash_n']} = "
          f"{ABS['cash_pct'] / 100 * ABS['cash_n']:.1f}; {ABS['plac_pct']}% of {ABS['plac_n']} = "
          f"{ABS['plac_pct'] / 100 * ABS['plac_n']:.1f}")
    # interval of counts compatible with each printed percentage after rounding to 0.1
    for lab, pc, n in (("cash", ABS["cash_pct"], ABS["cash_n"]),
                       ("placebo", ABS["plac_pct"], ABS["plac_n"])):
        lo_k = math.ceil((pc - 0.05) / 100 * n)
        hi_k = math.floor((pc + 0.05) / 100 * n)
        print(f"  {lab}: counts that round to {pc}% of {n}: {lo_k} to {hi_k}")

    print("\n== 3. Difference, naive two-proportion CI and P from the counts")
    d, lo, hi, pv = wald(ABS["cash_k"], ABS["cash_n"], ABS["plac_k"], ABS["plac_n"])
    print(f"  difference = {d:.2f} pp, naive 95% CI ({lo:.1f}, {hi:.1f}), P = {pv:.4f}")
    print(f"  abstract: {ABS['diff']} ({ABS['lo']}, {ABS['hi']}), P = {ABS['p']}")
    print(f"  Results:  {RES['diff']} ({RES['lo']}, {RES['hi']}), P = {RES['p']}")
    print(f"  difference of the Results' rounded rates: {RES['cash_pct']} - {RES['plac_pct']} = "
          f"{RES['cash_pct'] - RES['plac_pct']:.1f}")
    print(f"  difference of the abstract's rates: {ABS['cash_pct']} - {ABS['plac_pct']} = "
          f"{ABS['cash_pct'] - ABS['plac_pct']:.1f}")
    print(f"  overstatement in the abstract: {ABS['diff']} - {RES['diff']} = "
          f"{ABS['diff'] - RES['diff']:.1f} pp (vs recomputed: {ABS['diff'] - d:.2f} pp)")

    print("\n== 4. Relative effect")
    print(f"  from the counts: {d:.2f}/{p:.2f} = {100 * d / p:.1f}%  "
          f"(Results: {RES['rel']}; {RES['diff']}/{RES['plac_pct']} = "
          f"{100 * RES['diff'] / RES['plac_pct']:.1f}%)")
    print(f"  implied by the abstract: {ABS['diff']}/{ABS['plac_pct']} = "
          f"{100 * ABS['diff'] / ABS['plac_pct']:.1f}%")
    print(f"  ratio of verified to self-reported effect: abstract {ABS['diff']}/{SELF_DIFF} = "
          f"{ABS['diff'] / SELF_DIFF:.2f}; Results {RES['diff']}/{SELF_DIFF} = "
          f"{RES['diff'] / SELF_DIFF:.2f}")

    print("\n== 5. Spillover paragraph (Extended Data Table 5 as described in Results)")
    print(f"  abstract figures vs spillover figures: 36.6/6.3/(2.4, 10.2) vs "
          f"{SPILL['cash_placebo']}/{SPILL['cash_diff']}/({SPILL['lo']}, {SPILL['hi']}); "
          f"P {ABS['p']} vs {SPILL['p']}")
    print(f"  low-cash placebo check: {SPILL['lowcash_placebo']} - {SPILL['lowcash_diff']} = "
          f"{SPILL['lowcash_placebo'] - SPILL['lowcash_diff']:.1f} "
          f"(stated placebo-village rate {SPILL['placebo_village']})")
    print(f"  combined-cash placebo check: {SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
          f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f} "
          f"(stated placebo-village rate {SPILL['placebo_village']}; "
          f"{SPILL['cash_placebo']} - {SPILL['placebo_village']} = "
          f"{SPILL['cash_placebo'] - SPILL['placebo_village']:.1f})")
    print(f"  abstract placebo rate {ABS['plac_pct']} equals "
          f"{SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
          f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f}")
    print(f"  combined-cash placebo rate must lie between the two cash-village placebo rates: "
          f"{SPILL['highcash_placebo']} <= {SPILL['cash_placebo']} <= {SPILL['lowcash_placebo']}")

    make_fig1(c, p, d, lo, hi)
    make_fig2(d, lo, hi)


# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
    "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


def panel(ax, x, y, w, h, title, lines, edge, wrap=36):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.015, y + h - 0.025, title, ha="left", va="top", fontsize=11, weight="bold",
            color=TXT)
    yy = y + h - 0.10
    for txt, col in lines:
        for ln in textwrap.wrap(txt, wrap):
            ax.text(x + 0.015, yy, ln, ha="left", va="top", fontsize=10, color=col)
            yy -= 0.055
        yy -= 0.02


def make_fig1(c, p, d, lo, hi):
    fig = plt.figure(figsize=(7.2, 4.9))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")

    # Left: abstract (reported, orange)
    panel(ax, 0.015, 0.47, 0.42, 0.50, "Abstract", [
        ("Cash: 36.6% (355 of 1,058)", ORANGE),
        ("Placebo: 30.3% (439 of 1,544)", ORANGE),
        ("Difference 6.3% (95% CI 2.4, 10.2; P = 0.001)", ORANGE),
    ], ORANGE)
    # Right: Results / Table 2 (same sample)
    panel(ax, 0.565, 0.47, 0.42, 0.50, "Results text and Table 2", [
        ("Table 2, phase IV n: placebo 1,544; cash 548 + 510 = 1,058", SEC),
        ("Cash 33.6% vs placebo 28.4%", ORANGE),
        ("Difference 5.2% (95% CI 1.5, 8.7; P = 0.005)", ORANGE),
    ], SEC)

    # Connector between the two panels
    ax.annotate("", xy=(0.565, 0.70), xytext=(0.437, 0.70),
                arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5))
    ax.text(0.501, 0.73, "same n,\nsame\noutcome", ha="center", va="bottom", fontsize=10,
            color=SEC)
    ax.text(0.501, 0.67, "different\n% and\nestimate", ha="center", va="top", fontsize=10,
            color=TXT, weight="bold")

    # Recomputed strip (blue)
    ax.add_patch(FancyBboxPatch((0.015, 0.20), 0.97, 0.20,
                                boxstyle="round,pad=0.008,rounding_size=0.015",
                                fc="white", ec=BLUE, lw=1.5))
    ax.text(0.03, 0.375, "Recomputed from the abstract's own counts", ha="left", va="top",
            fontsize=11, weight="bold")
    ax.text(0.03, 0.305, f"355 / 1,058 = {c:.1f}%      439 / 1,544 = {p:.1f}%", ha="left",
            va="top", fontsize=10.5, color=BLUE)
    ax.text(0.03, 0.25, f"Difference {d:.1f} pp (naive unclustered 95% CI {lo:.1f}, {hi:.1f})",
            ha="left", va="top", fontsize=10.5, color=BLUE)

    note = ("The abstract's 36.6%, 6.3% and CI 2.4, 10.2 also appear in Results > Spillover, "
            "for placebo-video participants in cash villages (a different subgroup; P = 0.002 "
            "there). Orange: values as printed in the paper. Blue: recomputed.")
    for i, ln in enumerate(textwrap.wrap(note, 92)):
        ax.text(0.015, 0.15 - 0.05 * i, ln, ha="left", va="top", fontsize=10, color=SEC)
    save(fig, "fig1")


def make_fig2(d, lo, hi):
    rows = [
        ("Abstract", ABS["diff"], ABS["lo"], ABS["hi"], ORANGE, "o"),
        ("Results text", RES["diff"], RES["lo"], RES["hi"], ORANGE, "s"),
        ("Recomputed, 355/1,058 vs 439/1,544", d, lo, hi, BLUE, "D"),
        ("Spillover subgroup (Results)", SPILL["cash_diff"], SPILL["lo"],
         SPILL["hi"], SEC, "^"),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 3.4))
    for i, (lab, m, l, h, col, mk) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt")
        ax.plot([m], [y], marker=mk, color=col, ms=9, ls="none")
        ax.text(h + 0.25, y, f"{m:.1f} ({l:.1f}, {h:.1f})", va="center", fontsize=10, color=col)
    ax.axvline(0, color=SEC, lw=0.8)
    ax.set_yticks(range(len(rows)))
    ax.set_yticklabels([r[0] for r in rows][::-1])
    ax.set_xlim(-0.5, 14)
    ax.set_ylim(-0.6, len(rows) - 0.4)
    ax.set_xlabel("Cash minus placebo, verified vaccination (pp, 95% CI)")
    ax.set_title("Orange: as printed in the paper. Blue: recomputed. Gray: a different subgroup.",
                 loc="right", fontsize=10, color=SEC)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    fig.tight_layout()
    save(fig, "fig2")


if __name__ == "__main__":
    main()

Abstract says VEGFR1 is downregulated in hyperlipidemic mice; the Results and Discussion report an increase

Nat Med (2012) · doi:10.1038/nm.2759 · PMC3959651

Summary

The Abstract states that "we observed VEGFR1 downregulation in hyperlipidemic mice", but the paper's only mouse experiments (Apoe-null and Ldlr-null versus wild-type C57BL/6 mice, Fig. 3c–e) are reported in the Results as showing "increased endothelial-specific Vegfr1 expression", and the Discussion says Ldlr-null endothelial cells "express elevated amounts of Vegfr1". Of the eight main-text statements that give VEGFR1's direction when ApoB-lipoprotein levels are raised or lowered, seven have VEGFR1 moving with the lipoproteins, as the paper's model requires; the Abstract's mouse clause is the only exception. The data and the main conclusion are unaffected, but the Abstract gives the paper's only mammalian in vivo result the wrong sign, which, read alone, implies that hyperlipidemia lowers an anti-angiogenic decoy receptor. The same wording is in the version of record and the PubMed abstract, and no correction has been issued.

What the paper reports

The Abstract sentence:

Mechanistically, we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response to lipoproteins, and we observed VEGFR1 downregulation in hyperlipidemic mice.

Abstract, second-to-last sentence

The same mouse experiments as described in the body:

In addition, hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased endothelial-specific Vegfr1 expression compared to control WT mice (Fig. 3c–e), confirming conservation of this pathway in higher vertebrates.

Results, VEGFR1 paragraph (last sentence)

The finding that endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1 highlights the potential role of alternative lipoprotein receptors in mediating the endothelial response.

Discussion, paragraph 2

(e) Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.

Fig. 3 legend, panel e

All analyses were performed using mice at 8–38 weeks of age fed with a normal chow diet.

Online Methods, "Mice and immunofluorescence"

The zebrafish and cell-culture results in the same Results paragraph:

VEGFR1 protein concentrations were also strongly downregulated in stl mutants but were substantially upregulated in apoCII morphants, in which endothelial cells face high concentrations of ApoB lipoproteins (Fig. 3b). We found similar effects on VEGFR1 levels in endothelial cells in vitro. Treatment of cultured human umbilical vein endothelial cells (HUVECs) with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations (Fig. 3b).

Results, VEGFR1 paragraph

Every quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC3959651 --grep; recompute.py.txt repeats the check). The Abstract, Results and Discussion sentences and the Fig. 3e P value were also checked against a text dump of the typeset Nature Medicine PDF (see Status).

What the paper's own numbers show

Method. There is nothing to recompute numerically, because the error is in the stated direction of a change. The Fig. 3e group means are shown only as plotted bars, and the text states their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every main-text statement that gives a direction for VEGFR1 (protein, mRNA or reporter) under a condition in which ApoB-lipoprotein levels are raised or lowered relative to a control. To build it, every sentence of the Abstract, Results and Discussion was read. Each statement is copied verbatim and labelled with two signs: the lipoprotein direction of the condition (+1 higher, −1 lower) and the VEGFR1 direction stated (+1 up, −1 down). The product of the two signs is +1 when VEGFR1 moves with lipoproteins and −1 when it moves against them. The script checks each excerpt against the archived text, prints the table below and draws both figures.

Quantity Location Reported Recomputed from the paper's own text
VEGFR1 in hyperlipidemic mice Abstract "downregulation" Lipoproteins up, VEGFR1 down: product −1, the only one
Endothelial Vegfr1, Apoe- or Ldlr-null vs WT mice Results; Fig. 3c–e "increased" Lipoproteins up, VEGFR1 up: +1
Vegfr1, Ldlr-null endothelium Discussion, paragraph 2 "elevated amounts" Lipoproteins up, VEGFR1 up: +1
Vegfr1 mRNA, aortic arch, WT (n = 5) vs Apoe-null (n = 5) Fig. 3e legend P = 4 × 10⁻¹⁴ by z test; direction stated in Results as an increase Not re-tested (see note below)
vegfr1 mRNA, mtp morphants Results; Fig. 3a "lower" Lipoproteins down, VEGFR1 down: +1
VEGFR1 protein, stl mutants Results; Fig. 3b "strongly downregulated" Lipoproteins down, VEGFR1 down: +1
VEGFR1 protein, apoCII morphants Results; Fig. 3b "substantially upregulated" Lipoproteins up, VEGFR1 up: +1
VEGFR1, HUVECs + LDL Results; Fig. 3b "an increase" Lipoproteins up, VEGFR1 up: +1
yfp (vegfr1-promoter reporter) mRNA, mtp knockdown Results; Fig. 3i,j "decrease" (P = 0.0248) Lipoproteins down, VEGFR1 down: +1
Condition statements in which VEGFR1 moves with lipoproteins Main text — 7 of 8
Mouse statements: Abstract vs Results and Discussion Main text — Abstract down; Results up; Discussion up. They disagree.

The first clause of the same Abstract sentence ("we identified downregulation of ... VEGFR1 ... as a key mediator of the endothelial response to lipoproteins") is consistent with the body. It describes the lipoprotein-depleted stl and mtp-morphant zebrafish, in which VEGFR1 falls. The second clause makes a separate claim about the mice, and that claim has the opposite sign to the Results.

Related observation (not part of the main finding). One concluding phrase in the Results also states the relation with the opposite sign to the result it summarizes. After reporting that "mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression (Fig. 3i, j)", the sentence goes on: "providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the transcriptional level". Removing lipoproteins lowered the reporter, so the result shows lipoproteins promoting vegfr1 transcription, not inhibiting it. This wording is also in the typeset PDF. The rest of the paragraph makes clear what is meant, so this phrase is less prominent than the Abstract sentence, but a correction could fix both.

Not evaluated. The Fig. 3e statistics (P = 4 × 10⁻¹⁴ from a two-tailed z test of the ratio of two group means against 1, n = 5 per group, as described in the Online Methods) are not part of this finding and were not re-tested. The group means and standard errors are not given in the text.

Figure

The Abstract's mouse sentence beside the Results and Discussion statements on the same mice

Figure 1. The left panel (orange) shows the Abstract clause that states VEGFR1 goes down in hyperlipidemic mice. The right panel (blue) shows what the Results and Discussion report for the same mice: Apoe-null and Ldlr-null mice compared with WT C57BL/6 on a chow diet (Fig. 3c,d immunostaining; Fig. 3e aortic mRNA, n = 5 per group). The connector marks the conflict: one set of experiments, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.

Lipoprotein direction against VEGFR1 direction for every condition the paper reports

Figure 2. Each marker is one of the eight main-text statements that give a direction for VEGFR1 under a raised or lowered ApoB-lipoprotein level (inclusion rule under Method). The horizontal position gives the lipoprotein level of the condition relative to its control. The vertical position gives the VEGFR1 direction stated. Squares are mouse statements; circles are zebrafish or HUVEC statements. The seven blue statements lie in the two "VEGFR1 with lipoproteins" quadrants, as the paper's model requires. The orange Abstract statement is the only one in a "VEGFR1 against lipoproteins" quadrant. The other two mouse statements, for the same animals, lie in the opposite quadrant. Positions show direction only, not magnitude. Directions and locations come from recompute.py.txt.

Why it matters

Explanations considered

  1. Archiving or conversion artifact. Ruled out. The publisher's abstract page, the typeset PDF and the PubMed/Europe PMC abstract all carry the same wording as the PMC author manuscript. The conflict is in a word ("downregulation" against "increased"), not in a symbol or sign that a conversion could alter.
  2. A different mouse model or treatment. Ruled out. The only mouse experiments are Apoe-null and Ldlr-null mice versus WT C57BL/6 on a normal chow diet (Online Methods; Fig. 3c–e). No mouse experiment lowers lipids (for example with a statin or loss of Mtp) in a way that could produce a decrease.
  3. "Downregulation" describes the mechanism, not the mice. Not an explanation. The first clause of the sentence does use "downregulation" for the lipoprotein-depleted state. But the second clause, "we observed VEGFR1 downregulation in hyperlipidemic mice", is a direct observational claim about the mice, and the mice were hyperlipidemic, not depleted.
  4. A different measure (protein vs mRNA, endothelium vs whole tissue). Ruled out. Immunostaining of aortic endothelium (Fig. 3c,d), aortic-arch mRNA (Fig. 3e), the Results ("increased endothelial-specific Vegfr1 expression") and the Discussion ("elevated amounts of Vegfr1") all give an increase. The paper reports no mouse measure that falls. In the publisher's Fig. 3e bar chart, read by eye, relative Vegfr1 is about 1.0 for WT and about 1.7 for Apoe-null. This approximate reading is not used in any calculation or figure here.
  5. The supplement resolves it. It does not. The Supplementary Text and Figures (Supplementary Figs. 1–7) contain no mouse data. The mouse data that settle the direction are in main Fig. 3c–e.
  6. The paper's model predicts a decrease. It does not. In every zebrafish and HUVEC condition VEGFR1 moves the same way as ApoB lipoproteins (7 of 8 condition statements; Fig. 2), so hyperlipidemia should raise VEGFR1, as the Results say.

None of these explanations reconciles the Abstract clause with the paper's mouse results.

Status

As of 2026-10-23:

Suggested correction

Change the direction in the Abstract so that it matches the Results, Fig. 3c–e and the Discussion:

For consistency, the concluding phrase in the Results paragraph on Fig. 3i,j could also be changed, for example to "providing evidence that ApoB-containing lipoproteins promote vegfr1 expression at the transcriptional level" (or "regulate", as in the Discussion). No change to the data, figures, statistics or conclusions is needed.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC3959651 (Nat Med 2012, doi:10.1038/nm.2759)
# Abstract vs Results / Discussion: which way does VEGFR1 change in hyperlipidemic mice?
#
# Run:  python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of a change stated in words; there is no numeric value
# to recompute. The script therefore:
#   1. holds a hand-curated list of every statement in the main text (Abstract, Results,
#      Discussion) that reports a direction for VEGFR1 under a stated change in ApoB-lipoprotein
#      levels, each copied verbatim from the article as archived in PubMed Central and labelled
#      with the lipoprotein direction of the condition and the VEGFR1 direction stated;
#   2. if run inside the /home/ubuntu/papers repo, re-checks each excerpt verbatim against the
#      archived full text (src/show.py PMC3959651 --full) and confirms its section; skipped
#      elsewhere;
#   3. optionally re-checks the same excerpts in a text dump of the publisher's typeset PDF
#      (set VOR_TXT=/path/to/vor.txt, made with pdftotext); reported as "not run" otherwise;
#   4. multiplies the two directions for each statement (+1: VEGFR1 moves with lipoproteins,
#      -1: against them), tallies them and draws the figures.
# Group sizes and P values are transcribed from the Fig. 3 legend and are not re-tested.

import os, re, subprocess
from collections import OrderedDict

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"

# ------------------------------------------------------------------ statements
# key: (location, section tag in show.py --full, condition, lipoprotein direction of the
#       condition vs its control (+1 higher, -1 lower), VEGFR1 direction stated, verbatim excerpt)
# Section tags: "ABSTRACT" = abstract line, "BODY" = untitled main-text block (Results and
# Discussion are not separately headed in the archived XML), "FIG3" = Fig. 3 legend.
CONDITIONS = OrderedDict([
    ("abstract_mice", ("Abstract, last mechanistic sentence", "ABSTRACT",
        "Hyperlipidemic mice", +1, -1,
        "we observed VEGFR1 downregulation in hyperlipidemic mice")),
    ("results_mice", ("Results, VEGFR1 paragraph (Fig. 3c-e)", "BODY",
        "Apoe- or Ldlr-null mice vs WT", +1, +1,
        "hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased "
        "endothelial-specific Vegfr1 expression compared to control WT mice")),
    ("discussion_ldlr", ("Discussion, paragraph 2", "BODY",
        "Ldlr-null mice", +1, +1,
        "endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1")),
    ("results_mtp", ("Results, VEGFR1 paragraph (Fig. 3a)", "BODY",
        "mtp morphant zebrafish", -1, -1,
        "the vegfr1 mRNA levels were lower in mtp morpholino oligonucleotide-injected embryos "
        "than in uninjected siblings")),
    ("results_stl", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
        "stl mutant zebrafish", -1, -1,
        "VEGFR1 protein concentrations were also strongly downregulated in stl mutants")),
    ("results_apocii", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
        "apoCII morphant zebrafish", +1, +1,
        "were substantially upregulated in apoCII morphants, in which endothelial cells face high "
        "concentrations of ApoB lipoproteins")),
    ("results_huvec", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
        "HUVECs + LDL", +1, +1,
        "with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations")),
    ("results_yfp", ("Results, transcription paragraph (Fig. 3i,j)", "BODY",
        "mtp knockdown, flt1:YFP reporter", -1, -1,
        "mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression")),
])

# General statements of the lipoprotein -> VEGFR1 relation (no single condition).
# key: (location, section tag, implied sign of the relation, verbatim excerpt)
RELATIONS = OrderedDict([
    ("results_rescue_logic", ("Results, rescue paragraph (Fig. 3f-h)", "BODY", +1,
        "if lipoprotein depletion promotes angiogenesis by downregulation of VEGFR1")),
    ("results_yfp_conclusion", ("Results, transcription paragraph (Fig. 3i,j)", "BODY", -1,
        "providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the "
        "transcriptional level")),
    ("discussion_regulated", ("Discussion, paragraph 2", "BODY", 0,
        "expression of VEGFR1 is regulated in vivo by the availability of ApoB-containing "
        "lipoproteins")),
    ("abstract_mediator", ("Abstract, same sentence, first clause", "ABSTRACT", 0,
        "we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), "
        "which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response "
        "to lipoproteins")),
])

# Fig. 3 legend, panel e, and Online Methods (transcribed; not re-tested)
FIG3E = {"n_WT": 5, "n_Apoe_null": 5, "P_reported": 4e-14, "test": "two-tailed z test on ratio vs 1"}
FIG3_SNIPPETS = [
    "Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected "
    "from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.",
    "Confocal images of aortic roots in WT C57BL/6 (left, c) and Apoe-null (right, c) littermates "
    "and of abdominal aortas in WT C57BL/6 (left, d) and Ldlr-null (right, d) littermates",
]
METHODS_SNIPPET = ("All analyses were performed using mice at 8–38 weeks of age fed with a normal "
                   "chow diet.")

WORD = {1: "up", -1: "down", 0: "neutral"}


# ------------------------------------------------------------------ verification
def load_archived_text():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        return None
    try:
        out = subprocess.run([py, "src/show.py", "PMC3959651", "--full"], cwd=repo,
                             capture_output=True, text=True, timeout=120).stdout
    except Exception:
        return None
    return out or None


def split_sections(txt):
    abstract, body, fig3, methods, cur = [], [], [], [], None
    for line in txt.splitlines():
        if line.startswith("ABSTRACT:"):
            abstract.append(line); cur = None; continue
        if line.startswith("## ONLINE METHODS"):
            cur = "M"; continue
        if line.startswith("## "):
            cur = "B"; continue
        if line.startswith("[Figure 3]"):
            cur = "F3"
        elif line.startswith("[Figure"):
            cur = "OF"
        if cur == "B":
            body.append(line)
        elif cur == "F3":
            fig3.append(line)
        elif cur == "M":
            methods.append(line)
    return {"ABSTRACT": "\n".join(abstract), "BODY": "\n".join(body),
            "FIG3": "\n".join(fig3), "METHODS": "\n".join(methods)}


def verify(txt):
    secs = split_sections(txt)
    ok = True
    print("Verbatim check against the archived PMC full text (src/show.py PMC3959651 --full):")
    items = [(k, v[1], v[5]) for k, v in CONDITIONS.items()] + \
            [(k, v[1], v[3]) for k, v in RELATIONS.items()] + \
            [(f"fig3_legend_{i}", "FIG3", s) for i, s in enumerate(FIG3_SNIPPETS)] + \
            [("methods_mice", "METHODS", METHODS_SNIPPET)]
    for key, sec, s in items:
        n_all, n_sec = txt.count(s), secs.get(sec, "").count(s)
        good = n_sec >= 1
        ok &= good
        print(f"  [{'ok' if good else 'MISSING'}] {key:24s} {n_all}x in full text, {n_sec}x in {sec}")
    print("  all verbatim checks passed" if ok else "  SOME CHECKS FAILED")
    return ok


def verify_vor():
    p = os.environ.get("VOR_TXT")
    print("\nPublisher typeset PDF check:", end=" ")
    if not (p and os.path.exists(p)):
        print("not run (set VOR_TXT to a pdftotext dump of the typeset article)")
        return
    t = open(p, encoding="utf-8", errors="replace").read()
    t = re.sub(r"\s+", " ", re.sub(r"-\n", "", t))
    norm = lambda s: re.sub(r"\s+", "", s.replace("−", "-"))
    tn = norm(t)
    print()
    for key in ("abstract_mice", "results_mice", "discussion_ldlr"):
        print(f"  {key:24s} found {tn.count(norm(CONDITIONS[key][5]))}x")
    print(f"  {'results_yfp_conclusion':24s} found {tn.count(norm(RELATIONS['results_yfp_conclusion'][3]))}x")
    print(f"  {'fig3e P = 4 x 10-14':24s} found {tn.count(norm('*P = 4 × 10-14 by z test'))}x")


# ------------------------------------------------------------------ tally
def tally():
    print("\nCondition statements: lipoprotein direction x VEGFR1 direction")
    print(f"  {'key':18s} {'location':46s} {'condition':34s} lipo  VEGFR1  product")
    for k, (loc, _s, cond, l, v, _q) in CONDITIONS.items():
        print(f"  {k:18s} {loc:46s} {cond:34s} {WORD[l]:5s} {WORD[v]:7s} {l * v:+d}")
    prods = {k: v[3] * v[4] for k, v in CONDITIONS.items()}
    pos = [k for k, p in prods.items() if p > 0]
    neg = [k for k, p in prods.items() if p < 0]
    print(f"\n  statements with VEGFR1 moving WITH lipoproteins (+1): {len(pos)} of {len(prods)}")
    print(f"  statements with VEGFR1 moving AGAINST lipoproteins (-1): {len(neg)} of {len(prods)} -> {neg}")
    mice = [k for k, v in CONDITIONS.items() if "mice" in v[2]]
    print(f"  mouse statements: {len(mice)}; directions: "
          + ", ".join(f"{k}={WORD[CONDITIONS[k][4]]}" for k in mice))
    print(f"  Abstract mouse direction agrees with Results mouse direction: "
          f"{CONDITIONS['abstract_mice'][4] == CONDITIONS['results_mice'][4]}")
    print(f"  all non-Abstract condition statements give +1: "
          f"{all(p > 0 for k, p in prods.items() if k != 'abstract_mice')}")
    print("\nGeneral relation statements (implied sign of ApoB lipoproteins -> VEGFR1):")
    for k, (loc, _s, sgn, _q) in RELATIONS.items():
        print(f"  {k:24s} {loc:46s} {WORD[sgn] if sgn == 0 else f'{sgn:+d}'}")
    print("  note: results_yfp_conclusion (-1) is drawn from the result in results_yfp, "
          f"whose own product is {prods['results_yfp']:+d}")
    print(f"\nFig. 3e (transcribed, not re-tested): n = {FIG3E['n_WT']} WT vs "
          f"{FIG3E['n_Apoe_null']} Apoe-null; P = {FIG3E['P_reported']:.0e} by {FIG3E['test']}")
    return prods


# ------------------------------------------------------------------ figures
def style():
    plt.rcParams.update({
        "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
        "axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
        "svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
        "savefig.facecolor": BG,
    })


def panel(ax, x, y, w, h, color, title, sub, body):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
                                lw=2, ec=color, fc="white"))
    ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
    ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
    yy = y + h - 0.165
    for txt, kw in body:
        ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
                fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
        yy -= kw.get("step", 0.12)


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
    plt.close(fig)


def fig1():
    fig = plt.figure(figsize=(7.2, 5.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    ax.text(0.03, 0.98, "VEGFR1 in hyperlipidemic mice: two directions for one experiment",
            fontsize=11.5, fontweight="bold", va="top")
    panel(ax, 0.03, 0.27, 0.41, 0.64, ORANGE, "Abstract", "summary of the mouse result", [
        ("Abstract, last mechanistic sentence:", {"color": SEC, "size": 10, "step": 0.045}),
        ("“…and we observed VEGFR1\ndownregulation in hyperlipidemic\nmice.”", {"step": 0.26}),
        ("Stated: VEGFR1 goes down", {"color": ORANGE, "weight": "bold"}),
    ])
    n_wt, n_ko = FIG3E["n_WT"], FIG3E["n_Apoe_null"]
    panel(ax, 0.56, 0.27, 0.41, 0.64, BLUE, "Results and Discussion",
          "Apoe-null and Ldlr-null vs WT mice", [
        ("Results (Fig. 3c–e):", {"color": SEC, "size": 10, "step": 0.045}),
        ("“hyperlipidemic Apoe- or Ldlr-null\nmice … showed increased\nendothelial-specific "
         "Vegfr1\nexpression compared to control\nWT mice”", {"step": 0.20}),
        ("Discussion:", {"color": SEC, "size": 10, "step": 0.045}),
        ("“endothelial cells of Ldlr-null\nmice express elevated amounts\nof Vegfr1”",
         {"step": 0.125}),
        ("Reported: Vegfr1 goes up", {"color": BLUE, "weight": "bold"}),
    ])
    ax.add_patch(FancyArrowPatch((0.448, 0.59), (0.552, 0.59), arrowstyle="<|-|>",
                                 mutation_scale=16, lw=2, color=TXT))
    ax.text(0.5, 0.62, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
            fontweight="bold", linespacing=1.15)
    ax.text(0.03, 0.215,
            "These are the paper's only mouse experiments: Apoe-null and Ldlr-null mice vs WT\n"
            f"C57BL/6 on chow (Fig. 3c,d staining; Fig. 3e aortic mRNA, n = {n_wt} vs {n_ko}).",
            fontsize=10, va="top", linespacing=1.35)
    ax.text(0.03, 0.11,
            "In zebrafish and HUVECs the paper also reports VEGFR1 rising with lipoproteins\n"
            "(apoCII morphants, LDL) and falling without them (mtp morphants, stl mutants).",
            fontsize=10, color=SEC, va="top", linespacing=1.35)
    save(fig, "fig1")


def fig2(prods):
    fig, ax = plt.subplots(figsize=(7.2, 4.9))
    fig.subplots_adjust(left=0.2, right=0.98, top=0.84, bottom=0.2)
    ax.set_xlim(-1.75, 1.75); ax.set_ylim(-1.75, 1.75)
    for v in (0,):
        ax.axhline(v, color=GRID, lw=1.2, zorder=0); ax.axvline(v, color=GRID, lw=1.2, zorder=0)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.set_xticks([-0.9, 0.9]); ax.set_xticklabels(["lower than control", "higher than control"])
    ax.set_yticks([-0.9, 0.9]); ax.set_yticklabels(["VEGFR1\nlower", "VEGFR1\nhigher"])
    ax.tick_params(length=0, labelsize=10)
    ax.set_xlabel("ApoB-lipoprotein level in the condition", fontsize=10.5)
    # quadrant notes
    ax.text(-1.7, 1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="top")
    ax.text(1.7, 1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="top", ha="right")
    ax.text(-1.7, -1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="bottom")
    ax.text(1.7, -1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="bottom",
            ha="right")
    short = {"abstract_mice": "Abstract:\nhyperlipidemic mice",
             "results_mice": "Results: Apoe-/Ldlr-null mice",
             "discussion_ldlr": "Discussion: Ldlr-null mice",
             "results_apocii": "Results: apoCII morphants",
             "results_huvec": "Results: HUVECs + LDL",
             "results_mtp": "Results: mtp morphants",
             "results_stl": "Results: stl mutants",
             "results_yfp": "Results: mtp KD, YFP reporter"}
    slot = {}
    for k, (_loc, _s, _c, l, v, _q) in CONDITIONS.items():
        i = slot.get((l, v), 0); slot[(l, v)] = i + 1
        x0, y0 = 0.9 * l, 0.9 * v + 0.45 - 0.26 * i if (l, v) != (1, -1) else -0.9
        col = ORANGE if prods[k] < 0 else BLUE
        mk = "s" if "mice" in CONDITIONS[k][2] else "o"
        ax.plot(x0 - 0.72, y0, marker=mk, ms=9, color=col, mec="white", mew=1, zorder=3)
        ax.text(x0 - 0.6, y0, short[k], fontsize=10, va="center",
                color=ORANGE if prods[k] < 0 else TXT,
                fontweight="bold" if prods[k] < 0 else "normal")
    n_pos = sum(p > 0 for p in prods.values()); n = len(prods)
    ax.set_title(f"{n_pos} of {n} condition statements: VEGFR1 moves with lipoproteins (blue);\n"
                 f"the Abstract's mouse statement alone goes against them (orange)",
                 fontsize=10.5, loc="left", color=TXT)
    h = [Line2D([], [], marker="s", color=SEC, ls="none", ms=8, label="mouse statement"),
         Line2D([], [], marker="o", color=SEC, ls="none", ms=8, label="zebrafish or HUVEC")]
    ax.legend(handles=h, loc="upper left", bbox_to_anchor=(-0.27, -0.13), ncol=2, frameon=False,
              fontsize=10)
    save(fig, "fig2")


if __name__ == "__main__":
    txt = load_archived_text()
    if txt:
        verify(txt)
    else:
        print("(archived text not available here; verbatim check skipped)")
    verify_vor()
    prods = tally()
    style()
    fig1()
    fig2(prods)
    print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)

Abstract's baseline cognitive-component estimates differ from the Results and Table 2, changing one component's significance

Neurology (2026) · doi:10.1212/WNL.0000000000214804 · PMC13067375

Summary

The abstract reports cross-sectional estimates from the hurdle ("zero-inflation") model of CDR-SB, adjusted for EYO, for each of the three cognitive components (reserve CogR, demographic CogD, biomarker CogB). There are three estimates for asymptomatic carriers and two for symptomatic carriers. The Results text and Table 2 describe the same model and outcome and agree with each other, but all five of the abstract's estimates or intervals differ from them. The largest difference is for the demographic component in asymptomatic carriers: the abstract gives an odds ratio of 2.60 (95% CI 1.10–6.16), which excludes 1, while the Results and Table 2 give 1.24 (0.76–2.02), which includes 1. The abstract also understates the biomarker odds ratio (5.16 against 7.77). The headline reserve finding keeps its direction and significance in both versions. However, the abstract's statement that the demographic component is associated with being unimpaired at baseline is not supported by the paper's own Table 2.

What the paper reports

Abstract, "Result":

A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), and 155 symptomatic DIAD carriers.

In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively.

Among symptomatic carriers, only the reserve and the biomarker components were significant. A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score (95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.50–0.71).

Results, "Cognitive Reserve, Clinical Status, and Disease Progression", first paragraph (the quotation keeps the original wording, including "multiple"):

For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, 1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 (95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2).

Once symptomatic, only the reserve and the biomarker components were significant. One SD increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and 0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2).

Table 2 ("Relationships of Cognitive Components With Baseline Clinical Diagnosis in DIAD Carriers"), CDR-SB = 0 part (columns: characteristic | odds ratio | 95% CI):

Reserve component (cogR) | 3.66 | 2.11–6.38

Demographic component (cogD) | 1.24 | 0.76–2.02

Biomarker component (cogB) | 7.77 | 3.40–17.80

cogR:EYO | 1.07 | 1.02–1.13

cogD:EYO | 0.99 | 0.95–1.04

cogB:EYO | 1.06 | 0.98–1.14

Table 2, "Cognition decline" part:

Reserve component (cogR) | 0.83 | 0.75–0.92

Demographic component (cogD) | 0.94 | 0.86–1.04

Biomarker component (cogB) | 0.60 | 0.52–0.69

Methods, "Statistical Analysis":

Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive components, and their interaction with EYO.

Results, same section, second paragraph. These are the longitudinal estimates in Table 3, a different model:

One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold (95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase in odds, respectively.

Discussion, fourth paragraph:

Our study also indicates that demographic factors, for example., age, sex, education, and SES, play a significant role in maintaining asymptomatic status in individuals with DIAD.

However, after symptom onset, the demographic component's effect diminishes, with only the reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease progression.

What the paper's own numbers show

Method. I set each estimate in the abstract beside the matching estimate in the Results text and Table 2. The two body sources print identical values. I checked each printed interval for internal consistency: a log-symmetric 95% CI should have its geometric midpoint, √(lower × upper), at the point estimate. From each interval I derived an approximate standard error on the log scale, SE = (ln upper − ln lower)/3.92, and a Wald z = ln(estimate)/SE with its two-sided p. These p-values are reconstructions from rounded printed values; the paper does not print p-values for these rows. I then tested three ways the two sets could both be right: rounding, evaluation of the component effects at a different EYO through the printed EYO interactions, and a mix-up with the longitudinal Table 3. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
CogR, odds of CDR-SB = 0, asymptomatic Abstract 4.06 (1.84–8.95) geometric midpoint 4.06; SE 0.404; z = 3.47, p ≈ 0.0005
CogR, odds of CDR-SB = 0, asymptomatic Results text; Table 2 3.66 (2.11–6.38) midpoint 3.67; SE 0.282; z = 4.60, p < 0.0001; abstract ÷ Table 2 = 1.11
CogD, odds of CDR-SB = 0, asymptomatic Abstract 2.60 (1.10–6.16) midpoint 2.60; SE 0.439; z = 2.17, p ≈ 0.030; CI excludes 1
CogD, odds of CDR-SB = 0, asymptomatic Results text; Table 2 1.24 (0.76–2.02) midpoint 1.24; SE 0.249; z = 0.86, p ≈ 0.39; CI includes 1; abstract ÷ Table 2 = 2.10
CogB, odds of CDR-SB = 0, asymptomatic Abstract 5.16 (2.00–13.33) midpoint 5.16; SE 0.484; z = 3.39, p ≈ 0.0007
CogB, odds of CDR-SB = 0, asymptomatic Results text; Table 2 7.77 (3.40–17.80) midpoint 7.78; SE 0.422; z = 4.85, p < 0.0001; abstract ÷ Table 2 = 0.66
CogR, ratio on CDR-SB, symptomatic Abstract vs Results text; Table 2 0.81 (0.72–0.92) vs 0.83 (0.75–0.92) SE 0.063 vs 0.052; ratio 0.98; both p < 0.001
CogB, ratio on CDR-SB, symptomatic Abstract vs Results text; Table 2 0.60 (0.50–0.71) vs 0.60 (0.52–0.69) same point estimate; SE 0.089 vs 0.072 (different intervals)
Size of the gaps against rounding all five pairs — OR-scale differences 0.40, 1.36, 2.61, 0.02 and 0.00, with interval limits differing in every pair. Two-decimal rounding can move a value by at most 0.01
EYO shift that would turn Table 2 into the abstract value, OR × interactionᵉ Table 2 interaction rows (1.07, 0.99, 1.06) — CogR +1.5 y, CogD −73.7 y, CogB −7.0 y: no single shift fits
Abstract vs longitudinal estimates Table 3; Results paragraph 2 3.29, 1.52, 11.82 (asymptomatic); 0.92, 0.73 (symptomatic) none equals the abstract's 4.06, 2.60, 5.16, 0.81, 0.60
Abstract SE ÷ Table 2 SE derived — 1.43, 1.76, 1.15 (asymptomatic); 1.20, 1.24 (symptomatic): every abstract interval is wider

Each set is internally consistent: every interval centres on its own point estimate. So the abstract values are not a body estimate printed with the wrong interval, or the reverse. The two sets are distinct estimates of what the paper presents as the same quantity.

Figure

Two text panels, "Abstract" and "Results text and Table 2", listing the five estimates side by side and joined by connectors that give the abstract-to-Table 2 ratio; the CogD connector is marked "CI excludes 1 → includes 1"

The left panel lists the abstract's five estimates (95% CI) and the right panel the matching values in the Results text and Table 2, which print identical numbers. Both panels quote the model description each source gives. Each connector gives the ratio of the abstract estimate to the Table 2 estimate, computed in recompute.py.txt. Only the demographic component (CogD) in asymptomatic carriers changes significance: its interval excludes 1 in the abstract and includes 1 in Table 2. The other four pairs differ in point estimate, in interval, or in both, but keep the same direction and significance.

Dot-and-interval plot on a log scale of the five estimates, abstract versus Results text and Table 2, with a reference line at 1

The same ten estimates on one logarithmic axis. Orange marks the abstract and blue the Results text and Table 2, each labelled with its printed value; the thin vertical line marks 1 (no association). The plot shows the sizes of the gaps that the text panels state as ratios. The biomarker odds ratio is a third lower in the abstract. The Table 2 interval for the demographic component crosses 1, while the abstract's does not.

Why it matters

The abstract is the part of the paper most readers see and the part that indexing services reproduce. In it, the demographic component (age, education, sex, race, socioeconomic status and vascular burden score, per the Methods) is presented as significantly associated with being clinically unimpaired at baseline: OR 2.60, 95% CI 1.10–6.16. The paper's own cross-sectional model gives 1.24 (0.76–2.02), an approximate p of 0.39, so the association the abstract reports is not supported by the paper's own analysis. The Discussion says that demographic factors "play a significant role in maintaining asymptomatic status". That sentence matches the abstract's figure. Among the paper's own results, it is supported only by the separate longitudinal estimate in Table 3 (1.52, 1.04–2.23; approximate p 0.031), not by the baseline Table 2 that the abstract describes.

The relative sizes of the components also change. In the abstract, the reserve odds ratio (4.06) is close to the biomarker odds ratio (5.16). In Table 2, the biomarker effect (7.77) is about twice the reserve effect (3.66). A reader comparing the protective weight of cognitive reserve with pathology burden would draw a different picture from each source.

What is not affected: the reserve component is significant in the same direction in every version (abstract 4.06, Table 2 3.66, Table 3 3.29), so the paper's central conclusion, that higher cognitive reserve goes with remaining unimpaired, does not depend on which set is correct. The two symptomatic-carrier estimates differ only slightly and stay significant.

Explanations considered

Status

Suggested correction

The authors should confirm which analysis is final and bring the abstract into line with it. If the Results text and Table 2 are correct, the abstract's Result paragraph should read:

… a 1 SD increase in the reserve component (CogR) was associated with a 3.66-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 2.11–6.38). A 1 SD increase in the biomarker component (CogB) increased the odds of being CDR-SB = 0 by 7.77 (95% CI 3.40–17.80); the demographic component (CogD) was not significantly associated (1.24, 95% CI 0.76–2.02). … A 1 SD increase in CogR was associated with a 0.83-fold reduction in baseline CDR-SB score (95% CI 0.75–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.52–0.69).

The Discussion sentence on demographic factors playing "a significant role in maintaining asymptomatic status" should then name the result that supports it, the longitudinal Table 3 estimate, or else be qualified. If instead the abstract's values are the intended ones, Table 2 and the Results text need correcting, along with an explanation of the sample and adjustment that produced them.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13067375 (Neurology 2026, doi:10.1212/WNL.0000000000214804)
# Abstract vs Results text / Table 2: baseline estimates for the cognitive components.
#
# Run:  python recompute.py.txt                 (prints every number in writeup.md,
#                                                writes fig1.* and fig2.*)
#       python recompute.py.txt --candidates    (also writes five candidate figure designs,
#                                                candidates/cand_A..E.{svg,png})
# Needs numpy, scipy and matplotlib. Figures are written next to this file.
#
# Inputs
#  * Every estimate below is transcribed from the article as archived in PubMed Central
#    (PMC13067375): Abstract (Results paragraph), Results section "Cognitive Reserve,
#    Clinical Status, and Disease Progression", Table 2 and Table 3.
#  * For context only (not part of the paper): the 2024 conference abstract
#    "Cognitive reserve influences symptom onset and longitudinal decline in Dominantly
#    Inherited Alzheimer's Disease", Alzheimer's & Dementia 2024, doi:10.1002/alz.086295,
#    PMC11710562, as given in its Europe PMC full-text XML.
#  * When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
#    archived full text with src/show.py (skipped elsewhere).
#  * Approximate Wald statistics assume each printed 95% CI is symmetric on the log scale:
#    SE(log OR) = (ln hi - ln lo) / (2 x 1.959964), z = ln(OR) / SE, p = 2 x (1 - Phi(|z|)).
#    These are reconstructions from rounded printed values, not values the paper reports.

import math, os, subprocess, sys

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
from scipy.stats import norm

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)

# ------------------------------------------------------------------ quotes
QUOTES = {
    "abs_n": ("Abstract, Result",
        "A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in "
        "the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), "
        "and 155 symptomatic DIAD carriers."),
    "abs_aMC": ("Abstract, Result",
        "In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD "
        "increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds "
        "of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in "
        "the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 "
        "by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively."),
    "abs_sMC": ("Abstract, Result",
        "Among symptomatic carriers, only the reserve and the biomarker components were significant. "
        "A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score "
        "(95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in "
        "CDR-SB (95% CI 0.50–0.71)."),
    "res_aMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
        "For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, "
        "1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 "
        "(95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will "
        "multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2)."),
    "res_sMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
        "Once symptomatic, only the reserve and the biomarker components were significant. One SD "
        "increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and "
        "0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2)."),
    "res_long": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 2",
        "One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold "
        "(95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase "
        "in odds, respectively."),
    "methods": ("Methods, Statistical Analysis",
        "Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive "
        "components, and their interaction with EYO."),
    "disc_demo": ("Discussion, paragraph 4",
        "Our study also indicates that demographic factors, for example., age, sex, education, and SES, "
        "play a significant role in maintaining asymptomatic status in individuals with DIAD."),
    "disc_after": ("Discussion, paragraph 4",
        "However, after symptom onset, the demographic component's effect diminishes, with only the "
        "reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease "
        "progression."),
    "t2_cogR": ("Table 2, CDR-SB = 0 part", "Reserve component (cogR) | 3.66 | 2.11–6.38"),
    "t2_cogD": ("Table 2, CDR-SB = 0 part", "Demographic component (cogD) | 1.24 | 0.76–2.02"),
    "t2_cogB": ("Table 2, CDR-SB = 0 part", "Biomarker component (cogB) | 7.77 | 3.40–17.80"),
    "t2_int": ("Table 2, CDR-SB = 0 part",
        "cogR:EYO | 1.07 | 1.02–1.13\ncogD:EYO | 0.99 | 0.95–1.04\ncogB:EYO | 1.06 | 0.98–1.14"),
    "t2_dec": ("Table 2, Cognition decline part",
        "Reserve component (cogR) | 0.83 | 0.75–0.92\nDemographic component (cogD) | 0.94 | 0.86–1.04\n"
        "Biomarker component (cogB) | 0.60 | 0.52–0.69"),
    "t3_dec": ("Table 3, Longitudinal decline part",
        "Reserve component (cogR) | 0.92 | 0.85–1.01\nDemographic component (cogD) | 0.94 | 0.87–1.03\n"
        "Biomarker component (cogB) | 0.73 | 0.64–0.83"),
}


def check_quotes():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.")
        return
    try:
        full = subprocess.run([py, "src/show.py", "PMC13067375", "--full"], cwd=repo,
                              capture_output=True, text=True, timeout=300).stdout
    except Exception as e:  # pragma: no cover
        print(f"Quote check: could not load archived text ({e}); skipped.")
        return
    nrm = lambda s: " ".join(s.split())
    full_n = nrm(full)
    print("Quote check against the PMC-archived full text (occurrences; 0 = not found):")
    for key, (loc, q) in QUOTES.items():
        print(f"  {key:10s} {full_n.count(nrm(q))}   ({loc})")


# ------------------------------------------------------------------ estimates (OR, lo, hi)
ROWS = ["aMC CogR", "aMC CogD", "aMC CogB", "sMC CogR", "sMC CogB"]
ABSTRACT = {"aMC CogR": (4.06, 1.84, 8.95), "aMC CogD": (2.60, 1.10, 6.16),
            "aMC CogB": (5.16, 2.00, 13.33), "sMC CogR": (0.81, 0.72, 0.92),
            "sMC CogB": (0.60, 0.50, 0.71)}
TABLE2 = {"aMC CogR": (3.66, 2.11, 6.38), "aMC CogD": (1.24, 0.76, 2.02),
          "aMC CogB": (7.77, 3.40, 17.80), "sMC CogR": (0.83, 0.75, 0.92),
          "sMC CogD": (0.94, 0.86, 1.04), "sMC CogB": (0.60, 0.52, 0.69)}
RESULTS_TEXT = {k: TABLE2[k] for k in ROWS}      # the Results prose prints the same values
TABLE3 = {"aMC CogR": (3.29, 2.06, 5.26), "aMC CogD": (1.52, 1.04, 2.23),
          "aMC CogB": (11.82, 5.67, 24.63), "sMC CogR": (0.92, 0.85, 1.01),
          "sMC CogD": (0.94, 0.87, 1.03), "sMC CogB": (0.73, 0.64, 0.83)}
T2_INTERACTION = {"CogR": 1.07, "CogD": 0.99, "CogB": 1.06}   # cogX:EYO, CDR-SB = 0 part
# Context only: 2024 conference abstract (aMC n = 204, sMC n = 115), printed precision
CONF2024 = {"aMC CogR": (4.06, 1.8, 8.9), "aMC CogD": (2.6, 1.1, 6.2),
            "aMC CogB": (5.16, 2.0, 13.3), "sMC CogR": (0.81, 0.72, 0.92),
            "sMC CogB": (0.60, 0.50, 0.71)}
N_PAPER = {"aMC": 284, "sMC": 155}
N_CONF = {"aMC": 204, "sMC": 115}


def se(lo, hi):
    return (math.log(hi) - math.log(lo)) / (2 * Z975)


def wald(o, lo, hi):
    s = se(lo, hi)
    z = math.log(o) / s
    return math.sqrt(lo * hi), s, z, 2 * norm.sf(abs(z))


def report():
    print("\n=== 1. Internal consistency and approximate Wald statistics ===")
    print(f"{'source':14s} {'estimate':9s} {'OR':>6s} {'95% CI':>13s} {'geomean':>8s} "
          f"{'SE':>6s} {'z':>6s} {'p':>8s}  excl.1")
    for name, src in [("Abstract", ABSTRACT), ("Table 2", TABLE2), ("Table 3", TABLE3)]:
        for k, (o, lo, hi) in src.items():
            gm, s, z, p = wald(o, lo, hi)
            print(f"{name:14s} {k:9s} {o:6.2f} {lo:6.2f}-{hi:<6.2f} {gm:8.2f} {s:6.3f} "
                  f"{z:6.2f} {p:8.4f}  {lo > 1 or hi < 1}")

    print("\n=== 2. Abstract vs Results text / Table 2 ===")
    for k in ROWS:
        a, t = ABSTRACT[k], TABLE2[k]
        print(f"{k:9s} abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | Table 2 {t[0]:.2f} "
              f"({t[1]:.2f}-{t[2]:.2f}) | ratio abstract/Table2 = {a[0]/t[0]:.3f} | "
              f"SE ratio = {se(*a[1:]) / se(*t[1:]):.2f} | identical? {a == t}")
    print("Results text equals Table 2 for all five:", all(RESULTS_TEXT[k] == TABLE2[k] for k in ROWS))

    print("\n=== 3. Rounding: largest change the printing precision could cause ===")
    for k in ROWS:
        a, t = ABSTRACT[k], TABLE2[k]
        print(f"{k:9s} |abstract - Table 2| = {abs(a[0]-t[0]):.2f} on the OR scale "
              f"(two-decimal rounding of both: at most 0.01)")

    print("\n=== 4. Re-centring EYO: OR_at_e = OR_Table2 x interaction^e ===")
    for c in ["CogR", "CogD", "CogB"]:
        k = "aMC " + c
        e = math.log(ABSTRACT[k][0] / TABLE2[k][0]) / math.log(T2_INTERACTION[c])
        print(f"{c}: interaction {T2_INTERACTION[c]:.2f}; EYO shift needed to reach abstract value "
              f"= {e:+.1f} y")

    print("\n=== 5. Does the abstract match Table 3 (longitudinal)? ===")
    for k in ROWS:
        print(f"{k:9s} abstract {ABSTRACT[k][0]:.2f} vs Table 3 {TABLE3[k][0]:.2f}: "
              f"equal? {ABSTRACT[k][0] == TABLE3[k][0]}")

    print("\n=== 6. Context: abstract vs 2024 conference abstract, at the conference precision ===")
    def rnd(x, ref):
        d = len(f"{ref}".split(".")[1]) if "." in f"{ref}" else 0
        return round(x + 1e-9, d)
    for k in ROWS:
        a, c = ABSTRACT[k], CONF2024[k]
        same_pt = rnd(a[0], c[0]) == c[0]
        ci_gap = max(abs(a[1] - c[1]), abs(a[2] - c[2]))
        print(f"{k:9s} Neurology abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | conference "
              f"{c[0]} ({c[1]}-{c[2]}) | point estimate same? {same_pt} | largest CI-limit "
              f"difference {ci_gap:.2f} (within 1-decimal printing: {ci_gap <= 0.05 + 1e-9})")
    for g in ["aMC", "sMC"]:
        print(f"{g}: n = {N_CONF[g]} (conference) vs {N_PAPER[g]} (paper); "
              f"sqrt(n ratio) = {math.sqrt(N_PAPER[g] / N_CONF[g]):.2f}")


# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
    "axes.edgecolor": "#b9b8b3",
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "lines.linewidth": 2,
})
LABEL = {"aMC CogR": "Reserve (CogR)", "aMC CogD": "Demographic (CogD)",
         "aMC CogB": "Biomarker (CogB)", "sMC CogR": "Reserve (CogR)", "sMC CogB": "Biomarker (CogB)"}


def save(fig, stem, sub=""):
    d = os.path.join(HERE, sub)
    os.makedirs(d, exist_ok=True)
    fig.savefig(os.path.join(d, stem + ".svg"))
    fig.savefig(os.path.join(d, stem + ".png"), dpi=200)
    plt.close(fig)
    print(f"wrote {os.path.join(sub, stem)}.svg and .png")


def ci_txt(t):
    return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"


def log_axis(ax, lo, hi, ticks):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    ax.set_xticks(ticks)
    ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.minorticks_off()
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


# ------------------------------------------------------------------ A: text panels
def cand_A(stem="cand_A", sub="candidates"):
    W, H = 7.2, 5.8
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
    px = {"L": 0.10, "R": 4.40}; pw = 2.70
    top, bot = 5.68, 0.82

    def panel(x, title, sub, accent, src, model):
        ax.add_patch(FancyBboxPatch((x, bot), pw, top - bot, boxstyle="round,pad=0.02,rounding_size=0.08",
                                    fc="white", ec=accent, lw=1.5))
        ax.text(x + 0.12, top - 0.12, title, fontsize=11.5, fontweight="bold", va="top")
        ax.text(x + 0.12, top - 0.42, sub, fontsize=10, va="top", color=accent)
        ax.text(x + 0.12, top - 0.70, model, fontsize=10, va="top", color=TXT2, fontstyle="italic",
                linespacing=1.15)
        ys = {}
        y = top - 1.62
        for grp, keys in [("Asymptomatic: odds of CDR-SB = 0", ROWS[:3]),
                          ("Symptomatic: ratio on CDR-SB", ROWS[3:])]:
            ax.text(x + 0.12, y + 0.06, grp, fontsize=10, color=TXT2, va="bottom")
            y -= 0.30
            for k in keys:
                ax.text(x + 0.12, y, LABEL[k].split(" (")[1][:-1], fontsize=10, va="center",
                        fontweight="bold")
                ax.text(x + pw - 0.12, y, ci_txt(src[k]), fontsize=10, va="center", ha="right",
                        color=accent, fontweight="bold")
                ys[k] = y
                y -= 0.62
            y -= 0.0
        return ys

    ysL = panel(px["L"], "Abstract", "Result paragraph", ORANGE, ABSTRACT,
                "“…using a zero-inflation model\nadjusted for EYO…”")
    panel(px["R"], "Results text and Table 2", "both print the same values", BLUE, TABLE2,
          "“…using the zero-inflation model\nand after adjusting by EYO…”")
    notes = {}
    for k in ROWS:
        a, t = ABSTRACT[k], TABLE2[k]
        r = a[0] / t[0]
        notes[k] = f"×{r:.2f}" if abs(r - 1) > 1e-9 else "×1.00, CI differs"
    notes["aMC CogD"] = f"×{ABSTRACT['aMC CogD'][0]/TABLE2['aMC CogD'][0]:.2f}"
    x0, x1 = px["L"] + pw + 0.04, px["R"] - 0.04
    for k in ROWS:
        y = ysL[k]
        strong = k == "aMC CogD"
        ax.add_patch(FancyArrowPatch((x0, y), (x1, y), arrowstyle="<|-|>", mutation_scale=11,
                                     lw=2 if strong else 1.2, color=TXT if strong else GRAY))
        ax.text((x0 + x1) / 2, y + 0.06, notes[k], ha="center", va="bottom", fontsize=10,
                color=TXT if strong else TXT2, fontweight="bold" if strong else "normal",
                linespacing=1.1)
        if strong:
            ax.text((x0 + x1) / 2, y - 0.07, "CI excludes 1\n→ includes 1", ha="center", va="top",
                    fontsize=10, color=TXT, fontweight="bold", linespacing=1.05)
    ax.text((x0 + x1) / 2, top - 0.12, "conflict", ha="center", va="top", fontsize=11, fontweight="bold")
    ax.text((x0 + x1) / 2, top - 0.45, "same model\nand outcome;\nall 5 estimates\ndiffer",
            ha="center", va="top", fontsize=10, color=TXT2, linespacing=1.15)
    ax.text(0.12, 0.42,
            "Estimates (95% CI) per 1 SD of each component, as printed. Ratios abstract ÷ Table 2 are\n"
            "computed in recompute.py.txt. Only the demographic component changes significance; the\n"
            "symptomatic CogB estimate (0.60) is the same point value with a different interval.",
            fontsize=10, color=TXT2, va="center", linespacing=1.25)
    save(fig, stem, sub)


# ------------------------------------------------------------------ B: forest plot
def cand_B(stem="cand_B", sub="candidates"):
    fig, ax = plt.subplots(figsize=(7.2, 4.8))
    fig.subplots_adjust(left=0.27, right=0.78, top=0.90, bottom=0.13)
    ypos, labels, y = {}, [], 0
    heads = []
    for grp, keys in [("Asymptomatic carriers\nodds of CDR-SB = 0", ROWS[:3]),
                      ("Symptomatic carriers\nfold change in CDR-SB", ROWS[3:])]:
        heads.append((y, grp)); y += 0.9
        for k in keys:
            ypos[k] = y; y += 1.25
        y += 0.1
    for k in ROWS:
        for src, col, off, name in [(ABSTRACT, ORANGE, -0.17, "Abstract"),
                                    (TABLE2, BLUE, 0.17, "Results text and Table 2")]:
            o, lo, hi = src[k]
            yy = ypos[k] + off
            ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt")
            ax.plot(o, yy, "o", ms=8, color=col, label=name if k == ROWS[0] else None)
            ax.text(1.03, yy, ci_txt(src[k]), transform=ax.get_yaxis_transform(), va="center",
                    fontsize=10, color=col, fontweight="bold")
    ax.axvline(1, color=TXT2, lw=0.8)
    log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
    ax.set_yticks([ypos[k] for k in ROWS]); ax.set_yticklabels([LABEL[k] for k in ROWS], fontsize=10,
                                                               color=TXT)
    for yh, g in heads:
        ax.text(-0.36, yh + 0.1, g, transform=ax.get_yaxis_transform(), fontsize=10, color=TXT2,
                va="center", ha="left", fontweight="bold", linespacing=1.1)
    ax.set_ylim(y - 0.2, -0.6)
    ax.set_xlabel("Odds ratio or fold change per 1 SD (log scale); thin line = 1 (no association)")
    ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
    ax.text(0.42, ypos["aMC CogD"] + 0.62, "Table 2 CogD interval crosses 1", fontsize=10, color=TXT,
            va="center", fontweight="bold")
    save(fig, stem, sub)


# ------------------------------------------------------------------ C: dumbbell (shift)
def cand_C(stem="cand_C", sub="candidates"):
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
    ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
    for k in ROWS:
        a, t = ABSTRACT[k][0], TABLE2[k][0]
        y = ys[k]
        if a != t:
            ax.annotate("", xy=(t, y), xytext=(a, y),
                        arrowprops=dict(arrowstyle="-", color=GRAY, lw=2))
        ax.plot(a, y, "o", ms=9, color=ORANGE, label="Abstract" if k == ROWS[0] else None, zorder=3)
        ax.plot(t, y, "D", ms=8, color=BLUE, label="Results text and Table 2" if k == ROWS[0] else None,
                zorder=3)
        lbl = f"×{a/t:.2f}" if a != t else "same point value; CI differs"
        ax.text(max(a, t) * 1.2, y, lbl, va="center", fontsize=10,
                color=TXT if k == "aMC CogD" else TXT2,
                fontweight="bold" if k == "aMC CogD" else "normal")
    ax.axvline(1, color=TXT2, lw=0.8)
    log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
    ax.set_yticks([ys[k] for k in ROWS])
    ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
                       color=TXT)
    ax.set_ylim(max(ys.values()) + 0.6, -0.6)
    ax.set_xlabel("Point estimate per 1 SD (log scale); thin line = 1")
    ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
    save(fig, stem, sub)


# ------------------------------------------------------------------ D: provenance matrix
def cand_D(stem="cand_D", sub="candidates"):
    W, H = 7.2, 4.3
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
    cols = [("Neurology 2026\nabstract", ABSTRACT, "A"),
            ("2024 conference\nabstract*", CONF2024, "A"),
            ("Neurology 2026\nResults text\nand Table 2", TABLE2, "T"),
            ("Neurology 2026\nTable 3\n(longitudinal)", TABLE3, "O")]
    x0, cw, rh, ytop = 1.27, 1.47, 0.42, 3.22
    for j, (h, _, _) in enumerate(cols):
        ax.text(x0 + j * cw + cw / 2, ytop + 0.08, h, ha="center", va="bottom", fontsize=10,
                fontweight="bold", linespacing=1.1)
    fill = {"A": "#fbe1d6", "T": "#d9e7f8", "O": "#f0efec"}
    edge = {"A": ORANGE, "T": BLUE, "O": "#c9c8c3"}
    for i, k in enumerate(ROWS):
        y = ytop - (i + 1) * rh - (0.14 if i >= 3 else 0)
        ax.text(0.10, y + rh / 2, ("Asympt. " if i < 3 else "Sympt. ") + k.split()[1], va="center",
                fontsize=10, fontweight="bold")
        for j, (_, src, grp) in enumerate(cols):
            o, lo, hi = src[k]
            ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
                                   fc=fill[grp], ec=edge[grp], lw=1))
            txt = f"{o:g} ({lo:g}–{hi:g})" if src is CONF2024 else ci_txt(src[k])
            ax.text(x0 + j * cw + cw / 2, y + rh / 2, txt, ha="center", va="center", fontsize=10)
    yl = 0.74
    for xx, grp, t in [(0.10, "A", "matches the Neurology abstract"),
                       (3.30, "T", "the paper's body"), (5.25, "O", "a different model")]:
        ax.add_patch(Rectangle((xx, yl - 0.09), 0.22, 0.18, fc=fill[grp], ec=edge[grp], lw=1))
        ax.text(xx + 0.30, yl, t, va="center", fontsize=10, color=TXT2)
    ax.text(0.10, 0.30, "*Alzheimer's & Dementia 2024, doi:10.1002/alz.086295 (aMC n = 204, sMC n = 115;\n"
            "adjusted by mutation mean AAO). The Neurology paper reports aMC n = 284, sMC n = 155.",
            fontsize=10, color=TXT2, va="center", linespacing=1.2)
    save(fig, stem, sub)


# ------------------------------------------------------------------ E: Wald z
def cand_E(stem="cand_E", sub="candidates"):
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
    ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
    ax.axvspan(-Z975, Z975, color="#efeeea", zorder=0)
    ax.axvline(0, color=TXT2, lw=0.8)
    for k in ROWS:
        y = ys[k]
        for src, col, mk, off, name in [(ABSTRACT, ORANGE, "o", -0.14, "Abstract"),
                                        (TABLE2, BLUE, "D", 0.14, "Results text and Table 2")]:
            _, _, z, p = wald(*src[k])
            ax.plot(z, y + off, mk, ms=8, color=col, label=name if k == ROWS[0] else None)
            ax.text(z + (0.25 if z > 0 else -0.25), y + off, f"p≈{p:.3f}" if p >= 0.001 else "p<0.001",
                    va="center", ha="left" if z > 0 else "right", fontsize=9.5, color=col)
    ax.text(0, -0.45, "|z| < 1.96\n(p > 0.05)", ha="center", va="top", fontsize=10,
            color=TXT2, linespacing=1.05)
    ax.set_xlim(-10.5, 7.5)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_yticks([ys[k] for k in ROWS])
    ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
                       color=TXT)
    ax.set_ylim(max(ys.values()) + 0.6, -1.0)
    ax.set_xlabel("Approximate Wald z = ln(estimate) / SE, SE from the printed 95% CI")
    ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
    save(fig, stem, sub)


if __name__ == "__main__":
    check_quotes()
    report()
    print()
    cand_A("fig1", "")
    cand_B("fig2", "")
    if "--candidates" in sys.argv:
        for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
            f()

Abstract reports 6-month BMI difference versus prednisone as 12-month result, and reuses it for height

Neurology (2026) · doi:10.1212/WNL.0000000000214756 · PMC13225239

Summary

The abstract gives the 12-month least-squares-mean (LSM) difference in BMI Z-score between vamorolone 6 mg/kg/d and prednisone as 0.57 [95% CI 0.24–0.90]. In the paper's Table 3, those three numbers are the 6-month value. The 12-month value is 0.24 (−0.04 to 0.52; p = 0.09), and its confidence interval includes zero. The abstract then gives the same triple, 0.57 [0.24–0.90], as the 12-month height Z-score difference versus prednisone, where Table 3 reports 0.44 (0.25 to 0.64). The deflazacort values in the same two sentences match Table 3 exactly, and the Results text and the Discussion agree with Table 3. So the error is confined to the two prednisone figures in the abstract. Those figures make the 12-month BMI excess on vamorolone look 2.4 times larger than the paper's own estimate, and statistically significant when it is not.

What the paper reports

Abstract, Results (BMI):

Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.29 [0.03–0.56])

Abstract, Results (height), same sentence:

(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.72 [0.53–0.91])

Table 3, "Comparisons of LSM Changes From Baseline Between Treatments at 6 and 12 Months for Anthropometric Outcomes Performed Using Weighted Data". Columns: n; LSM difference; 95% CI; p, at 6 mo and then at 12 mo:

BMI Z-score | Vam 6/Pred | 26/48 | 0.57; 0.24, 0.90 | 0.009 | 26/48 | 0.24; -0.04, 0.52 | 0.09

BMI Z-score | Vam 6/DFZ | 26/54 | 0.32; 0.00, 0.64 | 0.05 | 26/51 | 0.29; 0.03, 0.56 | 0.03

Height Z-score | Vam 6/Pred | 26/48 | 0.25; 0.14, 0.36 | <0.0001 | 26/48 | 0.44; 0.25, 0.64 | <0.0001

Height Z-score | Vam 6/DFZ | 26/51 | 0.50; 0.39, 0.61 | <0.0001 | 26/51 | 0.72; 0.53, 0.91 | <0.0001

Results, "Anthropometric Analyses" (labels 0.57 as the 6-month value):

LSM estimates of BMI Z-scores increased in all treatment groups at 12 months, with a decrease in BMI observed for prednisone at 6 months (prednisone vs vamorolone 6 mg/kg/d at 6 months, BMI Z-score LSM difference 0.57; 95% CI 0.24–0.90; p = 0.009) and a smaller increase for deflazacort compared with vamorolone at 12 months (LSM difference 0.29; 95% CI 0.03–0.56; p = 0.03) (Table 3; Figure 3A).

Discussion, paragraph 4:

To this end, we found that the increase in the BMI Z-score from baseline was greater on vamorolone than prednisone at 6 but not at 12 months.

What the paper's own numbers show

Method: each abstract triple (estimate, lower, upper) was compared for exact equality with all 12 cells of Table 3. Each cell was then checked for internal consistency. The standard error was recovered from the CI width as (upper − lower) / (2 × 1.96), and the two-sided p was recomputed from z = estimate / SE under a normal approximation. This is a check only: the paper's mixed model uses Kenward-Roger degrees of freedom, which give slightly larger p values. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
BMI Z-score, vamorolone 6 vs prednisone, "12-month" Abstract 0.57 [0.24–0.90] Table 3 at 12 mo: 0.24 (−0.04, 0.52). The triple equals only the 6-mo BMI cell
BMI Z-score, vamorolone 6 vs prednisone, 12 mo Table 3 0.24; −0.04, 0.52; p = 0.09 SE 0.143, z 1.68, p = 0.093 (consistent); CI includes 0
BMI Z-score, vamorolone 6 vs prednisone, 6 mo Table 3; Results text 0.57; 0.24, 0.90; p = 0.009 SE 0.168, z 3.39, p ≈ 0.0007 (see note)
Abstract minus Table 3, BMI vs prednisone, 12 mo Abstract vs Table 3 0.57 vs 0.24 Difference 0.33; ratio 2.38
Height Z-score, vamorolone 6 vs prednisone, "12-month" Abstract 0.57 [0.24–0.90] Table 3 at 12 mo: 0.44 (0.25, 0.64), and at 6 mo: 0.25. The triple equals the 6-mo BMI cell
Height Z-score, vamorolone 6 vs prednisone, 12 mo Table 3 0.44; 0.25, 0.64; p < 0.0001 SE 0.099, z 4.42, p ≈ 1e−5 (consistent); abstract overstates by 0.13
BMI Z-score, vamorolone 6 vs deflazacort, 12 mo Abstract; Table 3 0.29 [0.03–0.56]; p = 0.03 Abstract equals Table 3 12-mo cell; p = 0.032 (consistent)
Height Z-score, vamorolone 6 vs deflazacort, 12 mo Abstract; Table 3 0.72 [0.53–0.91]; p < 0.0001 Abstract equals Table 3 12-mo cell; p ≈ 1e−13 (consistent)

Every Table 3 CI midpoint equals its point estimate to rounding (for example 0.240 for 0.24, and 0.445 for 0.44). The table's columns are therefore not misaligned. Of the four comparisons the abstract labels "12-month", the two deflazacort ones match their 12-month cells and the two prednisone ones match neither 12-month cell.

Note on a separate, minor point: for the 6-month BMI cell, the CI 0.24–0.90 implies p ≈ 0.0007 under a normal approximation. With a t reference distribution, it implies p ≤ 0.003 for any df of 10 or more. The reported p = 0.009 would need about 4 degrees of freedom. This affects only the precision of a p value whose result is significant either way, and it is not the subject of this write-up.

Figure

Abstract versus Table 3 for the vamorolone 6 vs prednisone comparisons

Figure 1. The two abstract values labelled "12-month" for vamorolone 6 mg/kg/d versus prednisone (left, orange) set against Table 3 of the same paper (right, blue). Solid arrows join identical numbers: both abstract values (BMI and height) equal the 6-month BMI cell, 0.57; 0.24, 0.90. Dashed arrows lead to the 12-month cells the abstract label refers to: BMI 0.24 (−0.04 to 0.52, p = 0.09) and height 0.44 (0.25 to 0.64). All numbers are quoted from the article.

Dot-and-interval comparison on one axis

Figure 2. The same contrasts drawn on one LSM-difference axis, with a reference line at 0 (no difference). For BMI, the abstract's "12-month" interval (orange) lies exactly on the Table 3 6-month interval (blue diamond). The Table 3 12-month interval (blue dot) is shifted left and crosses 0. For height, the abstract interval is wider than and offset from both Table 3 height intervals. Values are from the paper; recomputed p values are given in the table above.

Why it matters

Steroid-related weight gain is a main safety concern in Duchenne muscular dystrophy, and comparing weight status with prednisone is one of the paper's key anthropometric results. Read alone, the abstract implies that at 12 months vamorolone raised BMI Z-score clearly more than prednisone did (0.57, CI excluding 0). The paper's own 12-month estimate is less than half that size (0.24) and is not statistically significant (CI −0.04 to 0.52, p = 0.09). That matches the Discussion's statement that the excess was present "at 6 but not at 12 months". Many readers see only the abstract, and abstract values are what get indexed and carried into reviews and meta-analyses, so the overstatement can spread.

The height error leaves the direction and significance of the growth finding intact: Table 3 gives 0.44 (0.25 to 0.64), p < 0.0001. Only its size is overstated, by 0.13 Z-score units. The paper's headline conclusions are unaffected: similar motor efficacy, growth protection with vamorolone, and BMI increases on all three treatments. Vamorolone's 12-month BMI excess over deflazacort (0.29, 0.03 to 0.56) is correctly reported.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Unless the existing correction (e218498) already covers it, the abstract's Results sentence could be amended to the 12-month values in Table 3:

Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.24 [−0.04 to 0.52]; vamorolone vs deflazacort 0.29 [0.03–0.56]), whereas growth slowdown occurred in prednisone- and deflazacort-treated boys but not vamorolone (12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.44 [0.25–0.64]; vamorolone vs deflazacort 0.72 [0.53–0.91]).

Because the 12-month BMI difference versus prednisone is not statistically significant, the phrase "greatest with vamorolone" could also be qualified, for example "greater with vamorolone than deflazacort at 12 months, and than prednisone at 6 months". That wording would match the paper's own Discussion. Separately, the authors could check the 6-month BMI p value (0.009) in Table 3 and the Results text against its CI (0.24–0.90).

Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13225239 write-up and regenerate its figures.

Paper: "Vamorolone for Duchenne Muscular Dystrophy", Neurology 2026,
doi:10.1212/WNL.0000000000214756 (PMC13225239).

All inputs are copied from the article as archived in PubMed Central: Table 3
(LSM differences, 95% CI and p values at 6 and 12 months) and the Abstract
(Results paragraph). Run:
    python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below).
"""
import os
import shutil
import textwrap

from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)

# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Table 3: (outcome, comparison, month) -> (LSM difference, CI low, CI high, reported p)
T3 = {
    ("BMI", "Pred", 6): (0.57, 0.24, 0.90, "0.009"),
    ("BMI", "Pred", 12): (0.24, -0.04, 0.52, "0.09"),
    ("BMI", "DFZ", 6): (0.32, 0.00, 0.64, "0.05"),
    ("BMI", "DFZ", 12): (0.29, 0.03, 0.56, "0.03"),
    ("Weight", "Pred", 6): (0.61, 0.34, 0.89, "<0.0001"),
    ("Weight", "Pred", 12): (0.58, 0.33, 0.82, "<0.0001"),
    ("Weight", "DFZ", 6): (0.60, 0.33, 0.87, "<0.0001"),
    ("Weight", "DFZ", 12): (0.78, 0.54, 1.02, "<0.0001"),
    ("Height", "Pred", 6): (0.25, 0.14, 0.36, "<0.0001"),
    ("Height", "Pred", 12): (0.44, 0.25, 0.64, "<0.0001"),
    ("Height", "DFZ", 6): (0.50, 0.39, 0.61, "<0.0001"),
    ("Height", "DFZ", 12): (0.72, 0.53, 0.91, "<0.0001"),
}
# Abstract, Results: values the abstract labels "12-month LSM difference [95% CI]"
ABSTRACT = {
    ("BMI", "Pred"): (0.57, 0.24, 0.90),
    ("BMI", "DFZ"): (0.29, 0.03, 0.56),
    ("Height", "Pred"): (0.57, 0.24, 0.90),
    ("Height", "DFZ"): (0.72, 0.53, 0.91),
}
NAME = {"Pred": "prednisone", "DFZ": "deflazacort"}


def wald(est, lo, hi):
    """SE from the CI width (normal approximation), z, two-sided p, CI midpoint."""
    se = (hi - lo) / (2 * Z975)
    z = est / se
    return se, z, 2 * stats.norm.sf(abs(z)), (lo + hi) / 2


def fmt_p(p):
    return f"{p:.2g}" if p >= 1e-4 else f"{p:.1e}"


# ---------------------------------------------------------------------------
# 1. Internal consistency of every Table 3 cell
# ---------------------------------------------------------------------------
print("1. Table 3 internal consistency (SE from CI width, normal approximation)")
print(f"   {'cell':24s} {'est':>5s} {'mid':>6s} {'SE':>6s} {'z':>6s} {'p_recomp':>9s}  reported p")
for (o, c, m), (e, lo, hi, p) in T3.items():
    se, z, pr, mid = wald(e, lo, hi)
    print(f"   {o+' vam6/'+c+' '+str(m)+' mo':24s} {e:5.2f} {mid:6.3f} {se:6.3f} {z:6.2f} {fmt_p(pr):>9s}  {p}")

# ---------------------------------------------------------------------------
# 2. Which Table 3 cell does each abstract "12-month" triple equal?
# ---------------------------------------------------------------------------
print("\n2. Abstract '12-month' values vs Table 3")
for (o, c), trip in ABSTRACT.items():
    hits = [f"{k[0]} vs {k[1]} {k[2]} mo" for k, v in T3.items() if v[:3] == trip]
    t12 = T3[(o, c, 12)][:3]
    status = "matches 12-mo cell" if trip == t12 else "DOES NOT match 12-mo cell"
    print(f"   {o} vs {NAME[c]:11s}: abstract {trip}; Table 3 12-mo {t12} -> {status}; "
          f"identical to Table 3 cell(s): {hits}")

a, t12, t6 = ABSTRACT[("BMI", "Pred")], T3[("BMI", "Pred", 12)], T3[("BMI", "Pred", 6)]
print("\n3. Size of the discrepancies")
print(f"   BMI vs prednisone: abstract {a[0]} - Table 3 12-mo {t12[0]} = {a[0]-t12[0]:.2f}; "
      f"ratio {a[0]/t12[0]:.2f}")
print(f"   BMI vs prednisone, 12-mo CI ({t12[1]}, {t12[2]}) contains 0: {t12[1] < 0 < t12[2]}; "
      f"abstract CI ({a[1]}, {a[2]}) contains 0: {a[1] < 0 < a[2]}")
ah, h12, h6 = ABSTRACT[("Height", "Pred")], T3[("Height", "Pred", 12)], T3[("Height", "Pred", 6)]
print(f"   Height vs prednisone: abstract {ah[0]} - Table 3 12-mo {h12[0]} = {ah[0]-h12[0]:.2f}; "
      f"Table 3 6-mo height value is {h6[0]}, so the abstract height triple equals neither height cell")
se12, z12, p12, _ = wald(*t12[:3])
print(f"   BMI vs prednisone 12 mo: SE {se12:.3f}, z {z12:.2f}, recomputed p {p12:.3f} (reported {t12[3]})")
sea, za, pa, _ = wald(*a)
print(f"   Abstract triple 0.57 [0.24-0.90] taken at face value: SE {sea:.3f}, z {za:.2f}, p {pa:.5f}")

# ---------------------------------------------------------------------------
# 4. Secondary observation: 6-month BMI vs prednisone, reported p = 0.009
# ---------------------------------------------------------------------------
print("\n4. 6-month BMI vs prednisone: what would reconcile p = 0.009 with 0.57 (0.24, 0.90)?")
e, lo, hi, _ = T3[("BMI", "Pred", 6)]
half = (hi - lo) / 2
for df in (3, 4, 5, 10, 20, 30, 50, 70):
    tc = stats.t.ppf(0.975, df)
    se = half / tc
    t = e / se
    print(f"   t-based CI with df={df:3d}: SE {se:.3f}, t {t:.2f}, two-sided p {2*stats.t.sf(t, df):.4f}")
print("   (p = 0.009 needs roughly df = 4; with df of 10 or more (26 + 48 boys) p is at most"
      " 0.003. This does not affect the abstract error.)")

# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW = 1.5      # points; 2 px at 96 px/in
MS = 8        # marker size in points (>= 8 px)
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})


def ci_str(v):
    e, lo, hi = v[:3]
    f = lambda x: f"{x:.2f}".replace("-", "−")
    return f"{f(e)} [{f(lo)} to {f(hi)}]"


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
    plt.close(fig)


def recessive(ax, axis="x"):
    ax.grid(axis=axis, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(INK2)


# --- A: two text panels joined by annotated connectors ----------------------
def cand_A():
    fig = plt.figure(figsize=(7.2, 5.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")

    def panel(x, y, w, h, title, sub, edge):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
                                    fc="white", ec=edge, lw=LW))
        ax.text(x + 0.018, y + h - 0.03, title, fontsize=11, weight="bold", va="top")
        ax.text(x + 0.018, y + h - 0.085, sub, fontsize=10, color=INK2, va="top")

    PX0, PX1, PW = 0.02, 0.58, 0.40
    panel(PX0, 0.17, PW, 0.79, "Abstract (Results)", "Values labelled \u201c12-month\u201d", ORANGE)
    q1 = ("\u201cBMI Z-scores increased in all groups, greatest with vamorolone "
          "(12-month LSM difference [95% CI]: vamorolone vs prednisone")
    q2 = "\u201c(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone"
    ax.text(0.038, 0.79, textwrap.fill(q1, 40), fontsize=10, va="top", style="italic")
    ax.text(0.038, 0.615, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
    ax.text(0.038, 0.53, textwrap.fill(q2, 40), fontsize=10, va="top", style="italic")
    ax.text(0.038, 0.405, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
    ax.text(0.038, 0.31, textwrap.fill("The paired deflazacort values (0.29 [0.03\u20130.56] and "
                                        "0.72 [0.53\u20130.91]) match Table 3 at 12 months.", 40),
            fontsize=10, color=INK2, va="top")

    panel(PX1, 0.17, PW, 0.79, "Table 3 (weighted MMRM)", "Vamorolone 6 vs prednisone:\nLSM difference; 95% CI; p", BLUE)
    rows = [("BMI Z-score, 6 mo", T3[("BMI", "Pred", 6)], 0.75),
            ("BMI Z-score, 12 mo", T3[("BMI", "Pred", 12)], 0.59),
            ("Height Z-score, 12 mo", T3[("Height", "Pred", 12)], 0.43)]
    ycen = {}
    for lab, v, y in rows:
        ax.text(0.598, y, lab, fontsize=10, color=INK2, va="top")
        p = "< " + v[3][1:] if v[3].startswith("<") else "= " + v[3]
        ax.text(0.598, y - 0.065, f"{v[0]:.2f}; {v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212")
                + f"   p {p}", fontsize=11, va="center", color=BLUE, weight="bold")
        ycen[lab] = y - 0.065
    ax.text(0.598, 0.31, textwrap.fill("The Results text and the Discussion agree with Table 3: "
                                        "the BMI excess was present \u201cat 6 but not at 12 months\u201d.", 40),
            fontsize=10, color=INK2, va="top")

    def conn(y0, y1, rad, ls="-"):
        ax.add_patch(FancyArrowPatch((PX0 + PW + 0.012, y0), (PX1 - 0.002, y1),
                                     connectionstyle=f"arc3,rad={rad}", arrowstyle="-|>",
                                     mutation_scale=13, lw=LW, color=INK, ls=ls, shrinkA=0, shrinkB=0))
        ax.plot([0.25, PX0 + PW + 0.012], [y0, y0], color=INK, lw=LW, ls=ls)

    conn(0.615, ycen["BMI Z-score, 6 mo"], -0.2)
    conn(0.615, ycen["BMI Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
    conn(0.405, ycen["BMI Z-score, 6 mo"], -0.45)
    conn(0.405, ycen["Height Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
    ax.text(0.02, 0.115, "Solid arrows: identical numbers. Both abstract values equal the 6-month BMI cell.",
            fontsize=10)
    ax.text(0.02, 0.07, "Dashed arrows: the 12-month cell each abstract label points to. BMI: 0.57 vs 0.24,",
            fontsize=10)
    ax.text(0.02, 0.025, "and the 12-month CI includes 0. Height: 0.57 vs 0.44.", fontsize=10)
    save(fig, "cand_A")


# --- B: dot-and-interval, one axis, the two prednisone contrasts --------------
def cand_B():
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    items = [
        ("BMI", "Abstract, labelled 12 mo", ABSTRACT[("BMI", "Pred")], ORANGE, "o", None),
        ("BMI", "Table 3, 12 mo", T3[("BMI", "Pred", 12)], BLUE, "o", T3[("BMI", "Pred", 12)][3]),
        ("BMI", "Table 3, 6 mo", T3[("BMI", "Pred", 6)], BLUE, "D", T3[("BMI", "Pred", 6)][3]),
        ("Height", "Abstract, labelled 12 mo", ABSTRACT[("Height", "Pred")], ORANGE, "o", None),
        ("Height", "Table 3, 12 mo", T3[("Height", "Pred", 12)], BLUE, "o", T3[("Height", "Pred", 12)][3]),
        ("Height", "Table 3, 6 mo", T3[("Height", "Pred", 6)], BLUE, "D", T3[("Height", "Pred", 6)][3]),
    ]
    ys = [6.2, 5.4, 4.6, 2.6, 1.8, 1.0]
    for (o, lab, v, col, mk, p), y in zip(items, ys):
        ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="round")
        ax.plot(v[0], y, marker=mk, ms=MS, color=col, mec=BG, mew=1.2,
                mfc=col if mk == "o" else BG, zorder=3)
        if mk == "D":
            ax.plot(v[0], y, marker=mk, ms=MS - 1, color=col, mfc=BG, mew=LW, zorder=4)
        txt = ci_str(v) + ((",  p < " + p[1:] if p.startswith("<") else f",  p = {p}") if p else "")
        ax.text(1.02, y, txt, va="center", fontsize=10, color=INK)
    ax.axvline(0, color=INK2, lw=0.8)
    ax.set_yticks(ys)
    ax.set_yticklabels([i[1] for i in items], fontsize=10)
    for t, it in zip(ax.get_yticklabels(), items):
        t.set_color(INK)
    ax.text(-0.42, 6.85, "BMI Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
            transform=ax.transData, ha="left")
    ax.text(-0.42, 3.25, "Height Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
            transform=ax.transData, ha="left")
    ax.set_xlim(-0.42, 1.0); ax.set_ylim(0.4, 7.2)
    ax.set_xlabel("LSM difference in Z-score from baseline (95% CI); 0 = no difference")
    recessive(ax)
    ax.tick_params(axis="y", length=0)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month"),
         Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3, 12 months"),
         Line2D([], [], color=BLUE, marker="D", ms=MS - 1, mfc=BG, lw=LW, label="Paper's Table 3, 6 months")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=3, frameon=False,
              fontsize=10, handlelength=1.6, columnspacing=1.2)
    save(fig, "cand_B")


# --- C: lookup map, each abstract triple wired to the Table 3 cell it equals --
def cand_C():
    fig = plt.figure(figsize=(7.2, 4.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    ax.text(0.03, 0.955, "Abstract: \u201c12-month LSM difference\u201d", weight="bold", fontsize=11)
    ax.text(0.55, 0.955, "Table 3: LSM difference (95% CI)", weight="bold", fontsize=11)
    cols = {12: 0.64, 6: 0.82}          # 12-month column first, next to the abstract
    W, H = 0.15, 0.12
    rows = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
    ry = {r: 0.72 - i * 0.15 for i, r in enumerate(rows)}
    for m, x in cols.items():
        ax.text(x, 0.215 - 0.06, f"{m} months", ha="center", fontsize=10, color=INK2)
    cell = {}
    for r, y in ry.items():
        for m, x in cols.items():
            v = T3[(r[0], r[1], m)]
            ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="white", ec=GRID, lw=1))
            ax.text(x, y + 0.02, f"{v[0]:.2f}", ha="center", va="center", fontsize=10.5,
                    color=BLUE, weight="bold")
            ax.text(x, y - 0.03, f"{v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212"), ha="center",
                    va="center", fontsize=10, color=BLUE)
            cell[(r, m)] = (x, y)
    for r in (("BMI", "Pred"), ("Height", "Pred")):
        x, y = cell[(r, 12)]
        ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="none", ec=INK, lw=LW, ls=(0, (3, 2))))
    ax.text(0.64, 0.80, "expected", ha="center", fontsize=10, color=INK)
    top = 0.905                      # routing lane above the table
    lane = {("BMI", "Pred"): 0.47, ("Height", "Pred"): 0.50}
    for r, y in ry.items():
        v = ABSTRACT[r]
        ax.text(0.03, y + 0.022, f"{r[0]} vs {NAME[r[1]]}", fontsize=10, color=INK2, va="center")
        ax.text(0.03, y - 0.028, ci_str(v), fontsize=10.5, color=ORANGE, weight="bold", va="center")
        if T3[(r[0], r[1], 12)][:3] == v:
            x, yy = cell[(r, 12)]
            ax.add_patch(FancyArrowPatch((0.32, y), (x - W / 2 - 0.004, yy), arrowstyle="-|>",
                                         mutation_scale=12, lw=LW, color=INK2))
            ax.text(0.45, y + 0.018, "match", ha="center", fontsize=10, color=INK2)
        else:
            src = [k for k, t in T3.items() if t[:3] == v][0]
            x, yy = cell[((src[0], src[1]), src[2])]
            lx = lane[r]
            ax.plot([0.32, lx, lx, x], [y, y, top, top], color=INK, lw=LW, solid_joinstyle="round")
            ax.add_patch(FancyArrowPatch((x, top), (x, yy + H / 2 + 0.004), arrowstyle="-|>",
                                         mutation_scale=12, lw=LW, color=INK, shrinkA=0))
    ax.text(0.03, 0.09, textwrap.fill("Arrows lead to the Table 3 cell whose three numbers equal the "
                                       "abstract value. Both prednisone values equal the 6-month BMI cell, "
                                       "not the 12-month cells (dashed) that the label implies.", 80),
            fontsize=10, va="center")
    save(fig, "cand_C")


# --- D: all four abstract 12-month contrasts vs Table 3 12-month values ------
def cand_D():
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    order = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
    base = [4.0, 3.0, 2.0, 1.0]
    for r, y in zip(order, base):
        a, t = ABSTRACT[r], T3[(r[0], r[1], 12)]
        for v, col, dy in ((a, ORANGE, 0.17), (t, BLUE, -0.17)):
            ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="round")
            ax.plot(v[0], y + dy, "o", ms=MS, color=col, mec=BG, mew=1.2, zorder=3)
        d = a[0] - t[0]
        note = "identical" if a == t[:3] else f"differ by {d:.2f}"
        ax.text(1.12, y, note, va="center", fontsize=10, color=INK if d else INK2,
                weight="bold" if d else "normal")
    ax.axvline(0, color=INK2, lw=0.8)
    ax.set_yticks(base)
    ax.set_yticklabels([f"{r[0]} Z-score\nvs {NAME[r[1]]}" for r in order], fontsize=10, color=INK)
    ax.set_xlim(-0.2, 1.1); ax.set_ylim(0.5, 4.6)
    ax.set_xlabel("12-month LSM difference in Z-score, vamorolone 6 minus comparator (95% CI)")
    recessive(ax); ax.tick_params(axis="y", length=0)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Abstract (labelled 12-month)"),
         Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Table 3, 12-month cell")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_D")


# --- E: trajectory 6 -> 12 months, with the abstract's 12-month claim overlaid -
def cand_E():
    fig, ax = plt.subplots(figsize=(7.2, 4.0))
    xs = {("BMI", 6): 0, ("BMI", 12): 1, ("Height", 6): 3.0, ("Height", 12): 4.0}
    for o in ("BMI", "Height"):
        v6, v12 = T3[(o, "Pred", 6)], T3[(o, "Pred", 12)]
        ax.plot([xs[(o, 6)], xs[(o, 12)]], [v6[0], v12[0]], color=BLUE, lw=1.0, alpha=0.6)
        for m, v in ((6, v6), (12, v12)):
            x = xs[(o, m)]
            ax.plot([x, x], [v[1], v[2]], color=BLUE, lw=LW)
            ax.plot(x, v[0], "o", ms=MS, color=BLUE, mec=BG, mew=1.2, zorder=3)
            ax.text(x - 0.07, v[0], f"{v[0]:.2f}", ha="right", va="center", fontsize=10, color=INK)
        a = ABSTRACT[(o, "Pred")]
        xa = xs[(o, 12)] + 0.75
        ax.plot([xa, xa], [a[1], a[2]], color=ORANGE, lw=LW)
        ax.plot(xa, a[0], "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
        ax.text(xa + 0.07, a[0], f"{a[0]:.2f}", ha="left", va="center", fontsize=10, color=INK)
    # the copied value: dotted guide from the 6-month BMI estimate
    ax.plot([xs[("BMI", 6)], xs[("Height", 12)] + 0.75], [0.57, 0.57], color=INK2, lw=0.9, ls=(0, (2, 2)))
    ax.text(2.4, 0.60, "0.57 [0.24–0.90] = 6-month BMI cell", ha="center", fontsize=10, color=INK2)
    ax.axhline(0, color=INK2, lw=0.8)
    ax.set_xticks([0, 1, 1.75, 3.0, 4.0, 4.75])
    ax.set_xticklabels(["6 mo", "12 mo", "Abstract\n“12 mo”", "6 mo", "12 mo",
                        "Abstract\n“12 mo”"], fontsize=10)
    ax.text(0.875, -0.36, "BMI Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
    ax.text(3.875, -0.36, "Height Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
    ax.set_xlim(-0.5, 5.3); ax.set_ylim(-0.12, 1.05)
    ax.set_ylabel("LSM difference, vamorolone 6\nminus prednisone (95% CI)")
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="x", length=0, pad=4)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3"),
         Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False, fontsize=10)
    fig.subplots_adjust(bottom=0.2)
    save(fig, "cand_E")


for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
    f()

CHOSEN = {"fig1": "cand_A", "fig2": "cand_B"}
for fig, cand in CHOSEN.items():
    for ext in ("svg", "png"):
        shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"), os.path.join(HERE, f"{fig}.{ext}"))
print("\nFigures written:", ", ".join(f"{c}.svg/.png" for c in "cand_A cand_B cand_C cand_D cand_E".split()),
      "; fig1 =", CHOSEN["fig1"], "; fig2 =", CHOSEN["fig2"])

Abstract reports 73.5% of trials used biomarkers; the Results and phase counts give 56.3%

Neurology (2026) · doi:10.1212/WNL.0000000000218373 · PMC13445500

Summary

This systematic review of 238 phase II–IV Alzheimer disease drug trials states in its abstract that "73.5% (175/238) used at least one biomarker as efficacy measures". The Results section gives 56.3% (134/238) for the same quantity. The phase-stratified counts in the Results (79/135 + 55/103 = 134/238) and the Discussion's "more than half" agree with 134, not 175. The figure 175/238 (73.5%) appears in the Results as the count of trials that used clinical outcome measures as primary measures, so the abstract seems to carry that count in the biomarker position. The abstract overstates one of the review's headline descriptive results by 41 trials (17.2 percentage points). The paper's qualitative conclusion that biomarker use is widespread still holds at 56.3%.

What the paper reports

Abstract, Results:

Among 238 included trials, 95.4% (227/238) used at least one clinical outcome and 73.5% (175/238) used at least one biomarker as efficacy measures.

Abstract, Discussion:

while biomarker use is widespread

Results, "Efficacy Measures in AD Drug Trials" (two consecutive sentences):

Among 238 included trials, 95.4% (227/238) used at least one clinical outcome measure as an efficacy measure, of which 73.5% (175/238) used clinical outcome measures as primary measures and 84.5% (201/238) as secondary measures. In addition, 56.3% (134/238) of trials used at least one biomarker as an efficacy measure, of which 11.3% (27/238) used at least one biomarker as a primary measure and 41.2% (98/238) as a secondary measure.

The next sentence in the same paragraph:

Overall, 18.1% (43/238) of trials included neither a clinical outcome measure nor a biomarker as their primary measure.

Results, "Sample Characteristics":

Of 238 included trials, 56.7% (135/238) were phase II and 43.3% (103/238) were phase III or IV.

Results, "Clinical Relevance of Efficacy Measures", phase II and phase III–IV (domains assessed overall):

functional ability and dependency (66.7%, 90/135), and biomarkers (58.5%, 79/135).

functional ability and dependency (77.7%, 80/103), and biomarkers (53.4%, 55/103).

The same paragraph, restricted to primary measures (phase II, then phase III–IV):

followed by biomarkers (14.8%, 20/135)

and biomarkers (5.8%, 6/103)

Results, same section, and Abstract, Results:

Notably, 8.4% (20/238) used biomarkers as the only primary efficacy measures.

whereas 8.4% (20/238) used biomarkers as the sole primary efficacy measure

Discussion, first paragraph and third paragraph:

Moreover, more than half of the trials evaluated biomarkers, predominantly as secondary measures

more than half of trials used biomarkers as efficacy measures

Methods, "Data Extraction" and "Data Analysis":

We also extracted whether outcome measures were evaluated as primary and secondary outcome measures or any exploratory outcome measures.

We determined the proportion of trials that used clinical outcome measures or biomarkers as any efficacy measures, primary or secondary.

What the paper's own numbers show

Method. All counts share one denominator: the 238 included trials, split into 135 phase II and 103 phase III–IV trials. I recomputed every printed percentage from its printed count. I then rebuilt the number of trials with any biomarker use from the phase-stratified counts, compared it with the abstract and the Results, and checked which other Results quantity equals 175/238. As a further check I used the primary-measure counts: the trials with a clinical or biomarker primary measure (238 − 43), the clinical-primary count (175) and the biomarker-primary count (27). Together they imply an overlap, and from it a biomarker-only primary count that can be compared with the paper's 20/238. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Trials using ≥ 1 biomarker as an efficacy measure Abstract, Results 73.5% (175/238) 175/238 = 73.53%: the arithmetic is right, but the count conflicts with the body
Trials using ≥ 1 biomarker as an efficacy measure Results, Efficacy Measures 56.3% (134/238) 134/238 = 56.30%
Abstract minus Results derived — 41 trials, 17.2 percentage points; the abstract count is 1.31 × the Results count
Trials using clinical outcomes as primary measures Results, Efficacy Measures 73.5% (175/238) 175/238 = 73.53%, the same count and percentage the abstract gives for biomarkers
Phase II, any biomarker use Results, Clinical Relevance 58.5% (79/135) 58.52%
Phase III–IV, any biomarker use Results, Clinical Relevance 53.4% (55/103) 53.40%
Sum of the phase strata derived — 135 + 103 = 238 trials; 79 + 55 = 134 biomarker trials (56.3%), equal to the Results and 41 below the abstract
Discussion: "more than half" used biomarkers Discussion — half of 238 = 119; 134 fits "more than half". 175 is close to three-quarters (178.5)
Biomarker primary + secondary Results, Efficacy Measures 27/238 (11.3%) and 98/238 (41.2%) 27 + 98 = 125, the most trials that could use a biomarker as a primary or secondary measure. 175 exceeds it by 50
Trials with a clinical or biomarker primary measure Results, Efficacy Measures 43/238 (18.1%) with neither 238 − 43 = 195
Implied trials with both clinical and biomarker primary measures derived — 175 + 27 − 195 = 7
Implied biomarker-only primary measures derived vs Results and Abstract 8.4% (20/238) 27 − 7 = 20, which matches. So 175 fits the paper's own figures as the clinical-primary count
Clinical outcome, any, primary and secondary use Results, Efficacy Measures 95.4% (227), 73.5% (175), 84.5% (201) 95.38%, 73.53%, 84.45%; 175 + 201 − 227 = 149 trials with both

Two smaller inconsistencies inside the Results came up. They do not bear on the choice between 134 and 175:

Quantity Location Reported Recomputed
Any biomarker use vs primary + secondary use Results, Efficacy Measures 134 vs 27 and 98 134 is 9 more than 27 + 98 = 125. The Methods say exploratory measures were also extracted, which could account for this, but the paper does not say so
Biomarker as primary measure, phase strata vs overall Results, Clinical Relevance vs Efficacy Measures 20/135 + 6/103 vs 27/238 20 + 6 = 26, one less than 27

Figure

The abstract's biomarker count is the Results' count of trials using clinical outcomes as primary measures

Figure 1. The two conflicting passages, quoted from the paper. Left, the abstract's Results sentence. Right, the Results paragraph "Efficacy Measures in AD Drug Trials", shortened with ellipses. Reported values are in orange. The gray connector links the abstract's biomarker figure to the identical Results figure for clinical outcomes used as primary measures. The orange connector links it to the Results figure for biomarkers, which it contradicts. The blue panel gives the value recomputed from the paper's phase-stratified counts. It matches the Results (134/238, 56.3%), not the abstract.

Counts of trials, on one axis from 0 to 238

Figure 2. The counts behind the conflict, on one axis of trials from 0 to 238. Orange circles are counts printed in the paper. Blue diamonds are counts recomputed from the paper's other figures: the sum of the phase II and phase III–IV biomarker counts (79 + 55 = 134), and the upper bound on primary or secondary biomarker use (27 + 98 = 125). The dashed line marks half of the 238 trials, the threshold of the Discussion's "more than half". The abstract's 175 equals the clinical-primary count and lies 41 trials above every other count of biomarker use in the paper.

Why it matters

The abstract's Results section opens with this figure, and for many readers the abstract is the only part of a review they read or quote. A secondary summary of the article, whose URL carries the date 25 July 2026, already repeats the abstract's figure ("73.5% of analyzed trials incorporated at least one biomarker as an efficacy endpoint").

Explanations considered

The smaller inconsistencies (134 vs 125; 26 vs 27) leave some uncertainty about the exact correct value, perhaps by a few trials. They do not bring it near 175.

Status

As of 2026-10-23:

Suggested correction

  1. In the abstract's Results, replace "73.5% (175/238) used at least one biomarker as efficacy measures" with "56.3% (134/238) used at least one biomarker as efficacy measures", if the authors' data confirm the Results figure.
  2. Reconcile the any-use count (134) with the primary and secondary counts (27 and 98, total 125). For example, state whether exploratory measures are included in "any" use. Also reconcile the phase-level biomarker-primary counts (20 + 6 = 26) with the overall 27.
  3. Check any figure or supplementary table that reports overall biomarker use (for example eTable 2) against the corrected value.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13445500 write-up and regenerate its figures.

Paper: "Efficacy Measures Used in Alzheimer Disease Clinical Trials Between 2015
and 2025", Neurology 2026, doi:10.1212/WNL.0000000000218373 (PMC13445500).

Run:  python recompute.py.txt
All inputs below are counts printed in the paper (location given for each).
Outputs: fig1.svg/png, fig2.svg/png and five candidate figure designs in
candidates/ (cand_A ... cand_E), all written next to this file.
"""
import os
import shutil

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs
N = 238                      # included trials (Abstract; Results)
N_PH2, N_PH34 = 135, 103     # Results, Sample Characteristics
ABS_BIO_ANY = 175            # Abstract: "73.5% (175/238) used at least one biomarker"
ABS_BIO_PCT = 73.5
RES_BIO_ANY = 134            # Results: "56.3% (134/238) ... at least one biomarker"
RES_BIO_PCT = 56.3
RES_CLIN_ANY = 227           # Results: 95.4% (227/238) at least one clinical outcome
RES_CLIN_PRIM = 175          # Results: 73.5% (175/238) clinical outcomes as primary
RES_CLIN_SEC = 201           # Results: 84.5% (201/238) clinical outcomes as secondary
RES_BIO_PRIM = 27            # Results: 11.3% (27/238) biomarker as primary
RES_BIO_SEC = 98             # Results: 41.2% (98/238) biomarker as secondary
RES_NEITHER_PRIM = 43        # Results: 18.1% (43/238) neither as primary
RES_BIO_ONLY_PRIM = 20       # Results & Abstract: 8.4% (20/238) biomarkers only primary
PH2_BIO_ANY, PH34_BIO_ANY = 79, 55       # Results: 58.5% (79/135), 53.4% (55/103)
PH2_BIO_PRIM, PH34_BIO_PRIM = 20, 6      # Results: 14.8% (20/135), 5.8% (6/103)


def pct(k, n=N):
    return 100.0 * k / n


def show(label, value, reported=None):
    s = f"{label:<64s} {value}"
    if reported is not None:
        s += f"   (paper: {reported})"
    print(s)


print("=" * 100)
print("1. Percentages as printed")
show("Abstract  biomarker any use 175/238", f"{pct(ABS_BIO_ANY):.2f}%", "73.5%")
show("Results   biomarker any use 134/238", f"{pct(RES_BIO_ANY):.2f}%", "56.3%")
show("Results   clinical outcome any use 227/238", f"{pct(RES_CLIN_ANY):.2f}%", "95.4%")
show("Results   clinical outcome as primary 175/238", f"{pct(RES_CLIN_PRIM):.2f}%", "73.5%")
show("Results   clinical outcome as secondary 201/238", f"{pct(RES_CLIN_SEC):.2f}%", "84.5%")
show("Results   biomarker as primary 27/238", f"{pct(RES_BIO_PRIM):.2f}%", "11.3%")
show("Results   biomarker as secondary 98/238", f"{pct(RES_BIO_SEC):.2f}%", "41.2%")
show("Results   neither as primary 43/238", f"{pct(RES_NEITHER_PRIM):.2f}%", "18.1%")
show("Results   biomarkers as only primary 20/238", f"{pct(RES_BIO_ONLY_PRIM):.2f}%", "8.4%")
show("Results   phase II biomarker any use 79/135", f"{pct(PH2_BIO_ANY, N_PH2):.2f}%", "58.5%")
show("Results   phase III-IV biomarker any use 55/103", f"{pct(PH34_BIO_ANY, N_PH34):.2f}%", "53.4%")
show("Results   phase II biomarker primary 20/135", f"{pct(PH2_BIO_PRIM, N_PH2):.2f}%", "14.8%")
show("Results   phase III-IV biomarker primary 6/103", f"{pct(PH34_BIO_PRIM, N_PH34):.2f}%", "5.8%")

print("\n2. Abstract vs Results")
gap = ABS_BIO_ANY - RES_BIO_ANY
show("Gap in trials (175 - 134)", gap)
show("Gap in percentage points", f"{pct(gap):.2f}")
show("Ratio abstract / Results (175/134)", f"{ABS_BIO_ANY / RES_BIO_ANY:.3f}")
show("Abstract biomarker count == Results clinical-primary count?",
     ABS_BIO_ANY == RES_CLIN_PRIM)

print("\n3. Phase subtotals")
show("Phase denominators 135 + 103", N_PH2 + N_PH34)
ph_sum = PH2_BIO_ANY + PH34_BIO_ANY
show("Phase biomarker any-use 79 + 55", ph_sum)
show("  as % of 238", f"{pct(ph_sum):.2f}%")
show("  equals Results 134?", ph_sum == RES_BIO_ANY)
show("  equals Abstract 175?", ph_sum == ABS_BIO_ANY)
show("Phase biomarker primary 20 + 6", PH2_BIO_PRIM + PH34_BIO_PRIM)
show("  vs overall biomarker primary 27, difference",
     RES_BIO_PRIM - (PH2_BIO_PRIM + PH34_BIO_PRIM))

print("\n4. Bounds from primary/secondary counts")
ub = RES_BIO_PRIM + RES_BIO_SEC
show("Biomarker primary + secondary (27 + 98), max primary-or-secondary use", ub)
show("  Abstract 175 exceeds this by", ABS_BIO_ANY - ub)
show("  Results 134 exceeds this by", RES_BIO_ANY - ub)
show("Clinical: primary + secondary - any (175 + 201 - 227) = both", RES_CLIN_PRIM + RES_CLIN_SEC - RES_CLIN_ANY)
any_prim = N - RES_NEITHER_PRIM
show("Trials with a clinical or biomarker primary measure (238 - 43)", any_prim)
both_prim = RES_CLIN_PRIM + RES_BIO_PRIM - any_prim
show("  implied both clinical and biomarker primary (175 + 27 - 195)", both_prim)
show("  implied biomarker-only primary (27 - 7)", RES_BIO_PRIM - both_prim, "20/238")
show("Half of 238 (Discussion: 'more than half')", N / 2)
show("Three-quarters of 238", 0.75 * N)

# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"   # values as reported in the paper
BLUE = "#2a78d6"     # values recomputed from the paper's own numbers
GRID = "#e4e3df"
W = 7.2

plt.rcParams.update({
    "font.family": "DejaVu Sans",
    "font.size": 10.5,
    "text.color": INK,
    "axes.labelcolor": INK2,
    "axes.edgecolor": INK2,
    "xtick.color": INK2,
    "ytick.color": INK2,
    "figure.facecolor": BG,
    "axes.facecolor": BG,
    "savefig.facecolor": BG,
    "lines.linewidth": 2,
    "svg.hashsalt": "PMC13445500",
})


def save(fig, name, folder=CAND):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200,
                    bbox_inches="tight", pad_inches=0.12,
                    metadata={"Date": None} if ext == "svg" else None)
    plt.close(fig)


def box(ax, x, y, w, h, edge=INK2, lw=1.2, fill="#ffffff"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                fc=fill, ec=edge, lw=lw, transform=ax.transAxes))


def arrow(ax, p0, p1, color=INK2, rad=0.0, style="-|>", lw=2, ls="-"):
    ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle=style,
                                 mutation_scale=14, color=color, lw=lw, ls=ls,
                                 connectionstyle=f"arc3,rad={rad}", shrinkA=2, shrinkB=2))


def legend_handles():
    return [Line2D([], [], marker="o", ls="", ms=9, mfc=ORANGE, mec=ORANGE,
                   label="Reported in the paper"),
            Line2D([], [], marker="D", ls="", ms=8, mfc=BLUE, mec=BLUE,
                   label="Recomputed from the paper's own counts")]


# ======================================================= Candidate A: text panels
def cand_A():
    fig = plt.figure(figsize=(W, 5.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.text(0.02, 0.975, "The abstract's biomarker count is the Results' clinical-primary count",
            fontsize=12, weight="bold", va="top")
    step = 0.052

    def lines(x, y0, items):
        """items: (text, kind); kind 'h' header, 'q' quote, 'v' reported value."""
        y = y0
        pos = {}
        for t, kind in items:
            if kind == "h":
                ax.text(x, y, t, fontsize=10.5, weight="bold", color=INK2, va="top")
            elif kind == "v":
                ax.text(x, y + 0.004, t, fontsize=12, weight="bold", color=ORANGE, va="top")
                pos[t] = y - 0.022
            else:
                ax.text(x, y, t, fontsize=10.5, color=INK, va="top")
            y -= step
        return pos

    PT, PB = 0.91, 0.30
    box(ax, 0.02, PB, 0.37, PT - PB)
    box(ax, 0.61, PB, 0.37, PT - PB)
    pl = lines(0.04, 0.885, [("Abstract, Results", "h"), ("", "q"),
                             ("\u201c95.4% (227/238) used at least", "q"),
                             ("one clinical outcome and", "q"),
                             ("73.5% (175/238)", "v"),
                             ("used at least one biomarker", "q"),
                             ("as efficacy measures.\u201d", "q")])
    pr = lines(0.63, 0.885, [("Results, Efficacy Measures", "h"),
                             ("in AD Drug Trials", "h"),
                             ("\u201c... of which", "q"),
                             ("73.5% (175/238)", "v"),
                             ("used clinical outcome measures", "q"),
                             ("as primary measures ...", "q"),
                             ("In addition,", "q"),
                             ("56.3% (134/238)", "v"),
                             ("of trials used at least one bio-", "q"),
                             ("marker as an efficacy measure\u201d", "q")])
    yl = 0.885 - 4 * step - 0.022            # left orange value
    yr1 = 0.885 - 3 * step - 0.022           # right 175
    yr2 = 0.885 - 7 * step - 0.022           # right 134
    arrow(ax, (0.30, yl + 0.012), (0.625, yr1), color=INK2, rad=-0.2)
    ax.text(0.50, yr1 + 0.075, "same count,\ndifferent measure", fontsize=10, color=INK2,
            ha="center", va="bottom", linespacing=1.2)
    arrow(ax, (0.30, yl - 0.012), (0.625, yr2), color=ORANGE, rad=0.2, style="<|-|>")
    ax.text(0.50, yr2 - 0.075, "conflict:\n175 vs 134", fontsize=10.5, weight="bold",
            color=ORANGE, ha="center", va="top", linespacing=1.2)
    # recomputation strip
    box(ax, 0.02, 0.03, 0.96, 0.22, edge=BLUE, lw=1.6)
    ax.text(0.04, 0.225, "Recomputed from the paper's own counts", fontsize=10.5, weight="bold",
            color=BLUE, va="top")
    ax.text(0.04, 0.165,
            f"Phase II {PH2_BIO_ANY}/135 + phase III\u2013IV {PH34_BIO_ANY}/103 = "
            f"{ph_sum}/238 = {pct(ph_sum):.1f}%, matching the Results",
            fontsize=10.5, va="top")
    ax.text(0.04, 0.105,
            f"Abstract minus Results: {gap} trials, {pct(gap):.1f} percentage points",
            fontsize=10.5, va="top")
    save(fig, "cand_A_text_panels")


# ======================================================= Candidate B: dot comparison
def cand_B():
    rows = [  # label, value, kind
        ("Abstract: ≥ 1 biomarker", ABS_BIO_ANY, "rep"),
        ("Results: ≥ 1 biomarker", RES_BIO_ANY, "rep"),
        ("Phase II + phase III–IV biomarker use (79 + 55)", ph_sum, "rec"),
        ("Biomarker primary + secondary (27 + 98), maximum", ub, "rec"),
        ("Results: clinical outcome as primary", RES_CLIN_PRIM, "rep"),
    ]
    fig, ax = plt.subplots(figsize=(W, 3.5))
    ys = list(range(len(rows)))[::-1]
    for (lab, v, kind), y in zip(rows, ys):
        c = ORANGE if kind == "rep" else BLUE
        m = "o" if kind == "rep" else "D"
        ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1, solid_capstyle="round")
        ax.plot(v, y, marker=m, ms=10 if m == "o" else 9, color=c, mec=BG, mew=1.5, zorder=3)
        ax.text(v + 4, y, f"{v}  ({pct(v):.1f}%)", va="center", fontsize=10.5, color=INK)
    ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
    ax.text(N / 2 - 2, -0.42, "half of 238 (Discussion: “more than half”)",
            fontsize=10, color=INK2, va="center", ha="right",
            bbox=dict(fc=BG, ec="none", pad=1))
    ax.set_yticks(ys, [r[0] for r in rows], fontsize=10.5)
    ax.set_xlim(0, N); ax.set_ylim(-0.7, len(rows) - 0.4)
    ax.set_xticks([0, 50, 100, 150, 200, 238])
    ax.set_xlabel("Trials (of 238)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.annotate("", xy=(RES_BIO_ANY, ys[1] + 0.5), xytext=(ABS_BIO_ANY, ys[1] + 0.5),
                arrowprops=dict(arrowstyle="<|-|>", color=ORANGE, lw=1.6))
    ax.text((ABS_BIO_ANY + RES_BIO_ANY) / 2, ys[1] + 0.56, "41 trials", ha="center",
            va="bottom", fontsize=10, color=INK, weight="bold")
    ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.35, -0.2),
              ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_B_dot_comparison")


# ======================================================= Candidate C: calculation flow
def cand_C():
    fig = plt.figure(figsize=(W, 3.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")

    def node(x, y, w, h, title, value, color, sub=None):
        box(ax, x, y, w, h, edge=color, lw=1.8)
        ax.text(x + w / 2, y + h - 0.045, title, ha="center", va="top", fontsize=10, color=INK2)
        ax.text(x + w / 2, y + h / 2 - (0.02 if sub else 0.04), value, ha="center", va="center",
                fontsize=12.5, weight="bold", color=INK)
        if sub:
            ax.text(x + w / 2, y + 0.05, sub, ha="center", va="bottom", fontsize=10, color=INK2)

    node(0.02, 0.62, 0.22, 0.28, "Results, phase II", "79 / 135", ORANGE, "58.5%")
    node(0.02, 0.22, 0.22, 0.28, "Results, phase III–IV", "55 / 103", ORANGE, "53.4%")
    node(0.34, 0.42, 0.22, 0.28, "Recomputed sum", f"{ph_sum} / {N_PH2 + N_PH34}", BLUE,
         f"{pct(ph_sum):.1f}%")
    node(0.70, 0.62, 0.28, 0.28, "Results: ≥ 1 biomarker", "134 / 238", ORANGE, "56.3%")
    node(0.70, 0.16, 0.28, 0.28, "Abstract: ≥ 1 biomarker", "175 / 238", ORANGE, "73.5%")
    arrow(ax, (0.24, 0.76), (0.34, 0.60))
    arrow(ax, (0.24, 0.36), (0.34, 0.52))
    ax.text(0.29, 0.56, "+", fontsize=14, ha="center", va="center", color=INK2)
    arrow(ax, (0.56, 0.60), (0.70, 0.74), color=BLUE)
    ax.text(0.615, 0.715, "matches", fontsize=10.5, color=BLUE, weight="bold", ha="center")
    arrow(ax, (0.56, 0.50), (0.70, 0.32), color=ORANGE, ls="--")
    ax.text(0.555, 0.30, "does not match\n(41 trials fewer)", fontsize=10.5, color=ORANGE,
            weight="bold", ha="center", va="top", linespacing=1.2)
    ax.text(0.84, 0.035, "175/238 is the Results count of trials using\nclinical outcomes as primary measures",
            fontsize=10, color=INK2, ha="center", va="bottom", linespacing=1.2)
    ax.text(0.02, 0.975, "Blue: recomputed from the paper's counts.  Orange: as printed in the paper.",
            fontsize=10, color=INK2, va="top")
    save(fig, "cand_C_calculation_flow")


# ======================================================= Candidate D: stacked bars
def cand_D():
    fig, ax = plt.subplots(figsize=(W, 2.9))
    rows = ["Abstract: ≥ 1 biomarker", "Results: ≥ 1 biomarker",
            "Phase subtotals (recomputed)"]
    y = [2, 1, 0]
    h = 0.55
    ax.barh(y[0], ABS_BIO_ANY, height=h, color=ORANGE)
    ax.barh(y[1], RES_BIO_ANY, height=h, color=ORANGE)
    ax.barh(y[2], PH2_BIO_ANY, height=h, color=BLUE, edgecolor=BG, linewidth=2)
    ax.barh(y[2], PH34_BIO_ANY, left=PH2_BIO_ANY, height=h, color=BLUE, alpha=0.6,
            edgecolor=BG, linewidth=2)
    ax.text(PH2_BIO_ANY / 2, y[2], "phase II 79", ha="center", va="center", color="white",
            fontsize=10, weight="bold")
    ax.text(PH2_BIO_ANY + PH34_BIO_ANY / 2, y[2], "III–IV 55", ha="center", va="center",
            color=INK, fontsize=10, weight="bold")
    for yy, v in zip(y, [ABS_BIO_ANY, RES_BIO_ANY, ph_sum]):
        ax.text(v + 3, yy, f"{v}  ({pct(v):.1f}%)", va="center", fontsize=10.5)
    ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)))
    ax.text(N / 2 + 1.5, 2.55, "half of 238", fontsize=10, color=INK2, va="bottom")
    ax.set_yticks(y, rows, fontsize=10.5)
    ax.set_xlim(0, N); ax.set_ylim(-0.5, 2.9)
    ax.set_xticks([0, 50, 100, 150, 200, 238]); ax.set_xlabel("Trials (of 238)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    hs = [plt.Rectangle((0, 0), 1, 1, color=ORANGE, label="Reported in the paper"),
          plt.Rectangle((0, 0), 1, 1, color=BLUE, label="Recomputed from the paper's own counts")]
    ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.35, -0.28), ncol=2,
              frameon=False, fontsize=10)
    save(fig, "cand_D_stacked_bars")


# ======================================================= Candidate E: percentage strip
def cand_E():
    fig, ax = plt.subplots(figsize=(W, 2.8))
    yb, yc = 1, 0
    ax.plot([0, 100], [yb, yb], color=GRID, lw=2, zorder=0)
    ax.plot([0, 100], [yc, yc], color=GRID, lw=2, zorder=0)
    ax.plot(ABS_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
    ax.plot(RES_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
    ax.plot(pct(ph_sum), yb - 0.17, "D", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
    ax.plot(pct(RES_CLIN_PRIM), yc, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
    ax.text(ABS_BIO_PCT - 1, yb + 0.16, "Abstract 73.5%", ha="left", va="bottom", fontsize=10.5)
    ax.text(RES_BIO_PCT + 1, yb + 0.16, "Results 56.3%", ha="right", va="bottom", fontsize=10.5,
            bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
    ax.text(pct(ph_sum) - 1.8, yb - 0.17, f"phases 79 + 55 = {pct(ph_sum):.1f}%",
            ha="right", va="center", fontsize=10.5, color=INK,
            bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
    ax.text(pct(RES_CLIN_PRIM), yc - 0.18, "Results 73.5% (175/238)", ha="center", va="top",
            fontsize=10.5)
    ax.plot([ABS_BIO_PCT, ABS_BIO_PCT], [yc + 0.1, yb - 0.1], color=INK2, lw=1.2,
            ls=(0, (2, 2)))
    ax.text(ABS_BIO_PCT + 1.2, 0.5, "identical: 175/238", fontsize=10, color=INK2, va="center")
    ax.axvline(50, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
    ax.text(49, -0.45, "50%: Discussion's “more than half”", ha="right", fontsize=10,
            color=INK2, va="center")
    ax.set_yticks([yb, yc], ["≥ 1 biomarker,\nany use", "Clinical outcome\nas primary"],
                  fontsize=10.5)
    ax.set_xlim(0, 100); ax.set_ylim(-0.65, 1.55)
    ax.set_xticks(range(0, 101, 25)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 25)])
    ax.set_xlabel("Share of the 238 included trials")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.4, -0.3),
              ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_E_percentage_strip")


for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
    f()

# Provisional choice for the write-up (pending the reader's pick):
# fig1 = candidate A (text panels), fig2 = candidate B (dot comparison).
FIG1, FIG2 = "cand_A_text_panels", "cand_B_dot_comparison"
for src, dst in ((FIG1, "fig1"), (FIG2, "fig2")):
    for ext in ("svg", "png"):
        shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nFigures written to", HERE, "and", CAND)

Abstract pairs Ala510 with dA and Asn513 with dC; the paper's Results and Conclusion pair them the other way round

Nucleic Acids Res (2015) · doi:10.1093/nar/gkv760 · PMC4652769

Summary

The abstract states that "Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively". The body of the paper states the reverse pairing three times. The N513A paragraph of the Results concludes that "Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA". The Conclusion states that "Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC". The double-mutant paragraph describes "the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively", and the paper ties the syn conformation to dA and the anti conformation to dC. The contradiction is between the abstract and the paper's own text, and that internal contradiction is the main basis of this report.

Table 1 supports the body text's assignment for Ala510 under both ways of summarising the data that I checked. For Asn513 the data support the body text (dA) on the absolute metric and are about even on the alternative metric; on neither metric do they support the abstract's dC assignment. The underlying kinetics and the Results text are internally consistent; the error is confined to one sentence of the abstract.

What the paper reports

Abstract (as archived in PMC):

Steady-state kinetics reveal that two residues lining the dNTP binding pocket, Ala510 and Asn513, play differential roles in dNTP selectivity. Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.

Results and Discussion, N513A paragraph:

However, the N513A substitution resulted in a much more drastic effect in the context of 8-oxo-dGMP incorporation opposite dA (25-fold, Figure 3D). Also, the catalytic efficiencies of dGMP and 8-oxo-dGMP incorporation opposite dC were reduced by 1.3 (Figure 3E) and 6.4-fold (Figure 3F) respectively. Taken together, these results suggest that Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA, implying that this residue selectively stabilizes the syn-conformation of 8-oxo-dGTP.

Results and Discussion, A510D paragraph:

Notably, the A510D substitution had the largest effect on the incorporation of 8-oxo-dGTP opposite dC (7.6-fold, Figure 3E) as opposed to dGTP (2.8-fold, Figure 3F). This suggests that Ala510 may preferentially stabilize the anti-conformation of 8-oxo-dGTP, promoting its incorporation opposite dC.

Results and Discussion, double-mutant paragraph:

Interestingly, the drop in catalytic efficiency was nearly identical for 8-oxo-dGMP incorporation opposite dA and dC. This is consistent with the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively.

Conclusion:

As a consequence, Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC.

So the Results, the double-mutant interpretation and the Conclusion all assign Asn513 to the dA (syn) context and Ala510 to the dC (anti) context. The abstract's sentence assigns them the other way round.

What the paper's own numbers show

Method. Table 1 gives the catalytic efficiency kcat/KM (min⁻¹ µM⁻¹, mean ± SD of three independent experiments) for WT and each mutant, for the correct dNTP and for 8-oxo-dGTP, opposite both dA and dC. From those values I recomputed the fold loss of kcat/KM caused by each substitution (WT ÷ mutant) and compared it with the fold values printed in the Results text. I summarise each residue's template preference two ways, because the answer for Asn513 depends on which one is used:

Approximate SDs are first-order (delta-method) propagations of the Table 1 SDs on the log scale, treating WT and mutant as independent; with n = 3 per estimate they are a rough guide only. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from Table 1
A510D fold loss, 8-oxo-dGTP opposite dA Results 5.1-fold 5.11× (SD ~0.65)
A510D fold loss, 8-oxo-dGTP opposite dC Results 7.6-fold 7.57× (SD ~1.67)
N513A fold loss, 8-oxo-dGTP opposite dA Results 25-fold 24.74× (SD ~3.19)
N513A fold loss, 8-oxo-dGTP opposite dC Results 6.4-fold 6.39× (SD ~0.71)
A510D fold loss, dTTP opposite dA Results 4.8-fold 4.78×
A510D fold loss, dGTP opposite dC Results 2.8-fold 2.76×
N513A fold loss, dTTP opposite dA Results 5.8-fold 5.82×
N513A fold loss, dGTP opposite dC Results 1.3-fold 1.33×

The Results-text values and the values recomputed from Table 1 agree throughout, so the contradiction is not between the Results and the table; it is between the abstract and the rest of the paper.

Which context does each residue affect more?

Residue Metric 1 (absolute): dA vs dC larger at Metric 2 (÷ correct dNTP): dA vs dC larger at
Ala510 (A510D) 5.11× vs 7.57× dC (ratio 1.48; |log diff|/SD ~1.5) 1.07 vs 2.74 dC (ratio 2.57; |log diff|/SD ~2.6)
Asn513 (N513A) 24.74× vs 6.39× dA (ratio 3.87; |log diff|/SD ~7.9) 4.25 vs 4.79 about equal (ratio 1.13; |log diff|/SD ~0.4)

Reading. On the absolute metric the pairing is the clean reverse of the abstract: Ala510's larger effect is opposite dC, Asn513's is opposite dA (and the Asn513 gap is large, ~8 approximate SDs). On the relative metric Ala510 is again larger opposite dC, by a clear margin, while Asn513 is essentially even between the two contexts (4.25 vs 4.79, well under one approximate SD apart). So:

A consistency check favours the body text. For the double mutant the fold loss is nearly identical opposite dA (55.8×) and dC (58.9×), which the paper reads as Ala510 and Asn513 having opposite template preferences that cancel. That reading requires Asn513 → dA and Ala510 → dC, i.e. the body text, not the abstract. The Results also note that removing Asn513 cripples 8-oxo-dGTP opposite dA (25-fold) far more than it does the correct dTTP in the same context (5.8-fold), which is the paper's own argument that Asn513 selectively helps the dA (syn) mispair.

Figure

Left, the abstract pairs Ala510 with dA and Asn513 with dC; right, the Results, Conclusion and double-mutant text pair them the other way, and the Table 1 fold losses recomputed below agree with the body text

Figure 1. The abstract's sentence (left, orange) pairs Ala510 with the dA context and Asn513 with the dC context. The Results, Conclusion and double-mutant paragraphs (right) pair them the other way: Asn513 with dA (syn), Ala510 with dC (anti). The strip below lists, recomputed from Table 1, the fold loss of kcat/KM for 8-oxo-dGTP in each context, both as the absolute loss and divided by the loss for the correct dNTP (blue). Blue values are recomputed by recompute.py.txt; the abstract gives no numbers. The Results text prints the same absolute values, rounded.

Design candidates for review. recompute.py.txt also writes five candidate layouts in candidates/ (cand1–cand5). fig1 is currently a copy of cand1 (the two-panel text diagram) and fig2 a copy of cand3 (a one-axis log plot of the fold losses). The final figure choice is open until the reader picks from the five candidates.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract, replace

Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.

with

Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dC and dA, respectively.

(or, equivalently, "Asn513 and Ala510 ... opposite dA and dC, respectively"), so that the abstract agrees with the Results, the Conclusion, the double-mutant interpretation and Table 1.

Recomputation script (python recompute.py.txt)
"""Recompute the Ala510 / Asn513 fold effects of PMC4652769
(Nucleic Acids Res 2015, doi:10.1093/nar/gkv760).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
  fig1.svg / fig1.png              abstract vs body text (two text panels), with Table 1 numbers
  fig2.svg / fig2.png              fold reductions on one log axis, two metrics, approximate SDs
  candidates/cand1..cand5.svg/png  five candidate figure designs, for the reader to choose from

All inputs are copied from Table 1 of the article as archived in PubMed Central:
catalytic efficiency kcat/KM in min^-1 uM^-1, mean +/- SD of three independent
experiments. No raw data are used. (The check of these values against the publisher's
page is recorded in the Status section of writeup.md, not here.)

Colour code, the same in every figure:
  orange = the pairing stated in the abstract (the abstract gives no numbers)
  blue   = recomputed from Table 1 by this script
  grey   = a fold value printed in the Results / Conclusion text (these agree with Table 1)

fig1 and fig2 are provisional copies of candidates 1 and 3 until a design is chosen.
"""
import os
import textwrap

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (Table 1)
# (mean, SD) of kcat/KM
T1 = {
    "A": {  # template dA
        "dTTP":  {"WT": (32, 4), "A510D": (6.7, 0.6), "N513A": (5.5, 1.0), "A510D/N513A": (0.62, 0.07)},
        "8oxo":  {"WT": (2.4, 0.3), "A510D": (0.47, 0.01), "N513A": (0.097, 0.003), "A510D/N513A": (0.043, 0.005)},
    },
    "C": {  # template dC
        "dGTP":  {"WT": (80, 10), "A510D": (29, 5), "N513A": (60, 4), "A510D/N513A": (11, 2)},
        "8oxo":  {"WT": (0.53, 0.05), "A510D": (0.070, 0.014), "N513A": (0.083, 0.005), "A510D/N513A": (0.009, 0.002)},
    },
}
NORMAL = {"A": "dTTP", "C": "dGTP"}

# Fold values printed in the Results / Conclusion text
REPORTED = {
    ("A510D", "A", "8oxo"): 5.1, ("A510D", "C", "8oxo"): 7.6,
    ("N513A", "A", "8oxo"): 25,  ("N513A", "C", "8oxo"): 6.4,
    ("A510D", "A", "dTTP"): 4.8, ("N513A", "A", "dTTP"): 5.8, ("A510D/N513A", "A", "dTTP"): 52,
    ("A510D", "C", "dGTP"): 2.8, ("N513A", "C", "dGTP"): 1.3, ("A510D/N513A", "C", "dGTP"): 7.3,
}

ABSTRACT = ("Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP "
            "opposite dA and dC, respectively.")
Q_N513 = ("Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA")
Q_A510 = ("Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC")
Q_DBL = ("the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- "
         "and syn-conformations, respectively")


def fold(t, nt, mut):
    """Fold decrease in kcat/KM relative to WT, and an approximate SD.

    The SD is a first-order (delta-method) propagation of the Table 1 SDs on the log scale,
    treating WT and mutant as independent: sd(ratio) ~ ratio * sqrt(cv_wt^2 + cv_mut^2).
    With n = 3 per estimate this is a rough guide only."""
    (w, sw), (m, sm) = T1[t][nt]["WT"], T1[t][nt][mut]
    r = w / m
    return r, r * np.hypot(sw / w, sm / m)


def rel(t, mut):
    """Fold loss for 8-oxo-dGTP divided by fold loss for the correct dNTP in the same template
    context (the comparison the paper itself makes, e.g. 7.6-fold vs 2.8-fold for A510D at dC).
    Approximate SD by the same first-order propagation over the four Table 1 entries."""
    r8, s8 = fold(t, "8oxo", mut)
    rn, sn = fold(t, NORMAL[t], mut)
    r = r8 / rn
    return r, r * np.hypot(s8 / r8, sn / rn)


# ------------------------------------------------------------------ 1. fold reductions
print("1. Fold reduction in kcat/KM vs WT (Table 1), with text values")
print(f"   {'mutant':12} {'template':8} {'dNTP':10} {'reported':>5} {'recomputed':>11} {'approx SD':>10}")
F = {}
for (mut, t, nt), rep in REPORTED.items():
    r, s = fold(t, nt, mut)
    F[(mut, t, nt)] = (r, s)
    print(f"   {mut:12} d{t:7} {('8-oxo-dGTP' if nt == '8oxo' else nt):10} {rep:>5} {r:11.2f} {s:10.2f}")
for t in "AC":
    r, s = fold(t, "8oxo", "A510D/N513A")
    F[("A510D/N513A", t, "8oxo")] = (r, s)
    print(f"   {'A510D/N513A':12} d{t:7} {'8-oxo-dGTP':10} {'-':>5} {r:11.2f} {s:10.2f}")

# ------------------------------------------------------------------ 2. WT comparisons
print("\n2. WT comparisons in the Results text")
wt = lambda t, nt: T1[t][nt]["WT"][0]  # noqa: E731
print(f"   dGMP vs 8-oxo-dGMP opposite dC: reported 150-fold, recomputed {wt('C','dGTP')/wt('C','8oxo'):.2f}")
print(f"   dTMP vs 8-oxo-dGMP opposite dA: reported 13-fold,  recomputed {wt('A','dTTP')/wt('A','8oxo'):.2f}")
print(f"   dA/dC selectivity for 8-oxo-dGTP: reported 5:1, recomputed {wt('A','8oxo')/wt('C','8oxo'):.2f}:1")

# ------------------------------------------------------------------ 3. the pairing, absolute metric
print("\n3. Metric 1 (absolute): fold loss of kcat/KM for 8-oxo-dGTP vs WT, dA vs dC")
pair_abs = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
    a, sa = F[(mut, "A", "8oxo")]
    c, sc = F[(mut, "C", "8oxo")]
    pair_abs[res] = "dA" if a > c else "dC"
    # log-scale difference in units of its approximate SD
    z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
    print(f"   {res} ({mut}): dA {a:.2f}x (SD ~{sa:.2f}) vs dC {c:.2f}x (SD ~{sc:.2f}); "
          f"ratio {max(a, c)/min(a, c):.2f}; larger at {pair_abs[res]}; |log diff|/SD ~ {z:.1f}")
print(f"   Larger effect: Ala510 -> {pair_abs['Ala510']}, Asn513 -> {pair_abs['Asn513']}")

# ------------------------------------------------------------------ 4. the pairing, relative metric
print("\n4. Metric 2 (relative): 8-oxo-dGTP fold loss / correct-dNTP fold loss in the same context")
print("   (dTTP opposite dA, dGTP opposite dC; the comparison the paper makes for each mutant)")
pair_rel = {}
REL = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
    (a, sa), (c, sc) = rel("A", mut), rel("C", mut)
    REL[(mut, "A")], REL[(mut, "C")] = (a, sa), (c, sc)
    z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
    pair_rel[res] = ("dA" if a > c else "dC") if z >= 1 else "about equal"
    print(f"   {res} ({mut}): dA {a:.2f} (SD ~{sa:.2f}) vs dC {c:.2f} (SD ~{sc:.2f}); "
          f"ratio {max(a, c)/min(a, c):.2f}; larger: {pair_rel[res]}; |log diff|/SD ~ {z:.1f}")
print(f"   Larger effect: Ala510 -> {pair_rel['Ala510']}, Asn513 -> {pair_rel['Asn513']}")

print("\n   Abstract:   Ala510 -> dA, Asn513 -> dC   (\"... opposite dA and dC, respectively\")")
print("   Paper text: Ala510 -> dC (anti), Asn513 -> dA (syn)")
print(f"   Ala510 abstract assignment supported by metric 1: {pair_abs['Ala510'] == 'dA'}, "
      f"by metric 2: {pair_rel['Ala510'] == 'dA'}")
print(f"   Asn513 abstract assignment supported by metric 1: {pair_abs['Asn513'] == 'dC'}, "
      f"by metric 2: {pair_rel['Asn513']} (point estimate larger at dC, |log diff|/SD < 1)")

# ------------------------------------------------------------------ 5. double mutant
print("\n5. Double mutant, 8-oxo-dGTP (Results: 'nearly identical' for dA and dC)")
print(f"   opposite dA {F[('A510D/N513A','A','8oxo')][0]:.1f}x, opposite dC {F[('A510D/N513A','C','8oxo')][0]:.1f}x")

# ================================================================== figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, BOX = "#eb6834", "#2a78d6", "#e4e3df", "#c9c8c3"
GREY = "#7a7975"  # Results-printed values
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 11, "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
    "axes.edgecolor": BOX, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
a510 = {t: F[("A510D", t, "8oxo")] for t in "AC"}
n513 = {t: F[("N513A", t, "8oxo")] for t in "AC"}


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def panel(ax, x, y, w, h, title):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                fc="white", ec=BOX, lw=1.2))
    ax.text(x + 0.025, y + h - 0.035, title, fontsize=11.5, weight="bold", color=SEC, va="top")


def wrap(s, n):
    return "\n".join(textwrap.wrap(s, n))


# ---------- candidate 1: two text panels joined by a connector (also fig1)
def cand1(path):
    H = 7.4
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    rend = fig.canvas.get_renderer()

    def height(t):  # height of a drawn text object in axes fraction
        bb = t.get_window_extent(renderer=rend)
        return bb.height / (fig.dpi * H)

    ax.text(0.03, 0.975, "Which residue facilitates 8-oxo-dGMP incorporation opposite dA, which opposite dC?",
            fontsize=10.5, weight="bold", va="top")
    top, bot = 0.93, 0.335
    # left: abstract
    panel(ax, 0.03, bot, 0.43, top - bot, "Abstract")
    ax.text(0.055, 0.87, wrap("\u201c" + ABSTRACT + "\u201d", 36), fontsize=10.5, va="top", linespacing=1.4)
    ax.text(0.055, 0.465, "Pairing as written (\u201crespectively\u201d):", fontsize=10, color=SEC, va="top")
    ax.text(0.055, 0.425, "Ala510  \u2192  opposite dA", fontsize=12, weight="bold", color=ORANGE, va="top")
    ax.text(0.055, 0.38, "Asn513  \u2192  opposite dC", fontsize=12, weight="bold", color=ORANGE, va="top")
    # right: body text
    panel(ax, 0.54, bot, 0.43, top - bot, "Results and Conclusion")
    y = 0.87
    for lab, q in (("Results, N513A paragraph:", Q_N513), ("Conclusion:", Q_A510),
                   ("Results, double-mutant paragraph:", Q_DBL)):
        t = ax.text(0.565, y, lab, fontsize=10, color=SEC, va="top")
        y -= height(t) + 0.006
        t = ax.text(0.565, y, wrap("\u201c\u2026" + q + "\u201d", 37), fontsize=10, va="top", linespacing=1.3)
        y -= height(t) + 0.018
    assert y > 0.48, y
    ax.text(0.565, 0.465, "Pairing in the body text:", fontsize=10, color=SEC, va="top")
    ax.text(0.565, 0.425, "Asn513  \u2192  opposite dA (syn)", fontsize=12, weight="bold", color=TXT, va="top")
    ax.text(0.565, 0.38, "Ala510  \u2192  opposite dC (anti)", fontsize=12, weight="bold", color=TXT, va="top")
    ax.add_patch(FancyArrowPatch((0.445, 0.40), (0.555, 0.40), arrowstyle="<->", mutation_scale=14,
                                 lw=2, color=SEC))
    ax.text(0.50, 0.385, "pairing\nreversed", ha="center", va="top", fontsize=10, style="italic", color=SEC)
    # bottom strip: Table 1, both metrics
    ax.text(0.03, 0.295, "Table 1, recomputed: loss of kcat/KM for 8-oxo-dGTP vs WT", fontsize=10.5,
            weight="bold", va="top")
    rows = [("absolute fold loss", a510, n513, "{:.1f}\u00d7"),
            ("\u00f7 loss for correct dNTP", {t: REL[("A510D", t)] for t in "AC"},
             {t: REL[("N513A", t)] for t in "AC"}, "{:.2f}")]
    ax.text(0.52, 0.250, "Ala510 (A510D)", fontsize=10, color=SEC, va="top", ha="center")
    ax.text(0.82, 0.250, "Asn513 (N513A)", fontsize=10, color=SEC, va="top", ha="center")
    for k, (lab, A, N, fmt) in enumerate(rows):
        yy = 0.210 - k * 0.042
        ax.text(0.03, yy, lab, fontsize=10, va="top")
        ax.text(0.52, yy, f"dA {fmt.format(A['A'][0])}  vs  dC {fmt.format(A['C'][0])}", fontsize=10.5,
                weight="bold", color=BLUE, va="top", ha="center")
        ax.text(0.82, yy, f"dA {fmt.format(N['A'][0])}  vs  dC {fmt.format(N['C'][0])}", fontsize=10.5,
                weight="bold", color=BLUE, va="top", ha="center")
    ax.text(0.03, 0.112, "Correct dNTP: dTTP opposite dA, dGTP opposite dC. The Results text prints the same\n"
            "absolute values, rounded: 5.1 and 7.6-fold (A510D), 25 and 6.4-fold (N513A).",
            fontsize=10, color=SEC, va="top", linespacing=1.35)
    ax.text(0.03, 0.040, "orange = pairing stated in the abstract;   blue = recomputed from Table 1 by recompute.py.txt",
            fontsize=10, color=SEC, va="top")
    save(fig, path)


# ---------- candidate 2: two side-by-side bipartite mappings
def cand2(path):
    fig = plt.figure(figsize=(W, 4.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    ax.text(0.03, 0.96, "Residue-to-template pairing for 8-oxo-dGMP incorporation", fontsize=12, weight="bold", va="top")

    def block(x0, span, title, edges, color, ls):
        ax.text(x0 + (span + 0.12) / 2, 0.85, title, ha="center", fontsize=11, weight="bold", color=SEC)
        L = {"Ala510": 0.70, "Asn513": 0.34}
        R = {"dA": 0.70, "dC": 0.34}
        for k, y in L.items():
            ax.text(x0 + 0.05, y, k, ha="center", va="center", fontsize=11.5, weight="bold",
                    bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
        rl = {"dA": "opposite dA\n(syn)", "dC": "opposite dC\n(anti)"}
        for k, y in R.items():
            ax.text(x0 + span + 0.07, y, rl[k], ha="center", va="center", fontsize=10.5,
                    bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
        for (l, r, lab, rep_lab) in edges:
            x1, y1, x2, y2 = x0 + 0.105, L[l], x0 + span, R[r]
            ax.plot([x1, x2], [y1, y2], color=color, lw=2.2, ls=ls)
            if not lab:
                continue
            frac = 0.25
            xm, ym = x1 + frac * (x2 - x1), y1 + frac * (y2 - y1)
            ax.text(xm, ym, lab + "\n" + rep_lab, ha="center", va="center", fontsize=10.5, color=BG,
                    linespacing=1.3, bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=color, lw=1), zorder=5)
            ax.text(xm, ym + 0.026, lab, ha="center", va="center", fontsize=10.5, color=color, weight="bold", zorder=6)
            ax.text(xm, ym - 0.026, rep_lab, ha="center", va="center", fontsize=10, color=GREY, zorder=6)

    block(0.03, 0.26, "Abstract (“respectively”)",
          [("Ala510", "dA", "", ""), ("Asn513", "dC", "", "")], ORANGE, "--")
    block(0.49, 0.36, "Table 1: larger absolute fold loss",
          [("Ala510", "dC", f"{a510['C'][0]:.2f}×", f"Results: {REPORTED[('A510D', 'C', '8oxo')]}"),
           ("Asn513", "dA", f"{n513['A'][0]:.2f}×", f"Results: {REPORTED[('N513A', 'A', '8oxo')]}")],
          BLUE, "-")
    ax.plot([0.46, 0.46], [0.24, 0.82], color=BOX, lw=1)
    ax.text(0.03, 0.185, f"Smaller effects: A510D {a510['A'][0]:.2f}× opposite dA; N513A {n513['C'][0]:.2f}× opposite dC "
            f"(Results: 5.1 and 6.4).", fontsize=10, color=SEC)
    ax.text(0.03, 0.135, f"Relative to the correct dNTP: A510D {REL[('A510D','A')][0]:.2f} (dA) vs "
            f"{REL[('A510D','C')][0]:.2f} (dC); N513A {REL[('N513A','A')][0]:.2f} (dA) vs {REL[('N513A','C')][0]:.2f} (dC).",
            fontsize=10, color=SEC)
    ax.legend(handles=[Line2D([], [], color=ORANGE, ls="--", lw=2, label="pairing stated in the abstract"),
                       Line2D([], [], color=BLUE, lw=2, label="recomputed from Table 1"),
                       Line2D([], [], color=GREY, lw=0, marker="s", ms=8, label="printed in the Results")],
              loc="lower center", bbox_to_anchor=(0.5, 0.01), ncol=3, frameon=False, fontsize=10,
              columnspacing=1.0, handlelength=1.6, handletextpad=0.5)
    save(fig, path)


# ---------- candidate 3: fold reductions on one log axis, two metrics, approximate SDs (also fig2)
def cand3(path):
    fig, ax = plt.subplots(figsize=(W, 4.6))
    fig.subplots_adjust(left=0.27, right=0.97, top=0.82, bottom=0.27)
    rows = [("N513A", "C"), ("N513A", "A"), ("A510D", "C"), ("A510D", "A")]
    labs = {("A510D", "A"): "A510D, opposite dA", ("A510D", "C"): "A510D, opposite dC",
            ("N513A", "A"): "N513A, opposite dA", ("N513A", "C"): "N513A, opposite dC"}
    abstract_rows = {("A510D", "A"): "abstract: Ala510 → dA", ("N513A", "C"): "abstract: Asn513 → dC"}
    off = 0.17

    def bar(r, s, y, marker, filled):
        lo, hi = np.exp(np.log(r) - s / r), np.exp(np.log(r) + s / r)
        ax.plot([lo, hi], [y, y], color=BLUE, lw=2)
        ax.plot(r, y, marker, ms=8, color=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
        return hi

    for i, (m, t) in enumerate(rows):
        r, s = F[(m, t, "8oxo")]
        hi = bar(r, s, i + off, "D", True)
        ax.plot(REPORTED[(m, t, "8oxo")], i + off, "o", ms=14, mfc="none", mec=GREY, mew=1.6, zorder=4)
        ax.text(hi * 1.08, i + off, f"{r:.1f}×", va="center", fontsize=10, color=BLUE)
        r2, s2 = REL[(m, t)]
        hi2 = bar(r2, s2, i - off, "s", False)
        ax.text(hi2 * 1.08, i - off, f"{r2:.2f}", va="center", fontsize=10, color=BLUE)
        if (m, t) in abstract_rows:
            ax.text(58, i, abstract_rows[(m, t)], ha="right", va="center", fontsize=10, color=ORANGE,
                    weight="bold")
    ax.set_yticks(range(4)); ax.set_yticklabels([labs[k] for k in rows], fontsize=10.5, color=TXT)
    ax.set_xscale("log"); ax.set_xlim(0.8, 60)
    ax.set_xticks([1, 2, 5, 10, 20, 50]); ax.set_xticklabels(["1", "2", "5", "10", "20", "50"])
    ax.axvline(1, color=SEC, lw=0.8)
    ax.axhline(1.5, color=BOX, lw=1)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ("top", "right"):
        ax.spines[sp].set_visible(False)
    ax.set_xlabel("Fold loss of kcat/KM vs WT (log scale; 1 = no effect)", fontsize=10.5)
    ax.set_ylim(-0.55, 3.55)
    fig.text(0.02, 0.97, "8-oxo-dGTP: effect of each substitution opposite dA and opposite dC", fontsize=11.5,
             weight="bold", va="top")
    fig.text(0.02, 0.915, wrap("Absolute loss is larger at dC for A510D and at dA for N513A. Relative to the "
                               "correct dNTP, A510D still favours dC; N513A is about equal.", 95),
             fontsize=10, color=SEC, va="top")
    ax.legend(handles=[Line2D([], [], marker="D", ls="-", lw=2, ms=8, color=BLUE,
                              label="absolute fold loss, Table 1 (±1 approx. SD)"),
                       Line2D([], [], marker="s", ls="-", lw=2, ms=8, color=BLUE, mfc=BG, mew=2,
                              label="÷ fold loss for the correct dNTP, Table 1"),
                       Line2D([], [], marker="o", ls="none", ms=12, mfc="none", mec=GREY, mew=1.6,
                              label="absolute value printed in the Results"),
                       Line2D([], [], ls="none", marker="$→$", ms=10, color=ORANGE,
                              label="pairing stated in the abstract")],
              loc="upper center", bbox_to_anchor=(0.36, -0.17), ncol=2, frameon=False, fontsize=10,
              columnspacing=1.2, handletextpad=0.5)
    save(fig, path)


# ---------- candidate 4: 2 x 2 grid with outlined cells
def cand4(path):
    fig = plt.figure(figsize=(W, 4.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    ax.text(0.03, 0.96, "Effect on 8-oxo-dGTP incorporation, by residue and template",
            fontsize=12, weight="bold", va="top")
    x0, y0, cw, ch = 0.30, 0.20, 0.30, 0.27
    cols, rowsr = ["A", "C"], [("A510D", "Ala510 (A510D)"), ("N513A", "Asn513 (N513A)")]
    abstract_cells = {("A510D", "A"), ("N513A", "C")}
    for j, t in enumerate(cols):
        ax.text(x0 + (j + 0.5) * cw, y0 + 2 * ch + 0.04, f"opposite d{t}", ha="center", fontsize=11, weight="bold", color=SEC)
    for i, (m, lab) in enumerate(rowsr):
        y = y0 + (1 - i) * ch
        ax.text(x0 - 0.02, y + ch / 2, lab, ha="right", va="center", fontsize=11, weight="bold")
        vals = {t: F[(m, t, "8oxo")][0] for t in cols}
        for j, t in enumerate(cols):
            x = x0 + j * cw
            ax.add_patch(Rectangle((x, y), cw, ch, fc="white", ec=BOX, lw=1))
            ax.text(x + cw / 2, y + ch * 0.68, f"{vals[t]:.1f}×", ha="center", va="center",
                    fontsize=16, weight="bold", color=BLUE)
            ax.text(x + cw / 2, y + ch * 0.44, f"÷ correct dNTP: {REL[(m, t)][0]:.2f}",
                    ha="center", va="center", fontsize=10, color=BLUE)
            ax.text(x + cw / 2, y + ch * 0.23, f"Results text: {REPORTED[(m, t, '8oxo')]}-fold",
                    ha="center", va="center", fontsize=10, color=GREY)
            if vals[t] == max(vals.values()):
                ax.add_patch(Rectangle((x + 0.008, y + 0.012), cw - 0.016, ch - 0.024, fc="none", ec=BLUE, lw=2.5))
            if (m, t) in abstract_cells:
                ax.add_patch(Rectangle((x + 0.022, y + 0.030), cw - 0.044, ch - 0.060, fc="none", ec=ORANGE,
                                       lw=2, ls="--"))
    ax.text(0.03, 0.135, "Large number: absolute fold loss of kcat/KM vs WT. Second line: that loss divided by the "
            "loss\nfor the correct dNTP (dTTP at dA, dGTP at dC). Both recomputed from Table 1.",
            fontsize=10, color=SEC, va="center")
    ax.legend(handles=[Rectangle((0, 0), 1, 1, fc="none", ec=BLUE, lw=2.5, label="larger absolute loss per residue"),
                       Rectangle((0, 0), 1, 1, fc="none", ec=ORANGE, lw=2, ls="--", label="pairing stated in the abstract")],
              loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
    save(fig, path)


# ---------- candidate 5: slope chart dA -> dC
def cand5(path):
    fig, ax = plt.subplots(figsize=(W, 5.0))
    fig.subplots_adjust(left=0.13, right=0.62, top=0.89, bottom=0.30)
    xs = [0, 1]
    for m, res, ls, mk in (("N513A", "Asn513 (N513A)", "-", "o"), ("A510D", "Ala510 (A510D)", "--", "s")):
        v = [F[(m, t, "8oxo")][0] for t in "AC"]
        ax.plot(xs, v, ls=ls, marker=mk, ms=8, lw=2, color=BLUE, zorder=3)
        for x, t, val in zip(xs, "AC", v):
            ax.plot(x, REPORTED[(m, t, "8oxo")], "o", ms=17, mfc="none", mec=GREY, mew=1.6, zorder=4)
            dy = 1.0 if x == 0 else (1.13 if val == max(F[(mm, "C", "8oxo")][0] for mm in ("A510D", "N513A")) else 0.88)
            ax.text(x + (0.09 if x else -0.09), val * dy, f"{val:.2f}×", ha="left" if x else "right",
                    va="center", fontsize=10, color=BLUE)
        mid = np.sqrt(v[0] * v[1])
        ax.text(0.5, mid * (1.75 if m == "N513A" else 0.68), res, ha="center", va="center",
                fontsize=10.5, weight="bold")
    ax.set_xticks(xs); ax.set_xticklabels(["opposite dA\n(syn)", "opposite dC\n(anti)"], fontsize=10.5, color=TXT)
    ax.set_xlim(-0.35, 1.3)
    ax.set_yscale("log"); ax.set_ylim(1, 40)
    ax.set_yticks([1, 2, 5, 10, 20]); ax.set_yticklabels(["1", "2", "5", "10", "20"])
    ax.set_ylabel("Absolute fold loss of kcat/KM,\n8-oxo-dGTP (log scale; 1 = no effect)", fontsize=10.5)
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ("top", "right"):
        ax.spines[sp].set_visible(False)
    ax.set_title("Each residue's effect in the two template contexts", fontsize=11.5, weight="bold", loc="left")
    fig.text(0.67, 0.83, "Table 1, absolute loss:", fontsize=10.5, weight="bold", color=BLUE)
    fig.text(0.67, 0.79, "Asn513 larger opposite dA\nAla510 larger opposite dC", fontsize=10.5, color=BLUE, va="top")
    fig.text(0.67, 0.66, "÷ correct-dNTP loss:", fontsize=10.5, weight="bold", color=BLUE)
    fig.text(0.67, 0.62, f"Ala510 {REL[('A510D','A')][0]:.2f} (dA), {REL[('A510D','C')][0]:.2f} (dC)\n"
             f"Asn513 {REL[('N513A','A')][0]:.2f} (dA), {REL[('N513A','C')][0]:.2f} (dC)",
             fontsize=10.5, color=BLUE, va="top")
    fig.text(0.67, 0.47, "Abstract states:", fontsize=10.5, weight="bold", color=ORANGE)
    fig.text(0.67, 0.43, "Ala510 → dA\nAsn513 → dC", fontsize=10.5, color=ORANGE, va="top")
    ax.legend(handles=[Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
                              label="Asn513 (N513A), recomputed from Table 1"),
                       Line2D([], [], marker="s", ls="--", lw=2, ms=8, color=BLUE,
                              label="Ala510 (A510D), recomputed from Table 1"),
                       Line2D([], [], marker="o", ls="none", ms=13, mfc="none", mec=GREY, mew=1.6,
                              label="value printed in the Results")],
              loc="upper left", bbox_to_anchor=(-0.05, -0.17), ncol=1, frameon=False, fontsize=10)
    save(fig, path)


for i, f in enumerate((cand1, cand2, cand3, cand4, cand5), 1):
    f(os.path.join(CAND, f"cand{i}"))
cand1(os.path.join(HERE, "fig1"))
cand3(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)

Abstract's age pattern for all STIs except HSV-2 is contradicted by the paper's syphilis estimates

PLOS Medicine (2018) · doi:10.1371/journal.pmed.1002511 · PMC5828349

Summary

This individual participant data meta-analysis of 18 HIV prevention studies reports STI prevalence among women in sub-Saharan Africa by age, region and population type. The abstract states that, in all three primary region/population groups, prevalence was higher among 15–24-year-old than 25–49-year-old women "for all STIs except HSV-2". The author summary repeats this, adding "regardless of region or population type". The paper's own syphilis estimates do not show this pattern. In South African clinic/community populations, overall syphilis prevalence was 4.3% (95% CI 2.5–7.2) at ages 25–49 and 1.5% (1.0–2.2) at 15–24. The intervals do not overlap, and the Results text itself says the older group's prevalence "was higher". In the other two groups, prevalence was higher or similar in older women. High-titer syphilis in South Africa (1.0% vs 0.4%) and trichomoniasis in South Africa (8.6% vs 7.9%) also have higher point estimates in older women. The age pattern does hold for chlamydia and gonorrhea. As written, though, the headline statement misdescribes syphilis, the STI most directly tied to antenatal screening and congenital-syphilis programs.

What the paper reports

Abstract, "Methods and findings". The sentence before it reads "Despite substantial study heterogeneity for some STIs/populations, several patterns emerged."

Across the three primary region/population groups (South Africa community-based, Southern/Eastern Africa community-based, and Eastern Africa higher-risk), prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2.

Author summary, "What did the researchers do and find?":

STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type.

Discussion, opening paragraph:

We observed age-, region- and population-specific patterns in the prevalence of these infections, including higher prevalence of STIs among younger (15–24-year-old) than older (25–49-year-old) women, …

Discussion, the paragraph that links the findings to the WHO Global Strategy for STIs. Unlike the abstract, this sentence says "generally":

In terms of the “who,” across all populations and regions, STI prevalence was generally higher among younger than older women, except for the prevalence of antibodies to HSV-2, which increased with age.

Results, "Syphilis, overall (high, low, and unknown titer infection)", first paragraph:

Among 15–24-year-old women, the summary estimate of syphilis prevalence overall (i.e., proportion of participants with a positive RPR and positive treponemal test) was less than 2% in clinic/community populations in both South Africa (1.5% [95% CI: 1.0%, 2.2%]; I2 = 57.7%) and Southern/Eastern Africa (1.4% [95% CI: 1.0%, 1.9%]; I2 = 25.8%), and higher among higher-risk populations in Eastern Africa (8.4% [95% CI: 6.8%, 10.3%]; I2 = 46.3%) (Fig 5).

Results, same section, second paragraph:

Among 25–49-year-olds in clinic/community populations in South Africa, the summary estimate of syphilis prevalence overall among 25–49-year-old women (4.3% [95% CI: 2.5%, 7.2%]; I2 = 94.8%) was higher than that among younger women (1.5%) (Fig 6). In clinic/community populations in Southern/Eastern Africa, the summary estimate of syphilis prevalence overall (2.0% [95% CI: 1.4%, 2.6%]; I2 = 61.8%) was similar to the summary estimate among younger women (1.4%). In the higher-risk populations in Eastern Africa, the summary estimate among older women (7.5% [95% CI: 5.5%, 10.1%]; I2 = 85.8%) was similar to prevalence among younger women (8.4%).

Results, "Syphilis, high-titer infection":

Point estimates for women aged 25–49 years were similar to those in younger populations, but were more heterogeneous (Fig 8).

The "Overall prevalence" rows of Table 4 (15–24 years) and Table 5 (25–49 years), South African clinic/community populations, as archived in PMC:

Table 4: Syphilis, high-titer infection — Overall prevalence: 0.4% (0.3, 0.5); Trichomoniasis — Overall prevalence: 7.9% (5.9, 10.5)

Table 5: Syphilis, high-titer infection — Overall prevalence: 1.0% (0.5, 2.1); Trichomoniasis — Overall prevalence: 8.6% (6.5, 11.5)

Methods, "Statistical analysis". This describes how the strata were compared. No formal test between age groups is described.

We generated forest plots in R and inspected them visually to compare prevalence estimates across strata, and combined strata if estimates were similar.

What the paper's own numbers show

Method. I collected the paper's 15–24 and 25–49 summary prevalence estimates, with their 95% CIs, for each STI in each of the three primary groups. Overall syphilis comes from the Results text, which reports Figs 5 and 6 in numbers. Every other infection comes from the "Overall prevalence" row of Tables 4–9. Where the Results text also states a value, the table rows agree with it: chlamydia SA 15–24, trichomoniasis SA 25–49, trichomoniasis E Africa 15–24 and high-titer syphilis SA 15–24 match exactly, and HSV-2 SA 15–24 differs by 0.1 in one CI bound.

The paper pools study log odds with a random-effects model and back-transforms the pooled estimate and its CI. Its CIs are therefore symmetric on the logit scale, so the standard error of each pooled log odds can be recovered as (logit(upper) − logit(lower)) / 3.92. I then computed the older-versus-younger odds ratio, its 95% CI and an approximate z statistic, treating the two age groups' pooled estimates as independent, since they come from different women. This is a descriptive check of direction and size using only the published numbers. It is not a re-analysis of the participant data. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Direction of the age difference, all STIs except HSV-2, three primary groups Abstract; Author summary higher in 15–24 than 25–49 in every group 9 of 12 infection × group cells (chlamydia, gonorrhea, overall syphilis, trichomoniasis) have the higher point estimate in 15–24, and 3 do not
Overall syphilis, SA clinic/community Results text (Figs 5, 6) 15–24: 1.5% (1.0, 2.2); 25–49: 4.3% (2.5, 7.2), "higher than that among younger women" ratio 2.87; CIs do not overlap (2.5 > 2.2); OR 2.95 (1.49–5.84), z = +3.10, p ≈ 0.002, older higher
Overall syphilis, S/E Africa clinic/community Results text 15–24: 1.4% (1.0, 1.9); 25–49: 2.0% (1.4, 2.6), "similar" OR 1.44 (0.91–2.26), z = +1.57, p ≈ 0.12; older point estimate higher, not significant
Overall syphilis, E Africa higher-risk Results text 15–24: 8.4% (6.8, 10.3); 25–49: 7.5% (5.5, 10.1), "similar" OR 0.88 (0.59–1.32), z = −0.60, p ≈ 0.55; no clear difference
High-titer syphilis, SA clinic/community Table 4 vs Table 5 15–24: 0.4% (0.3, 0.5); 25–49: 1.0% (0.5, 2.1); text: point estimates "similar" ratio 2.50; CIs touch at 0.5; OR 2.52 (1.16–5.43), z = +2.35, p ≈ 0.02, older higher
High-titer syphilis, S/E Africa; E Africa higher-risk Tables 6/7, 8/9 1.1% vs 0.9%; 4.5% vs 2.8% OR 0.82 (0.20–3.39); OR 0.61 (0.38–0.97)
Trichomoniasis, SA clinic/community Table 4 vs Table 5 15–24: 7.9% (5.9, 10.5); 25–49: 8.6% (6.5, 11.5) OR 1.10 (0.70–1.71), z = +0.41, p ≈ 0.68; older point estimate slightly higher, no clear difference
Trichomoniasis, S/E Africa; E Africa higher-risk Tables 6/7, 8/9 6.7% vs 5.8%; 12.7% vs 10.0% OR 0.86 (0.47–1.57); OR 0.76 (0.33–1.78); younger point estimate higher, not significant
Chlamydia, three groups Tables 4–9; Results 15.1 vs 7.0; 2.7 vs 1.2; 10.3 vs 6.2 younger higher in 3/3; OR 0.42, 0.44, 0.58, all with p < 0.05
Gonorrhea, three groups Tables 4–9; Results 4.6 vs 2.5; 1.7 vs 0.9; 8.2 vs 5.2 younger higher in 3/3; OR 0.53, 0.53, 0.61 (p ≈ 0.03, 0.07, 0.05)
HSV-2, three groups (excluded by the abstract) Tables 4–9 39.3 vs 77.8; 46.8 vs 70.0; 56.3 vs 83.3 older higher in 3/3, as the paper says

Of the 12 non-HSV-2 cells, younger women's prevalence is higher with approximate p < 0.05 in 4 cells (all chlamydia, plus gonorrhea in South Africa). Older women's prevalence is higher with p < 0.05 in 1 cell, overall syphilis in South Africa. That cell has non-overlapping CIs, so the paper's own numbers support the reverse of the abstract statement there, as the paper's own Results text says. Across the three groups, syphilis does not show the pattern the abstract describes: the difference is reversed in South Africa, the point estimate is higher in older women in Southern/Eastern Africa, and the estimates are similar in Eastern Africa.

A separate, minor slip turned up in the Results trichomoniasis paragraph: "among 25–29-year-olds in the South African clinic/community-based populations … (summary estimate: 8.6% …)". The value matches the 25–49 row of Table 5, so "25–29" appears to be a typographical error for "25–49".

Figure

The abstract's age statement next to the Results text on overall syphilis

Figure 1. On the left, the abstract and author-summary statements, quoted verbatim. On the right, the paper's Results text on overall syphilis (Figs 5 and 6), with the 95% CIs as printed. The connector marks the conflict. In South African clinic/community women, the paper reports 4.3% (2.5–7.2) at 25–49 and 1.5% (1.0–2.2) at 15–24, and its Results text calls the older group's prevalence higher. The orange-marked line restates the reported interval bounds, which do not overlap. The blue-marked line is the odds ratio recomputed from those reported CIs on the logit scale: 2.95 (1.49–5.84), z = +3.10. The abstract statement implies an odds ratio below 1.

Older-versus-younger odds ratios recomputed for every STI and group the abstract statement covers

Figure 2. For each STI the abstract statement covers (all except HSV-2) and each of the three primary groups, the blue dot and line are the odds ratio, 25–49 versus 15–24 years, with its 95% CI. Each is recomputed from the paper's reported summary prevalence and 95% CI (overall syphilis from the Results text; other infections from the "Overall prevalence" rows of Tables 4–9). The axis is logarithmic, and the thin vertical line marks OR = 1 (no age difference). The shaded orange band is where the abstract statement says every point estimate should lie (younger higher). Chlamydia and gonorrhea lie inside it. Overall syphilis in South Africa lies entirely outside it, and so does high-titer syphilis in South Africa. Overall syphilis in Southern/Eastern Africa and trichomoniasis in South Africa have point estimates on the "older higher" side.

Why it matters

The age statement is one of the few cross-cutting patterns the abstract reports. It is the first finding listed after "several patterns emerged". The author summary repeats it in stronger form ("regardless of region or population type"), and the Discussion uses it to answer the "who" of the WHO Global Health Sector Strategy on STIs. Abstracts and author summaries are what most readers, guideline writers and secondary reviews take from a prevalence paper. A reader relying on them would conclude that syphilis, like chlamydia and gonorrhea, is concentrated in women under 25 in all three settings. The paper's own estimates show that it is not. In South African clinic/community populations, syphilis prevalence is roughly three times higher in women aged 25–49. Elsewhere it is higher or similar in older women.

For syphilis this difference has practical weight. Antenatal syphilis screening and congenital-syphilis elimination programs cover women of all reproductive ages, so where seroprevalence peaks matters for targeting. The underlying estimates, the Results text and the forest plots appear to be correct and internally consistent. The error is in how they are summarized. The paper's other conclusions, on chlamydia and gonorrhea age gradients, regional differences, higher-risk populations, and HSV-2 and BV burden, are not affected.

Explanations considered

Status

Suggested correction

  1. Abstract: replace "prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2" with a statement that matches the Results. For example: "chlamydia and gonorrhea prevalence was higher among 15–24-year-old than 25–49-year-old women in all three groups; syphilis prevalence was similar or higher among 25–49-year-olds (e.g., 4.3% vs 1.5% in South African clinic/community populations); HSV-2 prevalence increased with age."
  2. Author summary: revise "STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type" in the same way, naming syphilis (and HSV-2) as exceptions.
  3. Discussion: qualify the opening summary and the "who" paragraph to state that syphilis did not follow the younger-higher pattern, notably in South African clinic/community populations.
  4. Results, "Syphilis, high-titer infection": consider stating the South African 25–49 estimate (1.0% [0.5, 2.1] vs 0.4% [0.3, 0.5]) rather than describing the point estimates as "similar".
  5. Minor: in the trichomoniasis paragraph, change "25–29-year-olds" to "25–49-year-olds".
Recomputation script (python recompute.py.txt)
"""Recompute the age comparisons behind the abstract's age statement in PMC5828349
(PLOS Medicine 2018, doi:10.1371/journal.pmed.1002511).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are the paper's own summary prevalence estimates, % (95% CI):
  * overall syphilis: Results text, "Syphilis, overall" paragraphs (Figs 5 and 6);
  * all other infections: the "Overall prevalence" row of the co-infection tables
    (Table 4/5 South Africa clinic/community 15-24/25-49, Table 6/7 Southern/Eastern
    Africa clinic/community, Table 8/9 Eastern Africa higher-risk).
The paper pools study log odds with a random-effects model and back-transforms the pooled
log odds and its 95% CI (Methods), so each CI is symmetric on the logit scale. The standard
error of each pooled log odds is therefore recovered as (logit(hi) - logit(lo)) / (2*1.96).
The 15-24 and 25-49 estimates come from different women, so the two pooled estimates are
treated as independent when comparing them (see writeup.md for the caveat).
"""
import os

import numpy as np
from scipy.stats import norm

import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)

GROUPS = ["South Africa clinic/community", "Southern/Eastern Africa clinic/community",
          "Eastern Africa higher-risk"]
SHORT = {GROUPS[0]: "SA clinic/community", GROUPS[1]: "S/E Africa clinic/community",
         GROUPS[2]: "E Africa higher-risk"}

# (infection, group): {age: (point, lo, hi)}, source
D = {
    ("Chlamydia", GROUPS[0]): ({"15-24": (15.1, 12.7, 17.8), "25-49": (7.0, 5.5, 8.8)}, "Tables 4/5"),
    ("Chlamydia", GROUPS[1]): ({"15-24": (2.7, 1.5, 3.9), "25-49": (1.2, 0.8, 1.7)}, "Tables 6/7"),
    ("Chlamydia", GROUPS[2]): ({"15-24": (10.3, 7.4, 14.1), "25-49": (6.2, 4.7, 8.2)}, "Tables 8/9"),
    ("Gonorrhea", GROUPS[0]): ({"15-24": (4.6, 3.3, 6.4), "25-49": (2.5, 1.6, 4.0)}, "Tables 4/5"),
    ("Gonorrhea", GROUPS[1]): ({"15-24": (1.7, 1.2, 2.6), "25-49": (0.9, 0.5, 1.6)}, "Tables 6/7"),
    ("Gonorrhea", GROUPS[2]): ({"15-24": (8.2, 5.8, 11.6), "25-49": (5.2, 3.9, 7.0)}, "Tables 8/9"),
    ("Syphilis, overall", GROUPS[0]): ({"15-24": (1.5, 1.0, 2.2), "25-49": (4.3, 2.5, 7.2)}, "Results text (Figs 5/6)"),
    ("Syphilis, overall", GROUPS[1]): ({"15-24": (1.4, 1.0, 1.9), "25-49": (2.0, 1.4, 2.6)}, "Results text (Figs 5/6)"),
    ("Syphilis, overall", GROUPS[2]): ({"15-24": (8.4, 6.8, 10.3), "25-49": (7.5, 5.5, 10.1)}, "Results text (Figs 5/6)"),
    ("Syphilis, high-titer", GROUPS[0]): ({"15-24": (0.4, 0.3, 0.5), "25-49": (1.0, 0.5, 2.1)}, "Tables 4/5"),
    ("Syphilis, high-titer", GROUPS[1]): ({"15-24": (1.1, 0.3, 1.9), "25-49": (0.9, 0.2, 1.7)}, "Tables 6/7"),
    ("Syphilis, high-titer", GROUPS[2]): ({"15-24": (4.5, 3.6, 5.5), "25-49": (2.8, 1.9, 4.2)}, "Tables 8/9"),
    ("Trichomoniasis", GROUPS[0]): ({"15-24": (7.9, 5.9, 10.5), "25-49": (8.6, 6.5, 11.5)}, "Tables 4/5"),
    ("Trichomoniasis", GROUPS[1]): ({"15-24": (6.7, 4.3, 10.1), "25-49": (5.8, 4.0, 8.4)}, "Tables 6/7"),
    ("Trichomoniasis", GROUPS[2]): ({"15-24": (12.7, 7.5, 20.6), "25-49": (10.0, 5.7, 17.1)}, "Tables 8/9"),
    ("HSV-2", GROUPS[0]): ({"15-24": (39.3, 34.4, 44.6), "25-49": (77.8, 75.6, 79.9)}, "Tables 4/5"),
    ("HSV-2", GROUPS[1]): ({"15-24": (46.8, 38.2, 55.6), "25-49": (70.0, 61.8, 77.1)}, "Tables 6/7"),
    ("HSV-2", GROUPS[2]): ({"15-24": (56.3, 49.2, 63.1), "25-49": (83.3, 79.3, 86.6)}, "Tables 8/9"),
}


def logit(p_pct):
    p = p_pct / 100.0
    return np.log(p / (1 - p))


def se_from_ci(lo, hi):
    return (logit(hi) - logit(lo)) / (2 * Z975)


def compare(v):
    a, b = v["15-24"], v["25-49"]
    se = np.hypot(se_from_ci(a[1], a[2]), se_from_ci(b[1], b[2]))
    d = logit(b[0]) - logit(a[0])
    z = d / se
    p = 2 * norm.sf(abs(z))
    orr, lo, hi = np.exp(d), np.exp(d - Z975 * se), np.exp(d + Z975 * se)
    overlap = not (a[2] < b[1] or b[2] < a[1])
    return dict(ratio=b[0] / a[0], OR=orr, lo=lo, hi=hi, z=z, p=p, overlap=overlap,
                younger_higher=a[0] > b[0])


# --------------------------------------------------------------- 1. all cells
print("=== 1. Older (25-49) vs younger (15-24) summary prevalence, every infection and group ===")
print("OR = odds ratio older/younger, recomputed from the reported estimates and CIs (logit scale)\n")
hdr = f"{'Infection':21s} {'Group':41s} {'15-24 %':>17s} {'25-49 %':>17s} {'ratio':>6s} {'OR (95% CI)':>20s} {'z':>6s} {'p':>7s}  CIs overlap  younger higher"
print(hdr)
R = {}
for (inf, grp), (v, src) in D.items():
    c = compare(v)
    R[(inf, grp)] = c
    a, b = v["15-24"], v["25-49"]
    print(f"{inf:21s} {grp:41s} {a[0]:5.1f} ({a[1]:.1f}, {a[2]:.1f}) {b[0]:5.1f} ({b[1]:.1f}, {b[2]:.1f}) "
          f"{c['ratio']:6.2f} {c['OR']:6.2f} ({c['lo']:.2f}, {c['hi']:.2f}) {c['z']:+6.2f} {c['p']:7.2g}  "
          f"{str(c['overlap']):11s}  {c['younger_higher']}")

# --------------------------------------------------------------- 2. the abstract statement
print("\n=== 2. Abstract: 'prevalence was higher among 15-24-year-old than 25-49-year-old women "
      "for all STIs except HSV-2' (three primary groups) ===")
main_inf = ["Chlamydia", "Gonorrhea", "Syphilis, overall", "Trichomoniasis"]
cells = [(i, g) for i in main_inf for g in GROUPS]
yh = [k for k in cells if R[k]["younger_higher"]]
print(f"Non-HSV-2 STIs x groups (overall syphilis): {len(cells)} cells; younger point estimate higher in {len(yh)}")
sig_y = [k for k in cells if R[k]["younger_higher"] and R[k]["p"] < 0.05]
sig_o = [k for k in cells if (not R[k]["younger_higher"]) and R[k]["p"] < 0.05]
print(f"  younger higher with approx. p < 0.05: {len(sig_y)}; older higher with approx. p < 0.05: {len(sig_o)}")
print("Cells where the 15-24 estimate is NOT higher (including high-titer syphilis):")
for k in cells + [("Syphilis, high-titer", g) for g in GROUPS]:
    c = R[k]
    if not c["younger_higher"]:
        v = D[k][0]
        print(f"  {k[0]:21s} {SHORT[k[1]]:28s} 15-24 {v['15-24'][0]}% vs 25-49 {v['25-49'][0]}%  "
              f"OR {c['OR']:.2f} ({c['lo']:.2f}-{c['hi']:.2f}), z = {c['z']:+.2f}, p = {c['p']:.3f}, "
              f"CIs overlap: {c['overlap']}")
for inf in ["Chlamydia", "Gonorrhea"]:
    print(f"{inf}: younger higher in {sum(R[(inf, g)]['younger_higher'] for g in GROUPS)}/3 groups")

k = ("Syphilis, overall", GROUPS[0])
v = D[k][0]
print(f"\nKey cell, overall syphilis, SA clinic/community: 25-49 lower CI bound {v['25-49'][1]} > "
      f"15-24 upper CI bound {v['15-24'][2]}: {v['25-49'][1] > v['15-24'][2]}")
print(f"  ratio of prevalences 4.3/1.5 = {v['25-49'][0] / v['15-24'][0]:.2f}")
print(f"  SE(logit) 15-24 = {se_from_ci(1.0, 2.2):.3f}, 25-49 = {se_from_ci(2.5, 7.2):.3f}")
k2 = ("Syphilis, high-titer", GROUPS[0])
v2 = D[k2][0]
print(f"High-titer syphilis, SA: 25-49 lower bound {v2['25-49'][1]} vs 15-24 upper bound {v2['15-24'][2]} "
      f"(touch: {v2['25-49'][1] == v2['15-24'][2]}); ratio {v2['25-49'][0] / v2['15-24'][0]:.2f}")

# --------------------------------------------------------------- 3. consistency of table rows with text
print("\n=== 3. Table 'Overall prevalence' rows vs Results text (same estimates?) ===")
checks = [
    ("Chlamydia SA 15-24", "Table 4", (15.1, 12.7, 17.8), "Results", (15.1, 12.7, 17.8)),
    ("Trichomoniasis SA 25-49", "Table 5", (8.6, 6.5, 11.5), "Results", (8.6, 6.5, 11.5)),
    ("Trichomoniasis E Africa HR 15-24", "Table 8", (12.7, 7.5, 20.6), "Results", (12.7, 7.5, 20.6)),
    ("High-titer syphilis SA 15-24", "Table 4", (0.4, 0.3, 0.5), "Results", (0.4, 0.3, 0.5)),
    ("HSV-2 SA 15-24", "Table 4", (39.3, 34.4, 44.6), "Results", (39.3, 34.3, 44.6)),
]
for lab, s1, x1, s2, x2 in checks:
    print(f"  {lab:34s} {s1}: {x1}  {s2}: {x2}  identical: {x1 == x2}")

# --------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


# ---- fig1: abstract vs results text panels
c = R[("Syphilis, overall", GROUPS[0])]
W, H = 7.2, 4.75
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")


def panel(x, y, w, h, title, body):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=INK2, lw=1.2))
    ax.text(x + 0.14, y + h - 0.16, title, fontsize=10.5, fontweight="bold", va="top", color=INK,
            linespacing=1.3)
    ax.text(x + 0.14, y + h - 0.78, body, fontsize=10, va="top", color=INK, linespacing=1.45)


PY, PH = 1.5, 3.15
panel(0.1, PY, 2.95, PH, "Abstract (Methods and findings)\nand Author summary",
      "“… prevalence was higher\namong 15–24-year-old than\n25–49-year-old women for\n"
      "all STIs except HSV-2.”\n(three primary groups)\n\n"
      "“STI prevalence was highest\namong young women aged\n15–24 years for all STIs except\n"
      "HSV-2, regardless of region\nor population type.”")
panel(4.15, PY, 2.95, PH, "Results, “Syphilis, overall”\n(Figs 5 and 6)",
      "South Africa clinic/community\n  15–24 y: 1.5% (1.0, 2.2)\n  25–49 y: 4.3% (2.5, 7.2)\n"
      "  “… was higher than that\n  among younger women”\n\n"
      "S/E Africa clinic/community\n  25–49 vs 15–24: 2.0% vs 1.4%\n"
      "E Africa higher-risk\n  25–49 vs 15–24: 7.5% vs 8.4%")
ym = PY + PH / 2 + 0.2
ax.annotate("", xy=(4.12, ym), xytext=(3.08, ym),
            arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.6, ym + 0.12, "contradicts", ha="center", va="bottom", fontsize=10, color=INK, fontweight="bold")
ax.text(3.6, ym - 0.12, "for syphilis\nin South\nAfrica", ha="center", va="top", fontsize=10, color=INK2)
# legend-style lines: reported (orange) and recomputed (blue)
ax.plot([0.12, 0.42], [1.08, 1.08], color=ORANGE, lw=2)
ax.plot([0.27], [1.08], "o", color=ORANGE, ms=8)
ax.text(0.55, 1.08, "Reported in the paper: SA 25–49 lower CI bound 2.5% > 15–24 upper CI\n"
        "bound 2.2%, so the two intervals do not overlap.", fontsize=10, color=INK, va="center")
ax.plot([0.12, 0.42], [0.48, 0.48], color=BLUE, lw=2)
ax.plot([0.27], [0.48], "o", color=BLUE, ms=8)
ax.text(0.55, 0.48, f"Recomputed from those CIs (logit scale): odds ratio 25–49 vs 15–24\n"
        f"= {c['OR']:.2f} (95% CI {c['lo']:.2f}–{c['hi']:.2f}), z = {c['z']:+.2f}, two-sided p ≈ "
        f"{c['p']:.3f}.", fontsize=10, color=INK, va="center")
save(fig, "fig1")

# ---- fig2: OR older/younger for every non-HSV-2 cell, log axis
order = ["Chlamydia", "Gonorrhea", "Trichomoniasis", "Syphilis, overall", "Syphilis, high-titer"]
fig, ax = plt.subplots(figsize=(7.2, 7.4), facecolor=BG)
ax.set_facecolor(BG)
ys, labels, rows, heads = [], [], [], []
y = 0.0
for inf in order:
    heads.append((y, inf))
    y -= 0.85
    for g in GROUPS:
        ys.append(y); labels.append(SHORT[g]); rows.append((inf, g))
        y -= 0.8
    y -= 0.35
ax.axvspan(0.1, 1, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(1, color=INK2, lw=0.8)
for yy, k in zip(ys, rows):
    c = R[k]
    ax.plot([c["lo"], c["hi"]], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
    ax.plot([c["OR"]], [yy], "o", color=BLUE, ms=8, mec=BG, mew=1.5)
    ax.text(1.02, yy, f"{c['OR']:.2f} ({c['lo']:.2f}–{c['hi']:.2f})", va="center",
            fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
for yy, inf in heads:
    ax.text(-0.02, yy, inf, ha="right", va="center", fontsize=10.5, fontweight="bold", color=INK,
            transform=ax.get_yaxis_transform())
ax.set_xscale("log")
ax.set_xlim(0.15, 7)
ax.set_xticks([0.2, 0.5, 1, 2, 5])
ax.set_xticklabels(["0.2", "0.5", "1", "2", "5"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10, color=INK)
top = 2.2
ax.set_ylim(min(ys) - 0.6, top)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s_ in ("top", "right", "left"):
    ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, 25–49 vs 15–24 years (log scale)", color=INK)
ax.text(0.95, top - 0.05, "← younger higher\n(abstract: every row\nbelongs in this band)", ha="right",
        va="top", fontsize=10, color=INK)
ax.text(1.05, top - 0.05, "older higher →", ha="left", va="top", fontsize=10, color=INK2)
ax.text(1.02, top - 0.05, "OR (95% CI)", ha="left", va="top", fontsize=10, color=INK, fontweight="bold",
        transform=ax.get_yaxis_transform())
h1, = ax.plot([], [], "o-", color=BLUE, lw=2, ms=8)
h2 = plt.Rectangle((0, 0), 1, 1, color=ORANGE, alpha=0.25, lw=0)
ax.legend([h1, h2], ["Recomputed from the paper's estimates and 95% CIs",
                     "Region implied by the abstract statement"],
          loc="upper left", bbox_to_anchor=(-0.55, -0.075), frameon=False, fontsize=10, ncol=1)
fig.subplots_adjust(left=0.36, right=0.77, top=0.98, bottom=0.15)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Abstract reverses the direction of the paper's exposed-versus-occluded skin connectivity result

PLoS Comput Biol (2018) · doi:10.1371/journal.pcbi.1006099 · PMC5927471

Summary

The abstract says that skin sites exposed to the external environment have less connected phage-bacteria networks and are "likely more susceptible to network degradation". The Results say the opposite: occluded sites were less connected than exposed and intermittently occluded sites (p < 0.001, Fig 4B). The Discussion also says exposed and intermittently exposed sites were more connected and predicted to be more resilient. Fig 4B as published agrees with the Results and Discussion: the median eigen centrality is about 0.176 for exposed sites and 0.138 for occluded sites, and the paper reads higher values as "more connected" throughout. The body of the paper is internally consistent, so the analysis is unaffected. But the abstract, which is what most readers and indexing services see, states one of the paper's three headline findings in the reverse direction, and so does its claim about vulnerability to degradation.

What the paper reports

Abstract:

Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.

Results, "Network structures across the human skin landscape", second paragraph:

The average centrality of each sample was quantified using the weighted eigenvector centrality metric. Intermittently moist skin sites (dynamic sites that fluctuate between being moist and dry) were significantly more connected than the moist and sebaceous environments (p-value < 0.001, Fig 4A). Also, skin sites that were occluded from the environment were less connected than those that were constantly exposed to the environment or only intermittently occluded (p-value < 0.001, Fig 4B).

Discussion, third paragraph:

Network structure differed between environmentally exposed and occluded skin sites. The sites under greater environmental fluctuation and exposure (the exposed and intermittently exposed sites) were more connected and therefore were predicted to have a higher resilience against network degradation when random nodes are removed from the network.

Together these data suggested that body sites under greater degrees of fluctuation harbored more highly connected microbiomes that are potentially more robust to network disruption by extinction events.

Fig 4 legend:

(A) Notched box-plot depicting differences in average eigenvector centrality between moist, intermittently moist, and sebaceous skin sites and (B) occluded, intermittently occluded, and exposed sites.

Sample sizes for each group were: Moist = 81, Sebaceous = 56, IntMoist = 56, Occluded = 106, Exposed = 61, IntOccluded = 26.

What the paper's own numbers show

Method. The paper prints no group values for connectivity in text or tables. They appear only as box plots in Fig 4A and 4B. I took the Fig 4 raster (3800 × 3729 px) embedded on page 11 of the publisher's PDF. For each box I located the median line and the two hinges (quartiles) by pixel row, and converted rows to eigen-centrality units with a straight line fitted to the panel's own y-axis ticks. The fit is exact to within 1e-4, and one pixel is about 0.0003 units, so each value is good to about ±0.002. I then checked the group sizes in the legend. I also took the pairwise Wilcoxon p-values that the authors' public code repository writes for this figure (rtables/skinboxsig.tsv; R's pairwise.wilcox.test, Holm adjustment by default). Finally I compared the direction each passage states with the direction the figure shows. The paper's convention, which the Results apply to both panels, is that a higher eigen centrality means "more connected". Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's figure and data
Exposed vs occluded connectivity Abstract Exposed sites "less connected" and "more susceptible to network degradation" Exposed median 0.176 > occluded median 0.138: exposed sites are the more connected
Occluded vs exposed and intermittently occluded Results; Fig 4B Occluded "less connected" (p < 0.001) Occluded 0.138 < exposed 0.176 and IntOccluded 0.185, which agrees
Exposed and intermittently exposed Discussion "more connected", "higher resilience" Agrees with Fig 4B
Fig 4B medians (Q1–Q3) Fig 4B Box plots only Exposed 0.176 (0.147–0.196); IntOccluded 0.185 (0.172–0.194); Occluded 0.138 (0.094–0.173)
Median differences Fig 4B Not printed Exposed − Occluded = +0.038; IntOccluded − Occluded = +0.047; IntOccluded − Exposed = +0.009
Separation of the groups Fig 4B Not printed The exposed median (0.176) lies above the occluded upper quartile (0.173)
Fig 4A medians (Q1–Q3), the convention check Fig 4A; Results IntMoist "more connected" than Moist and Sebaceous IntMoist 0.187 (0.175–0.197) > Sebaceous 0.145 (0.122–0.167) and Moist 0.137 (0.078–0.176), so higher means "more connected"
Group sizes Fig 4 legend 81 + 56 + 56; 106 + 61 + 26 Both groupings sum to 193 samples, which is consistent
Occluded vs exposed p Results; repository table p < 0.001 Holm-adjusted Wilcoxon p = 1.38e-05, which agrees
Occluded vs IntOccluded p Results; repository table p < 0.001 p = 1.38e-05, which agrees
IntOccluded vs exposed p Repository table Not stated in text p = 0.277 (no difference)

The Results, the Discussion, the figure and the repository's p-values agree with each other. The abstract is the only place in the paper where exposed sites are described as less connected.

Figure

Abstract versus Results and Discussion on exposed versus occluded skin sites, with the Fig 4B medians

Figure 1. The left panel gives the abstract's sentence on exposed skin sites. The right panel gives the Results sentence on Fig 4B and the Discussion sentence on the same comparison. All three are quoted as printed and framed in orange. The connector marks the conflict: the abstract says exposed sites are "less connected", while the Results and the Discussion say they are more connected than occluded sites. The blue strip gives the Fig 4B medians that recompute.py.txt reads from the published figure. In those medians, exposed (0.176) and intermittently occluded (0.185) sites are above occluded (0.138) sites, which agrees with the body of the paper.

Median and interquartile range of eigen centrality for each skin-site group in Fig 4A and 4B

Figure 2. Median eigen centrality (dot) and interquartile range (bar) for the six skin-site groups. All values are read by recompute.py.txt from the published Fig 4A and 4B and are shown on one axis that starts at zero. The moisture panel (top) fixes the paper's convention: the Results call IntMoist, the highest group, "more connected". The occlusion panel (bottom) shows exposed sites above occluded sites. The orange label gives the abstract's statement, which runs the other way. Group sizes are from the Fig 4 legend.

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the Abstract, replace

Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.

with

Network structure differed between skin sites, with those occluded from the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.

or, stated from the exposed side,

Network structure differed between skin sites, with those exposed to the external environment being more connected and likely more resilient to network degradation by microbial extinction events.

Either wording matches the Results, Fig 4B and the Discussion. No other part of the paper needs to change.

Recomputation script (python recompute.py.txt)
"""Recompute the skin-site connectivity check for PMC5927471
(PLoS Comput Biol 2018, doi:10.1371/journal.pcbi.1006099).

Run:  python recompute.py.txt
Needs numpy, matplotlib and Pillow; PyMuPDF (pymupdf / fitz) to read the figure out of
the published PDF. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and candidates/cand1..cand5.svg/png next to this file.

Inputs
  * Text quoted from the article as archived in PubMed Central (Abstract; Results,
    "Network structures across the human skin landscape"; Discussion; Fig 4 legend).
  * Fig 4 as printed in the publisher's PDF (page 11, embedded 3800 x 3729 px raster).
    The PDF is fetched from the public PMC open-data bucket, which holds the same
    publisher PDF (24 pages):
        https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf
    Box-plot medians and hinges (quartiles) of panels 4A and 4B are read off that
    raster by pixel position against the panel's own y-axis ticks.
  * The authors' public code repository (linked from the article's data-availability
    statement), table rtables/skinboxsig.tsv: pairwise Wilcoxon p-values from R's
    pairwise.wilcox.test (default Holm adjustment). Its six values are transcribed
    below as REPO_P.
If the PDF download fails, the script falls back to the values it read on 2026-10-23
(stored below as FALLBACK_FIG) and says so.

Reading convention used by the paper itself: a higher eigen-centrality value is
described as "more connected" (Results: IntMoist, the highest box in Fig 4A, is "more
connected" than Moist and Sebaceous; Occluded, the lowest box in Fig 4B, is "less
connected").
"""
import io
import os
import tempfile
import textwrap
import urllib.request

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

PDF_URL = "https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf"

# ------------------------------------------------------------------ text (verbatim)
Q_ABSTRACT = ("Network structure differed between skin sites, with those exposed to the "
              "external environment being less connected and likely more susceptible to "
              "network degradation by microbial extinction events.")
Q_RESULTS = ("Also, skin sites that were occluded from the environment were less connected "
             "than those that were constantly exposed to the environment or only "
             "intermittently occluded (p-value < 0.001, Fig 4B).")
Q_RESULTS_A = ("Intermittently moist skin sites (dynamic sites that fluctuate between being "
               "moist and dry) were significantly more connected than the moist and "
               "sebaceous environments (p-value < 0.001, Fig 4A).")
Q_DISCUSSION = ("The sites under greater environmental fluctuation and exposure (the exposed "
                "and intermittently exposed sites) were more connected and therefore were "
                "predicted to have a higher resilience against network degradation when "
                "random nodes are removed from the network.")
# Fig 4 legend
N = {"Moist": 81, "Sebaceous": 56, "IntMoist": 56,
     "Occluded": 106, "Exposed": 61, "IntOccluded": 26}

# ------------------------------------------------------------------ fallbacks (2026-10-23)
FALLBACK_FIG = {  # group: (Q1, median, Q3) in eigen-centrality units
    "IntMoist": (0.175, 0.187, 0.197), "Moist": (0.078, 0.137, 0.176),
    "Sebaceous": (0.122, 0.145, 0.167), "Exposed": (0.147, 0.176, 0.196),
    "IntOccluded": (0.172, 0.185, 0.194), "Occluded": (0.094, 0.138, 0.173)}
REPO_P = {  # rtables/skinboxsig.tsv, transcribed 2026-10-23
    ("IntOccluded", "Exposed"): 0.277498657068735,
          ("Occluded", "Exposed"): 1.38010591458678e-05,
          ("Occluded", "IntOccluded"): 1.38010591458678e-05,
          ("Sebaceous", "Moist"): 0.132875880147664,
          ("Moist", "IntMoist"): 2.66084000742917e-08,
          ("Sebaceous", "IntMoist"): 4.21495886720462e-08}


# ------------------------------------------------------------------ figure digitising
def fetch(url, timeout=60):
    req = urllib.request.Request(url, headers={"User-Agent": "recompute.py"})
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return r.read()


def load_fig4_raster():
    """Return Fig 4 as a greyscale numpy array, taken from the publisher PDF."""
    try:
        import pymupdf as fitz
    except ImportError:
        import fitz
    from PIL import Image
    pdf = fetch(PDF_URL)
    with tempfile.TemporaryDirectory() as td:
        path = os.path.join(td, "paper.pdf")
        with open(path, "wb") as f:
            f.write(pdf)
        doc = fitz.open(path)
        print(f"Publisher PDF: {doc.page_count} pages")
        for pno in range(doc.page_count):
            for img in doc[pno].get_images():
                if (img[2], img[3]) == (3800, 3729):
                    raw = doc.extract_image(img[0])["image"]
                    print(f"Fig 4 raster found on PDF page {pno + 1}: 3800 x 3729 px")
                    return np.asarray(Image.open(io.BytesIO(raw)).convert("L")).astype(int)
    raise RuntimeError("Fig 4 raster not found in PDF")


def axis_ticks(a, x0, x1):
    """y-axis column of a panel and the row centres of its tick marks (top to bottom)."""
    dark = a < 80
    sub = dark[:1150, x0:x1]
    ax = int(sub.sum(0).argmax())
    rows = [y for y in range(1150) if sub[y, ax - 12:ax - 2].all()]
    groups = []
    for y in rows:
        if groups and y - groups[-1][-1] <= 1:
            groups[-1].append(y)
        else:
            groups.append([y])
    return ax + x0, [float(np.mean(g)) for g in groups]


def dark_runs(a, col, ylo=150, yhi=1100, thr=80):
    out, cur = [], []
    for y in range(ylo, yhi):
        if a[y, col] < thr:
            cur.append(y)
        elif cur:
            out.append(cur)
            cur = []
    if cur:
        out.append(cur)
    return out


def digitise(a):
    res = {}
    panels = [("A", (0, 1900), ["IntMoist", "Moist", "Sebaceous"], [158, 323, 488]),
              ("B", (1900, 3800), ["Exposed", "IntOccluded", "Occluded"], [763, 928, 1093])]
    for panel, (x0, x1), labels, centres in panels:
        axcol, tk = axis_ticks(a, x0, x1)
        vals = [0.3, 0.2, 0.1, 0.0][:len(tk)]
        slope, icpt = np.polyfit(tk, vals, 1)
        resid = np.max(np.abs(np.polyval([slope, icpt], tk) - vals))
        print(f"Fig 4{panel}: y-axis at x={axcol}px, ticks at rows "
              f"{[round(t, 1) for t in tk]} = {vals}; 1 px = {abs(slope):.5f}; "
              f"max tick misfit {resid:.1e}")
        for lab, cx in zip(labels, centres):
            xc = int(cx * 3800 / 1200) - 60          # just off the whisker, inside the notch
            runs = dark_runs(a, xc)
            imed = max(range(len(runs)), key=lambda i: len(runs[i]))  # median: thickest line
            r_med = float(np.mean(runs[imed]))
            r_q3 = float(np.mean(runs[imed - 1]))    # upper hinge line
            r_q1 = float(np.mean(runs[imed + 1]))    # lower hinge line
            q1, med, q3 = (float(slope * r + icpt) for r in (r_q1, r_med, r_q3))
            res[lab] = (q1, med, q3)
            print(f"  {lab:12s} n={N[lab]:3d}  rows Q3/med/Q1 = {r_q3:6.1f}/{r_med:6.1f}/"
                  f"{r_q1:6.1f} (median line {len(runs[imed])} px thick)  ->  "
                  f"Q1 {q1:.3f}  median {med:.3f}  Q3 {q3:.3f}")
    return res


try:
    FIG = digitise(load_fig4_raster())
    FIG_SRC = "read from the published Fig 4 (publisher PDF, p. 11)"
except Exception as e:  # network or library problem
    print(f"!! could not read the figure ({e}); using values read on 2026-10-23")
    FIG, FIG_SRC = FALLBACK_FIG, "values read on 2026-10-23 (fallback)"
    for k, (q1, m, q3) in FIG.items():
        print(f"  {k:12s} Q1 {q1:.3f}  median {m:.3f}  Q3 {q3:.3f}")

P, P_SRC = REPO_P, "authors' repository table rtables/skinboxsig.tsv"

med = {k: v[1] for k, v in FIG.items()}

print()
print("Group sizes (Fig 4 legend):")
print(f"  moisture  {N['Moist']} + {N['Sebaceous']} + {N['IntMoist']} = "
      f"{N['Moist'] + N['Sebaceous'] + N['IntMoist']}")
print(f"  occlusion {N['Occluded']} + {N['Exposed']} + {N['IntOccluded']} = "
      f"{N['Occluded'] + N['Exposed'] + N['IntOccluded']}")

print()
print(f"Pairwise Wilcoxon p-values, Holm-adjusted ({P_SRC}):")
for (g1, g2), v in P.items():
    print(f"  {g1:11s} vs {g2:11s} p = {v:.3g}   < 0.001: {v < 1e-3}")

d_eo = med["Exposed"] - med["Occluded"]
d_io = med["IntOccluded"] - med["Occluded"]
print()
print("Median differences read from Fig 4B (eigen-centrality units):")
print(f"  Exposed - Occluded     = {med['Exposed']:.3f} - {med['Occluded']:.3f} = {d_eo:+.3f}")
print(f"  IntOccluded - Occluded = {med['IntOccluded']:.3f} - {med['Occluded']:.3f} = {d_io:+.3f}")
print(f"  IntOccluded - Exposed  = {med['IntOccluded'] - med['Exposed']:+.3f}")
print(f"  Exposed median lies above the Occluded upper hinge (Q3 {FIG['Occluded'][2]:.3f}): "
      f"{med['Exposed'] > FIG['Occluded'][2]}")

print()
print("Direction checks (paper's convention: higher centrality = more connected):")
print("  Fig 4A: IntMoist highest (Results: 'more connected')       ->",
      med["IntMoist"] > max(med["Moist"], med["Sebaceous"]))
print("  Fig 4B: Occluded lowest (Results: 'less connected')         ->",
      med["Occluded"] < min(med["Exposed"], med["IntOccluded"]))
print("  Discussion 'exposed ... more connected' agrees with Fig 4B  ->",
      med["Exposed"] > med["Occluded"])
print("  Abstract 'exposed ... less connected' agrees with Fig 4B    ->",
      med["Exposed"] < med["Occluded"])

# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none"})
W = 7.2


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def wrap(s, n):
    return "\n".join(textwrap.wrap(s, n))


def panel(ax, x, y, w, h, edge, title, body, n=46, body_size=10.5):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                fc="white", ec=edge, lw=2, transform=ax.transAxes))
    ax.text(x + 0.02, y + h - 0.035, title, transform=ax.transAxes, fontsize=11,
            fontweight="bold", va="top", color=INK)
    ax.text(x + 0.02, y + h - 0.115, body, transform=ax.transAxes, fontsize=body_size,
            va="top", color=INK, linespacing=1.35)


def blank(h):
    fig = plt.figure(figsize=(W, h))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    return fig, ax


SHORT_ABS = ("“…those exposed to the external environment being less connected "
             "and likely more susceptible to network degradation by microbial extinction "
             "events.”")
SHORT_RES = ("“…skin sites that were occluded from the environment were less "
             "connected than those that were constantly exposed to the environment or only "
             "intermittently occluded (p-value < 0.001, Fig 4B).”")
SHORT_DIS = ("“…(the exposed and intermittently exposed sites) were more connected "
             "and therefore were predicted to have a higher resilience against network "
             "degradation…”")


def legend_row(ax, y, items):
    x = 0.03
    for color, label in items:
        ax.add_patch(FancyBboxPatch((x, y - 0.012), 0.03, 0.024, boxstyle="round,pad=0,"
                     "rounding_size=0.004", fc="white", ec=color, lw=2, transform=ax.transAxes))
        ax.text(x + 0.04, y, label, transform=ax.transAxes, va="center", fontsize=10,
                color=INK2)
        x += 0.06 + 0.0105 * len(label)


# ---------- candidate 1: two text panels + connector + figure strip (used as fig1)
def cand1():
    fig, ax = blank(5.0)
    panel(ax, 0.01, 0.35, 0.39, 0.62, ORANGE, "Abstract", wrap(SHORT_ABS, 33))
    panel(ax, 0.60, 0.35, 0.39, 0.62, ORANGE, "Results and Discussion",
          wrap(SHORT_RES, 35) + "\n\n" + wrap(SHORT_DIS, 35), body_size=10)
    ax.add_patch(FancyArrowPatch((0.405, 0.665), (0.595, 0.665), arrowstyle="<|-|>",
                 mutation_scale=14, lw=2, color=INK2, transform=ax.transAxes))
    ax.text(0.5, 0.70, "Opposite\ndirection", transform=ax.transAxes, ha="center",
            va="bottom", fontsize=10.5, color=INK, fontweight="bold")
    ax.text(0.5, 0.63, "exposed sites:\n“less connected”\nvs\n“more connected”",
            transform=ax.transAxes, ha="center", va="top", fontsize=10, color=INK2)
    ax.add_patch(FancyBboxPatch((0.01, 0.10), 0.98, 0.20, boxstyle="round,pad=0,"
                 "rounding_size=0.012", fc="white", ec=BLUE, lw=2, transform=ax.transAxes))
    ax.text(0.03, 0.245, "Fig 4B medians read from the published figure (eigen centrality;"
            " higher = more connected)", transform=ax.transAxes,
            fontsize=10, color=INK2, va="center")
    ax.text(0.03, 0.155, f"Exposed {med['Exposed']:.3f}  ·  IntOccluded "
            f"{med['IntOccluded']:.3f}  ·  Occluded {med['Occluded']:.3f}"
            "  \u2192  exposed above occluded",
            transform=ax.transAxes, fontsize=10.5, color=INK, va="center", fontweight="bold")
    legend_row(ax, 0.04, [(ORANGE, "Text as printed in the paper"),
                          (BLUE, "Values read from the paper's Fig 4B")])
    return fig


# ---------- candidate 2: direction ledger
def cand2():
    fig, ax = blank(3.6)
    rows = [("Abstract", "Exposed sites less connected; more\nsusceptible to network degradation",
             "<", ORANGE),
            ("Results (Fig 4B)", "Occluded less connected than exposed\nand int. occluded (p < 0.001)",
             ">", ORANGE),
            ("Discussion", "Exposed and intermittently exposed\nmore connected; higher resilience",
             ">", ORANGE),
            ("Fig 4B medians", f"Exposed {med['Exposed']:.3f} vs Occluded {med['Occluded']:.3f}\n"
             f"(IntOccluded {med['IntOccluded']:.3f})", ">", BLUE)]
    ax.text(0.03, 0.93, "Source", transform=ax.transAxes, fontweight="bold", fontsize=10.5)
    ax.text(0.25, 0.93, "What it says", transform=ax.transAxes, fontweight="bold",
            fontsize=10.5)
    ax.text(0.71, 0.93, "Exposed vs occluded", transform=ax.transAxes, fontweight="bold",
            fontsize=10.5)
    for i, (src, txt, sign, col) in enumerate(rows):
        y = 0.76 - i * 0.19
        ax.plot([0.02, 0.98], [y + 0.09, y + 0.09], color=GRID, lw=1, transform=ax.transAxes)
        ax.plot([0.015, 0.015], [y - 0.07, y + 0.07], color=col, lw=4, transform=ax.transAxes,
                solid_capstyle="butt")
        ax.text(0.03, y, src, transform=ax.transAxes, va="center", fontsize=10.5)
        ax.text(0.25, y, txt, transform=ax.transAxes, va="center", fontsize=10, color=INK2)
        word = "▼ LESS connected" if sign == "<" else "▲ MORE connected"
        ax.text(0.71, y, word, transform=ax.transAxes, va="center", fontsize=10.5,
                fontweight="bold", color=INK)
    legend_row(ax, 0.03, [(ORANGE, "Stated in the paper's text"),
                          (BLUE, "Read from the paper's Fig 4B")])
    return fig


# ---------- candidate 3: dot + IQR plot of medians (used as fig2)
def cand3():
    order = ["IntMoist", "Moist", "Sebaceous", "Exposed", "IntOccluded", "Occluded"]
    fig, ax = plt.subplots(figsize=(W, 4.0))
    ys = [6.4, 5.4, 4.4, 2.6, 1.6, 0.6]
    for g, y in zip(order, ys):
        q1, m, q3 = FIG[g]
        ax.plot([q1, q3], [y, y], color=BLUE, lw=2, solid_capstyle="round", zorder=2)
        ax.plot(m, y, "o", ms=9, color=BLUE, mec="white", mew=1.5, zorder=3)
        ax.text(q3 + 0.004, y, f"{m:.3f}", va="center", fontsize=10, color=INK)
    ax.set_yticks(ys)
    ax.set_yticklabels([f"{g} (n={N[g]})" for g in order])
    ax.set_xlim(0, 0.32)
    ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20, 0.25])
    ax.set_ylim(0, 7.3)
    ax.set_xlabel("Eigen centrality: median (dot) and interquartile range (bar)")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.axhline(3.5, color=GRID, lw=1)
    ax.text(0.003, 7.0, "Fig 4A, moisture", fontsize=10, color=INK2, fontweight="bold")
    ax.text(0.003, 3.15, "Fig 4B, occlusion", fontsize=10, color=INK2, fontweight="bold")
    ax.text(0.232, 5.4, "Results: IntMoist\n“more connected”", fontsize=10,
            color=INK2, va="center")
    ax.text(0.232, 2.15, "Abstract: exposed\nsites “less\nconnected”", fontsize=10,
            color=ORANGE, fontweight="bold", va="center")
    ax.text(0.232, 0.75, "Results, Discussion:\noccluded less,\nexposed more", fontsize=10,
            color=INK2, va="center")
    h1, = ax.plot([], [], "o-", color=BLUE, ms=9, lw=2)
    fig.legend([h1], ["Values read from the paper's Fig 4 (publisher PDF)"],
               loc="lower center", bbox_to_anchor=(0.5, -0.04), frameon=False, fontsize=10)
    fig.tight_layout(rect=(0, 0.05, 1, 1))
    return fig


# ---------- candidate 4: two-item slope chart, ordering implied vs ordering shown
def cand4():
    fig, ax = plt.subplots(figsize=(W, 3.6))
    left = {"Occluded": 2, "Exposed": 1}
    right = {"Exposed": 2 if med["Exposed"] > med["Occluded"] else 1,
             "Occluded": 1 if med["Exposed"] > med["Occluded"] else 2}
    for g in ("Exposed", "Occluded"):
        ax.plot([0, 1], [left[g], right[g]], color=INK2, lw=2, zorder=1)
        ax.plot(0, left[g], "o", ms=11, color=ORANGE, mec="white", mew=1.5, zorder=3)
        ax.plot(1, right[g], "o", ms=11, color=BLUE, mec="white", mew=1.5, zorder=3)
        ax.text(-0.05, left[g], g, ha="right", va="center", fontsize=11)
        ax.text(1.05, right[g], f"{g}  (median {med[g]:.3f})", ha="left", va="center",
                fontsize=11)
    ax.text(0, 2.45, "Abstract\n(order it implies)", ha="center", fontsize=10.5,
            fontweight="bold", color=INK)
    ax.text(1, 2.45, "Fig 4B\n(order of medians)", ha="center", fontsize=10.5,
            fontweight="bold", color=INK)
    ax.text(0.5, 1.5, "lines cross:\nthe order is reversed", ha="center", va="center",
            fontsize=10, color=INK2, bbox=dict(fc=BG, ec="none", pad=2))
    ax.text(-0.62, 2.0, "more\nconnected", ha="center", va="center", fontsize=10, color=INK2)
    ax.text(-0.62, 1.0, "less\nconnected", ha="center", va="center", fontsize=10, color=INK2)
    ax.set_xlim(-0.8, 1.75)
    ax.set_ylim(0.6, 2.9)
    ax.axis("off")
    ax.plot([], [], "o", color=ORANGE, ms=10, label="Abstract's statement (ranks only)")
    ax.plot([], [], "o", color=BLUE, ms=10, label="Read from the paper's Fig 4B")
    ax.legend(loc="lower center", bbox_to_anchor=(0.5, -0.12), ncol=2, frameon=False,
              fontsize=10)
    return fig


# ---------- candidate 5: three stacked quotes with direction tags + compact value strip
def cand5():
    fig = plt.figure(figsize=(W, 5.0))
    axt = fig.add_axes([0.0, 0.115, 0.62, 0.885])
    axt.set_xlim(0, 1)
    axt.set_ylim(0, 1)
    axt.axis("off")
    items = [("Abstract", SHORT_ABS, "▼ exposed less connected"),
             ("Results (Fig 4B)", SHORT_RES, "▲ exposed more connected"),
             ("Discussion", SHORT_DIS, "▲ exposed more connected")]
    for i, (t, q, tag) in enumerate(items):
        y0 = 0.68 - i * 0.335
        axt.add_patch(FancyBboxPatch((0.01, y0), 0.97, 0.31, boxstyle="round,pad=0,"
                      "rounding_size=0.015", fc="white", ec=ORANGE, lw=2,
                      transform=axt.transAxes))
        axt.text(0.04, y0 + 0.28, f"{t}:  {tag}", fontsize=10.5, fontweight="bold",
                 va="top", transform=axt.transAxes)
        axt.text(0.04, y0 + 0.205, wrap(q, 54), fontsize=10, va="top", color=INK2,
                 transform=axt.transAxes, linespacing=1.3)
    ax = fig.add_axes([0.76, 0.20, 0.23, 0.70])
    order = ["Exposed", "IntOccluded", "Occluded"]
    for i, g in enumerate(order):
        q1, m, q3 = FIG[g]
        ax.plot([i, i], [q1, q3], color=BLUE, lw=2)
        ax.plot(i, m, "o", ms=9, color=BLUE, mec="white", mew=1.5)
        ax.text(i + 0.13, m, f"{m:.3f}", va="center", fontsize=10)
    ax.set_xticks(range(3))
    ax.set_xticklabels(["Exp.", "Int.\nOcc.", "Occ."])
    ax.set_xlim(-0.5, 2.9)
    ax.set_ylim(0, 0.25)
    ax.set_ylabel("Eigen centrality, Fig 4B")
    ax.grid(axis="y", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.set_title("Median and IQR", fontsize=10.5, color=INK, loc="left")
    fig.text(0.01, 0.0, "Orange frames: text as printed in the paper.  Blue: values read "
             "from the paper's Fig 4B.\nHigher eigen centrality = more connected, as the "
             "paper uses it.", fontsize=10, color=INK2, va="bottom")
    return fig


save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))
save(cand2(), os.path.join(CAND, "cand2_direction_ledger"))
save(cand3(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand3_median_dotplot"))
save(cand4(), os.path.join(CAND, "cand4_rank_slope"))
save(cand5(), os.path.join(CAND, "cand5_quotes_plus_values"))
print()
print("Wrote fig1.svg/png, fig2.svg/png and candidates/cand1..cand5 (.svg, .png)")

Abstract says LabMol-149 has EC50 below 500 nM; Table 3 gives 1450 and 509 nM

PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007025 · PMC7048302

Summary

The Abstract reports that two of the five compounds chosen by the paper's deep-learning screen, LabMol-149 and LabMol-152, showed antiplasmodial activity "at low nanomolar concentrations (EC50 <500 nM)". The final paragraph of the Discussion says both compounds "completely satisfy" a criterion of EC50 <500 nM against sensitive and resistant strains. The paper's own Table 3 gives LabMol-149 EC50 values of 1.450 ± 0.524 μM against the sensitive 3D7 strain and 0.509 ± 0.177 μM against the resistant W2 strain. These are 1450 nM and 509 nM, so the compound is above the stated threshold on both strains, by a factor of 2.9 on 3D7. Only LabMol-152 (49 and 78 nM) meets the criterion. The headline count of qualifying compounds is therefore one, not two.

What the paper reports

Abstract:

Among them, two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM) and low cytotoxicity in mammalian cells. Therefore, the computational approach employing deep learning developed here allowed us to discover two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria for antimalarial target candidates.

Author summary:

This allowed us to discover two new potential antimalarial candidates.

Discussion, final paragraph:

Moreover, two of these compounds (LabMol-149 and LabMol-152) completely satisfy the guidelines and criteria for discovery of new antimalarial drugs, i.e., activity at low nanomolar concentrations (EC50 <500nM) against sensitive and multiple resistant strains of Plasmodium spp. and SI greater than 10 folds [12,42].

Table 3, header and LabMol-149 row (columns: compound | EC50 (Pf3D7)(μM) | EC50 (PfW2)(μM) | CC50 (COS-7) (μM) | SIa | SIb):

LabMol-149 | 1.450 ± 0.524 | 0.509 ± 0.177 | >100 | >71.4 | >196

Table 3, the other two active candidate compounds:

LabMol-151 | 1.911 ± 0.292 | 1.616 ± 0.321 | 140.8 ± 11.2 | 73.7 | 87.1

LabMol-152 | 0.049 ± 0.029 | 0.078 ± 0.014 | 16.7 ± 10.3 | 340.8 | 214.1

Table 3 footer:

SIa: Selectivity index calculated by COS7 CC50/3D7 EC50; and SIb: Selectivity index calculated by COS7 CC50/W2 EC50; The data are expressed as mean ± SD of three independent assays.

Two related passages use different wording. Results, "Experimental validation":

The EC50 for each compound (Table 3) indicate that two compounds, 2-(4,6-diphenyl-1,2-dihydro-1,3,5-triazin-2-yl)phenol (LabMol-149), 4-{N-[3-(morpholin-4-yl)-1,4-dioxo-1,4-dihydronaphthalen-2-yl]acetamido}benzoic acid (LabMol-151) and N2-(3-fluorophenyl)-N4-[(oxolan-2-yl)methyl]quinazoline-2,4-diamine (LabMol-152), were potent at inhibiting the parasite growth, showing activities in submicromolar and low nanomolar range against both 3D7 and W2 strains.

Discussion, third paragraph:

Two compounds (LabMol-149 and LabMol-151) showed activity at submicromolar concentrations against asexual blood stages and low cytotoxicity in mammalian cells.

What the paper's own numbers show

Method. I converted every Table 3 EC50 from μM (the unit in the column header) to nM and compared each one with the 500 nM threshold stated in the Abstract and the Discussion. The Discussion applies that threshold to both sensitive and resistant strains, so a compound qualifies only if both its 3D7 and W2 values are below 500 nM. For LabMol-149 I also looked at mean − 1 SD, using the footer's statement that the values are mean ± SD of three assays. I recomputed every selectivity index from the footer's definitions (CC50/EC50), and checked the "submicromolar" (below 1 μM) wording against the same values. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
LabMol-149 EC50, 3D7 Table 3; Abstract and Discussion say <500 nM 1.450 ± 0.524 μM 1450 nM, 2.90 × the 500 nM threshold; mean − 1 SD = 926 nM
LabMol-149 EC50, W2 Table 3; Abstract and Discussion say <500 nM 0.509 ± 0.177 μM 509 nM, 1.018 × the threshold (9 nM above it)
LabMol-152 EC50, 3D7 / W2 Table 3 0.049 / 0.078 μM 49 / 78 nM, both below 500 nM
LabMol-151 EC50, 3D7 / W2 Table 3 1.911 / 1.616 μM 1911 / 1616 nM, both above 500 nM
Tested LabMol compounds with EC50 <500 nM on both strains Abstract; Discussion, final paragraph two (LabMol-149 and LabMol-152) one (LabMol-152)
LabMol-149 SIa Table 3 >71.4 100 / 1.450 = >69.0; 71.4 corresponds to an EC50 of 1.401 μM
LabMol-149 SIb Table 3 >196 100 / 0.509 = >196.5
LabMol-151 SIa / SIb Table 3 73.7 / 87.1 73.7 / 87.1
LabMol-152 SIa / SIb Table 3 340.8 / 214.1 340.8 / 214.1
Chloroquine SIa / SIb Table 3 4,545 / 289 >4545.5 / >289.0
Pyrimethamine SIa / SIb Table 3 2,702 / 5.5 >2702.7 / >5.5
LabMol-149 "submicromolar" Discussion, third paragraph submicromolar 3D7 1450 nM, not below 1 μM; W2 509 nM, below 1 μM
LabMol-151 "submicromolar" Discussion, third paragraph submicromolar 1911 / 1616 nM, below 1 μM on neither strain

The table's arithmetic is internally consistent. Every selectivity index recomputes from the printed EC50 and CC50 values, except LabMol-149's SIa, which is slightly high (71.4 against 69.0). The problem is the text. It counts LabMol-149 among the compounds below 500 nM, and the table does not support that for either strain. The Results passage also says "two compounds" and then names three.

Figure

The Abstract and Discussion claims set against the Table 3 values for LabMol-149 and LabMol-152

Figure 1. On the left (orange), the two passages that state the EC50 <500 nM criterion and name LabMol-149 and LabMol-152 as meeting it, quoted from the Abstract and the final paragraph of the Discussion. On the right (blue), the Table 3 values for the same two compounds, converted from μM to nM. LabMol-149 is 1450 nM against 3D7 (2.90 times the threshold) and 509 nM against W2 (1.02 times). LabMol-152 is 49 and 78 nM. Only LabMol-152 meets the criterion as the text states it.

EC50 mean ± SD for the three active candidates on a log axis, with the thresholds the text claims

Figure 2. Table 3 EC50 values (blue dots, mean; blue lines, ± 1 SD of three assays) for the three active candidate compounds on one log-scaled nM axis. The solid vertical line marks 500 nM and the dashed line 1 μM. The orange markers show what the text says about each compound: below 500 nM for LabMol-149 and LabMol-152 (Abstract and final paragraph of the Discussion), and "submicromolar" for LabMol-151 (Discussion, third paragraph; that sentence also calls LabMol-149 submicromolar). LabMol-149's 3D7 interval lies entirely above 500 nM, and both LabMol-151 values lie above 1 μM. The W2 value for LabMol-149 sits at the threshold, and its ± 1 SD interval spans it.

Why it matters

The count of compounds meeting the potency criterion is the paper's headline experimental result. It appears in the Abstract ("two compounds ... (EC50 <500 nM)"), in the Author summary ("two new potential antimalarial candidates") and in the final paragraph of the Discussion, which says the two compounds "completely satisfy" the criteria for new antimalarial drugs. The Abstract also concludes that the approach found "two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria".

By the paper's own Table 3, one compound (LabMol-152, a quinazoline) meets the stated criterion. LabMol-149 fails it on the sensitive strain by a wide margin. Its 3D7 EC50 of 1450 nM is also above the 1 μM cut-off that the paper itself uses to label training compounds "active" (Methods, "Datasets": "1,058 active compounds with EC50 ≤ 1 μM"). On the resistant strain it is essentially at the threshold, at 509 nM. LabMol-149 remains an active and selective hit (SI above about 69 on 3D7 and 196 on W2), and nothing here questions the measurements themselves. But it is a low-micromolar hit, not a compound that satisfies the sub-500 nM criterion.

Readers who rely on the Abstract, for example to compile hit lists or to judge how well deep-learning virtual screening performs in antimalarial discovery, would count two qualifying compounds from two scaffold families where the data show one. The rest of the paper's conclusions, such as the QSAR models, the screen and LabMol-152's potency, do not depend on this. These are in vitro preclinical results, so there is no direct clinical consequence.

Explanations considered

Status

Suggested correction

  1. In the Abstract, change "two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM)" so that it names LabMol-152 alone as meeting EC50 <500 nM. Describe LabMol-149 as active in the low-micromolar to high-nanomolar range (1.450 μM on 3D7, 0.509 μM on W2). Revise "two new families ... in compliance with the guidelines and criteria" and the Author summary's "two new potential antimalarial candidates" to match.
  2. In the final paragraph of the Discussion, restrict "completely satisfy the guidelines and criteria ... (EC50 <500nM)" to LabMol-152.
  3. In the third paragraph of the Discussion, replace "submicromolar" for LabMol-149 and LabMol-151, which are 1.450 / 0.509 μM and 1.911 / 1.616 μM, with "low micromolar" or the actual values.
  4. Minor: in the Results, "two compounds" is followed by three names; change it to "three compounds". Change LabMol-149's SIa from >71.4 to >69.0 (100/1.450), or state which EC50 value was used to compute it.
Recomputation script (python recompute.py.txt)
"""Recompute the potency claims of PMC7048302
(PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007025) from the paper's own Table 3.

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from Table 3 of the paper as archived in PMC (identical in the
publisher's version of record). Table 3 header: EC50 and CC50 in uM; footer: "The data
are expressed as mean +/- SD of three independent assays"; SIa = CC50/3D7 EC50,
SIb = CC50/W2 EC50.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (Table 3, uM)
# cc_censored: CC50 printed as ">x", so the SI is a lower bound
T3 = {
    "LabMol-149":    dict(e3=1.450, sd3=0.524, ew=0.509, sdw=0.177, cc=100.0, cc_censored=True,  SIa=71.4,  SIb=196.0),
    "LabMol-151":    dict(e3=1.911, sd3=0.292, ew=1.616, sdw=0.321, cc=140.8, cc_censored=False, SIa=73.7,  SIb=87.1),
    "LabMol-152":    dict(e3=0.049, sd3=0.029, ew=0.078, sdw=0.014, cc=16.7,  cc_censored=False, SIa=340.8, SIb=214.1),
    "Chloroquine":   dict(e3=0.011, sd3=0.001, ew=0.173, sdw=0.020, cc=50.0,  cc_censored=True,  SIa=4545,  SIb=289),
    "Pyrimethamine": dict(e3=0.037, sd3=0.007, ew=18.240, sdw=4.537, cc=100.0, cc_censored=True, SIa=2702,  SIb=5.5),
}
# LabMol-148 and LabMol-150: EC50 > 40 uM on both strains (inactive at the highest concentration)

THRESH_NM = 500.0     # Abstract and Discussion (final paragraph): "EC50 <500 nM"
SUBMICRO_NM = 1000.0  # "submicromolar" = below 1 uM
MODEL_ACTIVE_UM = 1.0 # Methods, Datasets: training compounds "active" if EC50 <= 1 uM
ASSAY_RANGE_UM = (0.019, 40.0)  # Methods, Antiplasmodial assay: 12-point dilution series

# What the text says about each compound
CLAIM_LT500 = {"LabMol-149", "LabMol-152"}        # Abstract; Discussion final paragraph
CLAIM_SUBMICRO = {"LabMol-149", "LabMol-151"}     # Discussion, third paragraph


def nm(um):
    return um * 1000.0


print("=== 1. Table 3 EC50 values in nM, against the stated <500 nM criterion ===")
for k, v in T3.items():
    n3, nw = nm(v["e3"]), nm(v["ew"])
    print(f"{k:14s} 3D7 {n3:7.0f} nM ({n3 / THRESH_NM:6.3f} x 500)  W2 {nw:7.0f} nM ({nw / THRESH_NM:6.3f} x 500)  "
          f"<500 nM on both strains: {n3 < THRESH_NM and nw < THRESH_NM}")

print("\n=== 2. LabMol-149 in detail ===")
v = T3["LabMol-149"]
for strain, m, s in (("3D7", v["e3"], v["sd3"]), ("W2", v["ew"], v["sdw"])):
    print(f"{strain}: mean {nm(m):.0f} nM, SD {nm(s):.0f} nM; mean - 1 SD = {nm(m - s):.0f} nM, "
          f"mean + 1 SD = {nm(m + s):.0f} nM; mean exceeds 500 nM by {nm(m) - THRESH_NM:.0f} nM")
print(f"3D7 mean - 1 SD above 500 nM: {nm(v['e3'] - v['sd3']) > THRESH_NM}; "
      f"above 1 uM (the paper's own model 'active' cut-off, EC50 <= {MODEL_ACTIVE_UM} uM): {v['e3'] > MODEL_ACTIVE_UM}")
print(f"3D7 EC50 {v['e3']} uM lies inside the tested range {ASSAY_RANGE_UM[0]}-{ASSAY_RANGE_UM[1]} uM: "
      f"{ASSAY_RANGE_UM[0] < v['e3'] < ASSAY_RANGE_UM[1]}")

print("\n=== 3. Selectivity indices recomputed (CC50 / EC50) ===")
for k, val in T3.items():
    sia, sib = val["cc"] / val["e3"], val["cc"] / val["ew"]
    pre = ">" if val["cc_censored"] else ""
    print(f"{k:14s} SIa reported {pre}{val['SIa']:<7} recomputed {pre}{sia:.1f}   "
          f"SIb reported {pre}{val['SIb']:<6} recomputed {pre}{sib:.1f}")
print(f"LabMol-149: 3D7 EC50 implied by SIa = 71.4 with CC50 = 100 uM: {100 / 71.4:.3f} uM")
print(f"LabMol-149: SIa if the 3D7 EC50 were a decimal slip for 0.145 uM: {100 / 0.145:.1f}")
print(f"LabMol-149: W2 EC50 implied by SIb = 196 with CC50 = 100 uM: {100 / 196:.3f} uM (Table 3: 0.509 uM)")

print("\n=== 4. Text descriptions checked against Table 3 ===")
for k in ("LabMol-149", "LabMol-151", "LabMol-152"):
    val = T3[k]
    n3, nw = nm(val["e3"]), nm(val["ew"])
    print(f"{k}: called '<500 nM' in Abstract/Discussion: {k in CLAIM_LT500}; actually <500 nM on both: "
          f"{n3 < THRESH_NM and nw < THRESH_NM} | called 'submicromolar' in Discussion: {k in CLAIM_SUBMICRO}; "
          f"actually <1 uM on 3D7: {n3 < SUBMICRO_NM}, on W2: {nw < SUBMICRO_NM}")
n_meet = sum(nm(T3[k]["e3"]) < THRESH_NM and nm(T3[k]["ew"]) < THRESH_NM for k in ("LabMol-149", "LabMol-151", "LabMol-152"))
print(f"Tested LabMol compounds meeting EC50 <500 nM on both strains: {n_meet} (text claims 2)")
print("Results text: 'two compounds' followed by three names (LabMol-149, LabMol-151, LabMol-152).")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: text-vs-table diagram
v = T3["LabMol-149"]
w = T3["LabMol-152"]
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.8)
ax.axis("off")


def panel(x, y, wd, ht, edge):
    ax.add_patch(FancyBboxPatch((x, y), wd, ht, boxstyle="round,pad=0,rounding_size=0.08",
                                fc="#ffffff", ec=edge, lw=1.5))


# left panel: what the text says (reported, orange)
panel(0.15, 0.95, 3.15, 3.7, ORANGE)
ax.text(0.30, 4.50, "Abstract and Discussion", fontsize=10.5, fontweight="bold", va="top")
ax.text(0.30, 4.20, "Reported in the text", fontsize=10, color=SEC, va="top")
ax.add_line(Line2D([0.30, 0.30], [3.84, 2.62], color=ORANGE, lw=2))
ax.text(0.42, 3.84, "Abstract:\n\u201ctwo compounds, LabMol-149 and\nLabMol-152, showed potent\n"
        "antiplasmodial activity at low\nnanomolar concentrations\n(EC50 <500 nM)\u201d",
        fontsize=10, va="top", linespacing=1.25)
ax.add_line(Line2D([0.30, 0.30], [2.30, 1.08], color=ORANGE, lw=2))
ax.text(0.42, 2.30, "Discussion, final paragraph:\n\u201c(LabMol-149 and LabMol-152)\ncompletely satisfy [\u2026] (EC50\n"
        "<500nM) against sensitive and\nmultiple resistant strains\u201d", fontsize=10, va="top", linespacing=1.25)

# right panel: Table 3 converted (recomputed, blue)
panel(3.95, 0.95, 3.1, 3.7, BLUE)
ax.text(4.10, 4.50, "Table 3 (\u03bcM), converted to nM", fontsize=10.5, fontweight="bold", va="top")
ax.text(4.10, 4.20, "Recomputed from the paper\u2019s table", fontsize=10, color=SEC, va="top")
rows = [
    ("LabMol-149", "3D7", v["e3"], v["sd3"]),
    ("LabMol-149", "W2", v["ew"], v["sdw"]),
    ("LabMol-152", "3D7", w["e3"], w["sd3"]),
    ("LabMol-152", "W2", w["ew"], w["sdw"]),
]
for (cpd, strain, m, s), yy in zip(rows, (3.74, 3.08, 2.22, 1.56)):
    n = nm(m)
    ok = n < THRESH_NM
    ax.text(4.10, yy, f"{cpd}, {strain}: {m:.3f} \u00b1 {s:.3f} \u03bcM", fontsize=10, va="center",
            fontweight="normal" if ok else "bold")
    ax.text(4.10, yy - 0.25, f"= {n:.0f} nM", fontsize=10, va="center", color=BLUE, fontweight="bold")
    verdict = f"{n / THRESH_NM:.2f} \u00d7 500 nM: fails" if not ok else "below 500 nM: meets"
    ax.text(5.15, yy - 0.25, verdict, fontsize=10, va="center", color=TXT if not ok else SEC)
ax.add_line(Line2D([4.10, 6.90], [2.52, 2.52], color=GRID, lw=1))

# connector
ax.add_patch(FancyArrowPatch((3.33, 3.25), (3.92, 3.25), arrowstyle="<|-|>", mutation_scale=14, color=SEC, lw=2))
ax.text(3.625, 3.40, "conflict", fontsize=10, ha="center", va="bottom", color=SEC, fontweight="bold")
ax.text(3.6, 0.48, f"LabMol-149 is {nm(v['e3']):.0f} nM (3D7) and {nm(v['ew']):.0f} nM (W2): above 500 nM on both strains.\n"
        f"Only LabMol-152 ({nm(w['e3']):.0f} and {nm(w['ew']):.0f} nM) meets the criterion the text states.",
        fontsize=10, ha="center", va="center", linespacing=1.3)
save(fig, "fig1")

# Figure 2: EC50 mean +/- SD (nM, log scale) against the text's thresholds
cpds = ["LabMol-149", "LabMol-151", "LabMol-152"]
claims = {"LabMol-149": "Abstract/Discussion: <500 nM", "LabMol-151": "Discussion: “submicromolar”",
          "LabMol-152": "Abstract/Discussion: <500 nM"}
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.20, right=0.97, top=0.84, bottom=0.15)
ypos, ylabels = [], []
y = 0
for c in cpds:
    val = T3[c]
    limit = THRESH_NM if c in CLAIM_LT500 else SUBMICRO_NM
    # claim: orange bracket from the claimed upper limit leftwards
    ax.annotate("", xy=(limit * 0.55, y + 0.5), xytext=(limit, y + 0.5),
                arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12))
    ax.plot([limit, limit], [y + 0.32, y + 0.68], color=ORANGE, lw=2)
    ax.text(limit * 1.06, y + 0.5, claims[c], color=TXT, fontsize=10, va="center", ha="left")
    for j, (strain, m, s) in enumerate((("3D7", val["e3"], val["sd3"]), ("W2", val["ew"], val["sdw"]))):
        yy = y - 0.18 - 0.42 * j
        ax.plot([nm(m - s), nm(m + s)], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
        ax.plot(nm(m), yy, "o", color=BLUE, ms=8, mec=BG, mew=1.5, zorder=3)
        ax.text(nm(m + s) * 1.08, yy, f"{strain}: {nm(m):.0f} nM", fontsize=10, va="center", color=TXT)
        ypos.append(yy)
        ylabels.append(f"{c}  {strain}")
    y -= 1.75
ax.axvline(THRESH_NM, color=SEC, lw=0.9, zorder=0)
ax.axvline(SUBMICRO_NM, color=SEC, lw=0.9, ls=(0, (3, 3)), zorder=0)
ax.text(THRESH_NM * 0.96, 1.12, "500 nM", ha="right", va="bottom", fontsize=10, color=SEC)
ax.text(SUBMICRO_NM * 1.04, 1.12, "1 μM", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(10, 20000)
ax.set_ylim(y + 0.85, 1.05)
ax.set_yticks(ypos)
ax.set_yticklabels(ylabels, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("EC50 (nM, log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
for sp in ("top", "right", "left"):
    ax.spines[sp].set_visible(False)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda x, _: f"{x:,.0f}"))
handles = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, mec=BG,
                  label="Table 3 mean ± SD (n = 3), converted to nM"),
           Line2D([], [], color=ORANGE, lw=2, marker=4, ms=8,
                  label="Text\u2019s claim: EC50 below this value")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)

Abstract names March and October as best twice-yearly spraying months; body, table and data give September–November

PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007743 · PMC7200023

Summary

The abstract says the best ultra-low volume (ULV) strategy was spraying twice a year "in March and October", with a median of 172,000 infections [IQR 158,000–183,000], a 52% reduction from no spraying. The Results text, Table 2, the Discussion and the legends of Figs 4, 8 and 10 give these same figures, and the same 549 spray days, for a different month pair: September and November. One Results sentence repeats the abstract's "October and March". The authors' public source data reproduce the reported 172,000 [158,000–183,000] for September–November, which ranks 1st of the 66 month pairs tested. March–October has a median of 219,402 infections, ranks 16th, and gives a 39% reduction, not 52%. The paper's stated question is when to spray, so the abstract gives the wrong answer to it, although every number it reports is correct.

What the paper reports

The abstract's ULV headline:

"The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." (Abstract)

The abstract also states the paper's question:

"The optimal time to deploy these interventions and their relative epidemiological effects are, however, not well understood." (Abstract)

The Results give the same numbers to a different pair, and describe the general timing pattern:

"The best strategy for spraying twice yearly was spraying in September and November (172,000; IQR: 158,000–183,000) (Fig 4C). Generally, undertaking the first spray in August or September (typically, just before the dengue season) and the second in October or November (typically, near the start of the dengue season) led to the fewest infections (Fig 4C). In this case, the strategy that led to the fewest infections also led to the lowest average mosquito abundance: spraying in September and November (S4 Fig)." (Results, "Ultra-low volume spraying", paragraph 1)

The next paragraph uses the abstract's pair:

"Comparing the best strategies, spraying twice yearly (in October and March) averted the most infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a median of 549 days spraying in total over 11 years." (Results, "Ultra-low volume spraying", paragraph 2)

Table 2 (ULV rows, columns: None | Adaptive threshold | Yearly | Twice yearly):

"Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November" "Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | 261 [250, 277] | 172 [158, 183]" "Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]" (Table 2)

The Discussion and three figure legends agree with Table 2:

"The best strategy for ULV spraying was to spray twice per year, in September and November." (Discussion, "Summary of main findings")

"twice yearly corresponds to spraying in September and November." (Fig 4 legend, panel d; Fig 8 legend, panel c has "and twice yearly corresponds to spraying in September and November;")

"the bottom row the best twice yearly strategy (spraying in September and November)" (Fig 10 legend)

The Methods show that the two pairs are separate strategies, each with its own result:

"We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in each pair of months (66 strategies) (Table 1)." (Methods, "Experiments")

"Comparison of the median predicted infections for twice yearly spraying strategies, beginning on the first days of the shown month." (Fig 4 legend, panel c)

What the paper's own numbers show

Method. I first listed every statement in the paper that names the best twice-yearly ULV pair, and checked each against the PMC archive. Seven statements name September and November (Results paragraph 1, twice; Table 2; the Discussion; the Fig 4, 8 and 10 legends). Two name March and October (the Abstract and Results paragraph 2). I then checked the arithmetic of the reported values. Finally, I used the source data from the GitHub repository given in the paper's Data Availability Statement. The file figures/scripts/spray_analysis_paired_all_years.RData holds 400 simulations × 3 outputs × 66 month pairs. I mapped pairs to columns with the pair-index formula from the authors' own Fig 4 script, and took infections from output column 3, the column that script uses for Fig 4. For each pair I computed the median, the IQR (R's default quantile, which numpy also uses) and the median number of spray days. These data reproduce the paper's no-spraying, yearly-September and best-threshold results, so they match the published analysis. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed
Month pair with the fewest median infections (twice yearly, ULV) Abstract; Results para 2 March and October September–November (rank 1 of 66)
Same Results para 1; Table 2; Discussion; Fig 4d, 8c, 10 legends September and November September–November (rank 1 of 66)
Median infections [IQR], September–November Results para 1; Table 2 172,000 [158,000–183,000] 171,650 [158,131–183,442] (consistent)
Median infections [IQR], March–October Abstract (as the best pair) 172,000 [158,000–183,000] 219,402 [208,280–230,152]; rank 16 of 66
Reduction from no spraying, best twice-yearly pair Abstract 52% 1 − 172/361 = 52.4%; from source data, Sep–Nov 52.4%, Mar–Oct 39.2%
No spraying, median [IQR] Abstract; Table 2 361,000 [347,000–383,000] 360,831 [346,748–383,264] (consistent)
Yearly September, median [IQR]; days Results; Table 2 261,000 [250,000–277,000]; 274 260,708 [249,589–276,903]; 274 (consistent)
Spray days, twice yearly Results para 2; Table 2 549 22 campaigns × 25 days = 550; median 549 for both Sep–Nov and Mar–Oct (median across all 66 pairs is 548–549, so this figure does not identify a pair)
Best ULV vs best TIRS Results, TIRS 17-fold 172,000 / 9,900 = 17.4 (consistent)
Ranks of the four pairs with first spray Aug/Sep and second Oct/Nov Results para 1 ("Generally, ...") fewest infections ranks 1–4 of 66 (consistent with the body)

Two further checks need the RData file. Mar–Oct is not the best pair on the file's other outcome column either (rank 14 of 66, best is Sep–Nov), and not on mean infections (rank 15 of 66, best is Sep–Nov).

A secondary inconsistency, unrelated to the month pair. For the best TIRS strategy, the abstract gives "(9,900; [IQR: 8,720–11,400], a 94% reduction)". The Results give "9,900 infections (IQR: 8,720–11,300)", Table 2 gives "9.90 [8.72, 11.3]", and 1 − 9.9/361 = 97.3%. The Discussion's "97% reduction ... for repeated TIRS campaigns" is consistent with 97.3% (yearly TIRS gives 1 − 11.8/361 = 96.7%, also about 97%). The abstract's 94% and 11,400 do not match the body.

Figure

The abstract and the body attach the same result to different month pairs

Figure 1. The two orange-bordered panels quote the paper. On the left, the abstract (and Results paragraph 2) name March and October as the best twice-yearly ULV pair. On the right, the Results, Table 2 and the Discussion (and the Fig 4d, 8c and 10 legends) name September and November. Both give the same 172,000 [158,000–183,000] infections and 549 spray days. The blue strip below gives values recomputed in recompute.py.txt from the authors' source data. September–November has median 171,650 [158,131–183,442], rank 1 of 66, a 52% reduction, and matches the reported result. March–October has median 219,402 [208,280–230,152], rank 16 of 66, a 39% reduction, and does not match.

Twice-yearly month pairs ranked by median infections

Figure 2. Median and IQR of total infections over 2000–2010 (400 simulations each) for the 20 best of the 66 twice-yearly ULV month pairs, recomputed from the authors' source data. The top row, in orange, is the paper's reported best result, 172 [158, 183] thousand. The two pairs the paper names are in blue. September–November, named in Table 2 and most of the body, sits at the reported value. March–October, named in the abstract, ranks 16th, with an IQR that does not overlap the reported one. The thin vertical line marks the no-spraying median (361,000). The axis starts at zero.

Why it matters

The month pair is the result the paper sets out to find. The abstract describes timing as "not well understood", and the title is about optimizing deployment. A reader of the abstract alone would plan the first of two yearly ULV campaigns for March. The body puts the best first spray in August or September, "just before the dengue season", and the Author summary advises starting non-residual spraying "close to the start of the dengue transmission season". On the authors' own data, March–October gives a median of about 47,800 more infections over 11 years than September–November (219,402 vs 171,650). That is a 39% reduction instead of 52%, and fifteen pairs do better.

The error does not change the infection counts, the 52% figure (which belongs to September–November), the ranking of twice-yearly over yearly and threshold ULV strategies, the advantage of TIRS, or the recommendation of threshold-based TIRS. The paper's conclusions in the body stand. The problem is that the abstract, the part most often read, indexed and cited, gives a timing the paper's own analysis does not support. Results paragraph 2 repeats it, so it also appears in the main text.

Explanations considered

Typesetting or archiving artifact. Ruled out. The conflicting items are plain month names, not symbols, decimals or table alignment. An earlier audit stage found the same wording in the PLOS XML, the PLOS PDF and the PLOS article page, matching the PMC copy word for word. On 2026-10-23 I fetched the PLOS article page and confirmed that the abstract still reads "in March and October".

March–October and September–November as the same strategy under a different calendar. Ruled out. The Methods treat twice-yearly spraying as 66 unordered month pairs, each starting "on the first days of the shown month" every calendar year. March–October and September–November are separate strategies with separate results. In the source data they have different distributions (medians 219,402 vs 171,650, non-overlapping IQRs).

The abstract refers to a different outcome or summary statistic. Not supported. The abstract says "fewest median infections" and gives the same 172,000 [158,000–183,000] as Table 2. In the source data, March–October is not the best pair on the other output column (rank 14) or on mean infections (rank 15). The Results also say September–November gave the lowest mosquito abundance (S4 Fig).

The 549 spray days as evidence for March–October. Not informative. Every twice-yearly pair has 22 campaigns of 25 days. The median number of spray days is 548–549 for all 66 pairs, so "549 days" in Results paragraph 2 fits either wording.

Two statements against seven. The abstract is not the only place with March and October: Results paragraph 2 uses it too. But the paper reports one best twice-yearly strategy, attaches one set of numbers to it, and those numbers belong to September–November in both the paper's table and the source data. The paper does not explain where the March/October wording came from, and nothing here depends on how it arose.

Status

As of 2026-10-23:

Sources: PMC7200023 (PubMed Central archive); https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007743; Crossref and PMC notice records for 10.1371/journal.pcbi.1007743; source data from the repository named in the paper's Data Availability Statement.

Suggested correction

Change the abstract sentence to: "The ULV strategy with the fewest median infections was spraying twice yearly, in September and November, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." In Results ("Ultra-low volume spraying", paragraph 2), change "spraying twice yearly (in October and March)" to "spraying twice yearly (in September and November)". In the same notice, the authors could also reconcile the abstract's TIRS figures ("[IQR: 8,720–11,400], a 94% reduction") with the Results, Table 2 and the Discussion (IQR upper bound 11,300; 1 − 9.9/361 = 97%), or explain the basis of the 94%.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC7200023 (PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007743)
# Abstract vs body: which month pair is the best twice-yearly ULV spraying strategy?
#
# Usage:   python recompute.py.txt [path/to/spray_analysis_paired_all_years.RData]
# Needs:   numpy, matplotlib.  Optional: the 'rdata' package (pip install rdata) plus the
#          source-data file figures/scripts/spray_analysis_paired_all_years.RData from the
#          GitHub repository named in the paper's Data Availability Statement.
# Output:  prints every recomputed number used in writeup.md; writes fig1.svg/png and
#          fig2.svg/png next to this script.
#
# Sections
#   1. Every statement in the paper that names the best twice-yearly ULV month pair
#      (checked verbatim against the PMC archive when src/show.py is available).
#   2. Arithmetic from the paper's own reported numbers.
#   3. Per-pair results from the authors' source data. PAIR_STATS below holds, for each of
#      the 66 month pairs, the median, 25th and 75th percentile of total infections over
#      400 simulations (output column 3, as in the authors' Fig 4 script) and the median
#      number of spray days (output column 1). The table was produced by this script's
#      from_rdata() function; when the RData path is given, it is recomputed and checked.
#   4. Figures.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
MON = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()

# ---------------------------------------------------------------- 1. statements in the paper
STATEMENTS = [  # (location, verbatim text, month pair named)
    ("Abstract",
     "The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, "
     "which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline.",
     "Mar-Oct"),
    ("Results, ULV para 1",
     "The best strategy for spraying twice yearly was spraying in September and November (172,000; "
     "IQR: 158,000–183,000) (Fig 4C).", "Sep-Nov"),
    ("Results, ULV para 1",
     "Generally, undertaking the first spray in August or September (typically, just before the dengue "
     "season) and the second in October or November (typically, near the start of the dengue season) "
     "led to the fewest infections (Fig 4C).", "Aug/Sep then Oct/Nov"),
    ("Results, ULV para 2",
     "Comparing the best strategies, spraying twice yearly (in October and March) averted the most "
     "infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a "
     "median of 549 days spraying in total over 11 years.", "Mar-Oct"),
    ("Table 2, ULV best strategy row",
     "ULV | Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November",
     "Sep-Nov"),
    ("Table 2, ULV infections row",
     "ULV | Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | "
     "261 [250, 277] | 172 [158, 183]", "-"),
    ("Table 2, ULV days row",
     "ULV | Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]", "-"),
    ("Discussion, Summary of main findings",
     "The best strategy for ULV spraying was to spray twice per year, in September and November.", "Sep-Nov"),
    ("Fig 4 legend, panel (d)",
     "twice yearly corresponds to spraying in September and November.", "Sep-Nov"),
    ("Fig 8 legend, panel (c)",
     "and twice yearly corresponds to spraying in September and November;", "Sep-Nov"),
    ("Fig 10 legend",
     "the bottom row the best twice yearly strategy (spraying in September and November)", "Sep-Nov"),
    ("Methods, Experiments",
     "We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in "
     "each pair of months (66 strategies) (Table 1).", "-"),
    ("Author summary",
     "For non-residual insecticides, we find that it is best to begin spraying close to the start of "
     "the dengue transmission season, as mosquito populations can rebound quickly and resume previous "
     "levels of transmission.", "-"),
    ("Abstract (TIRS)",
     "(9,900; [IQR: 8,720–11,400], a 94% reduction)", "-"),
    ("Results, TIRS para 1",
     "resulting in 9,900 infections (IQR: 8,720–11,300)", "-"),
    ("Results, TIRS para 1",
     "The result was many fewer infections (17-fold) than the best ULV strategy, which led to a median "
     "of 172,000 infections.", "-"),
    ("Abstract (aim)",
     "The optimal time to deploy these interventions and their relative epidemiological effects are, "
     "however, not well understood.", "-"),
    ("Results, ULV para 1",
     "In this case, the strategy that led to the fewest infections also led to the lowest average "
     "mosquito abundance: spraying in September and November (S4 Fig).", "Sep-Nov"),
    ("Fig 4 legend, panel (c)",
     "Comparison of the median predicted infections for twice yearly spraying strategies, beginning on "
     "the first days of the shown month.", "-"),
    ("Discussion, Comparison to other studies",
     "Our prediction of a 97% reduction in the number of infections for repeated TIRS campaigns", "-"),
]

def check_quotes():
    show = os.path.join("/home/ubuntu/papers", "src", "show.py")
    py = "/home/ubuntu/papers-data/venv/bin/python"
    print("1) Statements in the paper (verbatim check against the PMC archive)")
    full = None
    if os.path.exists(show) and os.path.exists(py):
        r = subprocess.run([py, show, "PMC7200023", "--full"], cwd="/home/ubuntu/papers",
                           capture_output=True, text=True)
        full = r.stdout if r.returncode == 0 else None
    for loc, q, pair in STATEMENTS:
        tag = "not checked (archive tool unavailable)" if full is None else \
              ("found verbatim" if q in full else "NOT FOUND")
        print("   %-40s pair named: %-22s %s" % (loc, pair, tag))
    if full is not None:
        assert all(q in full for _, q, _ in STATEMENTS), "a quote was not found verbatim"

# ---------------------------------------------------------------- 2. paper's own arithmetic
REP = dict(baseline=361000, ulv2=172000, ulv2_lo=158000, ulv2_hi=183000, ulv2_days=549,
           tirs=9900, tirs_lo_abs=8720, tirs_hi_abs=11400, tirs_hi_res=11300,
           tirs_yearly=11800, campaigns=22, days_per_campaign=25, years=11)

def arithmetic():
    b = REP["baseline"]
    out = {}
    out["red_ulv2"] = 100 * (1 - REP["ulv2"] / b)
    out["red_tirs"] = 100 * (1 - REP["tirs"] / b)
    out["red_tirs_yearly"] = 100 * (1 - REP["tirs_yearly"] / b)
    out["fold"] = REP["ulv2"] / REP["tirs"]
    out["days"] = REP["campaigns"] * REP["days_per_campaign"]
    print("\n2) Arithmetic from the paper's reported numbers")
    print("   ULV twice-yearly reduction vs no spraying: 1 - 172/361 = %.1f%%  (abstract: 52%%)" % out["red_ulv2"])
    print("   22 campaigns = 2 per year x %d years: %d" % (REP["years"], 2 * REP["years"]))
    print("   22 campaigns x 25 days = %d days  (Table 2: median 549 [545, 552])" % out["days"])
    print("   Best ULV / best TIRS: 172,000 / 9,900 = %.1f-fold  (Results: 17-fold)" % out["fold"])
    print("   TIRS adaptive threshold reduction: 1 - 9.9/361 = %.1f%%  (abstract: 94%%; Discussion: 97%%)" % out["red_tirs"])
    print("   TIRS yearly (Sep) reduction: 1 - 11.8/361 = %.1f%%" % out["red_tirs_yearly"])
    print("   TIRS IQR upper bound: abstract 11,400 vs Results 11,300 and Table 2 11.3 (thousand)")
    return out

# ---------------------------------------------------------------- 3. authors' source data
# (i, j) 1-based months, i < j : (median, q25, q75 infections; median spray days)
PAIR_STATS = {
    (1, 2): (441708.00, 422530.50, 454094.25, 549.0),  # Jan-Feb
    (1, 3): (432794.00, 416724.00, 445201.75, 549.0),  # Jan-Mar
    (1, 4): (424317.50, 406502.25, 434176.75, 548.0),  # Jan-Apr
    (1, 5): (402984.50, 388331.75, 414365.25, 549.0),  # Jan-May
    (1, 6): (386051.00, 375338.75, 395504.25, 549.0),  # Jan-Jun
    (1, 7): (359459.50, 348058.50, 369248.25, 549.0),  # Jan-Jul
    (1, 8): (312679.00, 301817.75, 327553.00, 549.0),  # Jan-Aug
    (1, 9): (272079.00, 260225.00, 285449.25, 549.0),  # Jan-Sep
    (1, 10): (244997.50, 234358.25, 251953.75, 549.0),  # Jan-Oct
    (1, 11): (277347.00, 261847.25, 290349.25, 549.0),  # Jan-Nov
    (1, 12): (385329.00, 367194.00, 399237.00, 549.0),  # Jan-Dec
    (2, 3): (430668.50, 406035.50, 449033.00, 548.0),  # Feb-Mar
    (2, 4): (421545.00, 397439.75, 439812.00, 548.0),  # Feb-Apr
    (2, 5): (408950.50, 389504.75, 421645.25, 549.0),  # Feb-May
    (2, 6): (390143.00, 376047.50, 402808.00, 549.0),  # Feb-Jun
    (2, 7): (363220.50, 350213.75, 374916.50, 549.0),  # Feb-Jul
    (2, 8): (308439.00, 292948.00, 321977.75, 549.0),  # Feb-Aug
    (2, 9): (259513.00, 244273.75, 272738.75, 549.0),  # Feb-Sep
    (2, 10): (234932.00, 227319.50, 245102.25, 549.0),  # Feb-Oct
    (2, 11): (273375.50, 261919.75, 285630.00, 548.0),  # Feb-Nov
    (2, 12): (384562.50, 366859.00, 396050.25, 548.0),  # Feb-Dec
    (3, 4): (410443.00, 380868.50, 429001.00, 549.0),  # Mar-Apr
    (3, 5): (397934.50, 377044.50, 414733.25, 549.0),  # Mar-May
    (3, 6): (386533.00, 371343.00, 399381.25, 549.0),  # Mar-Jun
    (3, 7): (361814.50, 344152.50, 375015.50, 549.0),  # Mar-Jul
    (3, 8): (309186.00, 294559.75, 321067.00, 549.0),  # Mar-Aug
    (3, 9): (241259.00, 223740.00, 253670.50, 549.0),  # Mar-Sep
    (3, 10): (219402.50, 208280.50, 230152.00, 549.0),  # Mar-Oct
    (3, 11): (247274.00, 235556.50, 257797.25, 549.0),  # Mar-Nov
    (3, 12): (367927.00, 349634.25, 380767.75, 549.0),  # Mar-Dec
    (4, 5): (391103.00, 371128.25, 405921.00, 549.0),  # Apr-May
    (4, 6): (382233.00, 365624.50, 396880.50, 549.0),  # Apr-Jun
    (4, 7): (354168.50, 341128.75, 366589.25, 548.0),  # Apr-Jul
    (4, 8): (300416.50, 284996.50, 317564.00, 548.5),  # Apr-Aug
    (4, 9): (225956.00, 210163.00, 242784.25, 548.0),  # Apr-Sep
    (4, 10): (207010.00, 194652.50, 219858.50, 549.0),  # Apr-Oct
    (4, 11): (233842.50, 224488.50, 243371.25, 549.0),  # Apr-Nov
    (4, 12): (353418.00, 340148.00, 367024.50, 548.0),  # Apr-Dec
    (5, 6): (378949.00, 359401.25, 395366.00, 548.0),  # May-Jun
    (5, 7): (355846.00, 339660.00, 374676.00, 548.5),  # May-Jul
    (5, 8): (306893.50, 290227.25, 327889.75, 548.0),  # May-Aug
    (5, 9): (230192.50, 211397.25, 253140.75, 549.0),  # May-Sep
    (5, 10): (197582.00, 186184.00, 213469.75, 549.0),  # May-Oct
    (5, 11): (216630.00, 209058.00, 223355.25, 549.0),  # May-Nov
    (5, 12): (334639.00, 322462.75, 344777.25, 549.0),  # May-Dec
    (6, 7): (364683.00, 346838.50, 386625.75, 549.0),  # Jun-Jul
    (6, 8): (317711.00, 293881.75, 340245.75, 549.0),  # Jun-Aug
    (6, 9): (234496.00, 214429.00, 256260.50, 549.0),  # Jun-Sep
    (6, 10): (199118.00, 182987.75, 217445.00, 549.0),  # Jun-Oct
    (6, 11): (206853.50, 196577.75, 214035.00, 548.5),  # Jun-Nov
    (6, 12): (311791.00, 299312.75, 322937.25, 549.0),  # Jun-Dec
    (7, 8): (321920.00, 297495.75, 344139.50, 548.0),  # Jul-Aug
    (7, 9): (233935.50, 216610.50, 255196.75, 549.0),  # Jul-Sep
    (7, 10): (195357.50, 180265.75, 210540.25, 549.0),  # Jul-Oct
    (7, 11): (190111.00, 181210.75, 200435.50, 549.0),  # Jul-Nov
    (7, 12): (280071.00, 268760.75, 292520.75, 549.0),  # Jul-Dec
    (8, 9): (218566.50, 206763.50, 235713.50, 548.0),  # Aug-Sep
    (8, 10): (183435.00, 168294.25, 200184.25, 549.0),  # Aug-Oct
    (8, 11): (175365.50, 163921.25, 187042.25, 549.0),  # Aug-Nov
    (8, 12): (239777.00, 228854.50, 250519.25, 549.0),  # Aug-Dec
    (9, 10): (176823.50, 160305.75, 192265.75, 549.0),  # Sep-Oct
    (9, 11): (171650.50, 158131.00, 183441.50, 549.0),  # Sep-Nov
    (9, 12): (211443.50, 200702.00, 222217.50, 549.0),  # Sep-Dec
    (10, 11): (188972.50, 165478.00, 203092.00, 549.0),  # Oct-Nov
    (10, 12): (211302.00, 194398.25, 221180.25, 548.0),  # Oct-Dec
    (11, 12): (266319.00, 252818.25, 283819.75, 549.0),  # Nov-Dec
}
NONE_STATS = (360831.0, 346748.0, 383264.5)        # no spraying: median, q25, q75 (none.output col 3)
YEARLY_SEP = (260707.5, 249588.75, 276903.0, 274.0)  # yearly September: median, q25, q75, days
THRESH_BEST = (254206.0, 351.0)                      # best adaptive threshold (1 sd, monthly): median, days

def pair_index(i, j):
    """Column of tot.output[['spray_twice_yearly_all']] for months i<j (1-based), from the authors'
    Fig 4 script: pairs.by.month[i,j] <- -0.5*(i^2-25*i+24)+(j-i)."""
    return int(-0.5 * (i * i - 25 * i + 24) + (j - i))

def from_rdata(path):
    import rdata  # pip install rdata
    conv = rdata.conversion.convert(rdata.parser.parse_file(path), default_encoding="utf8")
    tw = np.asarray(conv["tot.output"]["spray_twice_yearly_all"])   # 400 sims x 3 outputs x 66 pairs
    yr = np.asarray(conv["tot.output"]["spray_yearly_all"])
    th = np.asarray(conv["tot.output"]["spray_threshold_mean"])
    no = np.asarray(conv["none.output"])
    assert tw.shape == (400, 3, 66)
    stats = {}
    for i in range(1, 12):
        for j in range(i + 1, 13):
            k = pair_index(i, j) - 1
            x = tw[:, 2, k]
            q25, med, q75 = np.quantile(x, [0.25, 0.5, 0.75])   # = R quantile type 7
            stats[(i, j)] = (round(med, 2), round(q25, 2), round(q75, 2), round(np.median(tw[:, 0, k]), 1))
    none = tuple(np.quantile(no[:, 2], [0.5, 0.25, 0.75]))
    ysep = tuple(np.quantile(yr[:, 2, 8], [0.5, 0.25, 0.75])) + (np.median(yr[:, 0, 8]),)
    tbest = (np.median(th[:, 2, 0]), np.median(th[:, 0, 0]))
    # Is Mar-Oct the best pair under any other summary in the file? (only with the RData file)
    pairs = list(stats)
    for label, f in [("output column 2, median", lambda k: np.median(tw[:, 1, k])),
                     ("output column 3 (infections), mean", lambda k: np.mean(tw[:, 2, k]))]:
        v = {p: f(pair_index(*p) - 1) for p in pairs}
        order = sorted(pairs, key=v.get)
        print("   [RData] %s: best pair %s-%s; Mar-Oct ranks %d of 66"
              % (label, MON[order[0][0] - 1], MON[order[0][1] - 1], order.index((3, 10)) + 1))
    return stats, none, ysep, tbest

def source_data(rpath):
    print("\n3) Authors' source data (400 simulations per strategy)")
    if rpath:
        stats, none, ysep, tbest = from_rdata(rpath)
        for k, v in PAIR_STATS.items():
            assert np.allclose(v, stats[k], atol=0.01), (k, v, stats[k])
        assert np.allclose(none, NONE_STATS) and np.allclose(ysep, YEARLY_SEP) and np.allclose(tbest, THRESH_BEST)
        print("   recomputed from %s: matches the embedded table for all 66 pairs" % os.path.basename(rpath))
    else:
        print("   (no RData path given: using the embedded table produced by from_rdata())")
    ranked = sorted(PAIR_STATS.items(), key=lambda kv: kv[1][0])
    rank = {k: r + 1 for r, (k, _) in enumerate(ranked)}
    b = NONE_STATS[0]
    print("   No spraying: median %.0f [IQR %.0f-%.0f]  (paper: 361 [347, 383] thousand)" % NONE_STATS)
    print("   Yearly Sep: median %.0f [IQR %.0f-%.0f], %.0f days  (paper: 261 [250, 277]; 274 days)" % YEARLY_SEP)
    print("   Best threshold: median %.0f, %.0f days  (paper: 254; 351 days)" % THRESH_BEST)
    print("   Five best twice-yearly pairs:")
    for (i, j), v in ranked[:5]:
        print("     %2d. %s-%s  median %.0f [IQR %.0f-%.0f], %.0f spray days" % ((rank[(i, j)], MON[i-1], MON[j-1]) + v))
    out = {}
    for (i, j), name in [((9, 11), "Sep-Nov"), ((3, 10), "Mar-Oct")]:
        m, lo, hi, d = PAIR_STATS[(i, j)]
        red = 100 * (1 - m / b)
        out[name] = (m, lo, hi, d, rank[(i, j)], red)
        print("   %s: median %.1f [IQR %.1f-%.1f], %.0f spray days, rank %d of 66, %.1f%% fewer than no spraying"
              % (name, m, lo, hi, d, rank[(i, j)], red))
    print("   Mar-Oct minus Sep-Nov median infections: %.0f; pairs better than Mar-Oct: %d"
          % (out["Mar-Oct"][0] - out["Sep-Nov"][0], out["Mar-Oct"][4] - 1))
    print("   IQRs overlap? %s" % (out["Mar-Oct"][1] <= out["Sep-Nov"][2]))
    days = sorted(set(v[3] for v in PAIR_STATS.values()))
    print("   Median spray days across all 66 pairs: %s (so 549 days does not identify a pair)" % days)
    print("   Pairs with first spray in Aug/Sep and second in Oct/Nov, ranks: %s" %
          sorted(rank[(i, j)] for i in (8, 9) for j in (10, 11)))
    return ranked, rank, out

# ---------------------------------------------------------------- 4. figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#b9b7b1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": FG,
                     "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
                     "axes.edgecolor": FG2, "figure.facecolor": BG, "axes.facecolor": BG,
                     "savefig.facecolor": BG, "svg.fonttype": "none"})

def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)

def fig1(src):
    sn, mo = src["Sep-Nov"], src["Mar-Oct"]
    fig = plt.figure(figsize=(7.2, 4.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")

    def panel(x, y, w, h, edge, title, body, sub):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                    fc="white", ec=edge, lw=1.5))
        ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=FG)
        ax.text(x + 0.14, y + h - 0.50, body, ha="left", va="top", fontsize=10, color=FG, linespacing=1.4)
        ax.text(x + 0.14, y + 0.14, sub, ha="left", va="bottom", fontsize=10, color=FG2, linespacing=1.3)

    panel(0.12, 1.95, 3.05, 2.85, ORANGE, "Abstract",
          "“spraying twice yearly, in\n"
          "March and October, which led to\n"
          "a median of 172,000 infections\n"
          "[IQR: 158,000–183,000], a 52%\n"
          "reduction from baseline”",
          "Same pair in Results, ULV para 2:\n“in October and March” … 549 days")
    panel(4.03, 1.95, 3.05, 2.85, ORANGE, "Results, Table 2, Discussion",
          "“The best strategy for spraying\n"
          "twice yearly was spraying in\n"
          "September and November (172,000;\n"
          "IQR: 158,000–183,000)”\n"
          "Table 2: 172 [158, 183]; 549 days",
          "Same pair in the legends of\nFig 4d, Fig 8c and Fig 10")
    ax.add_patch(FancyArrowPatch((3.19, 3.45), (4.01, 3.45), arrowstyle="<|-|>", mutation_scale=14,
                                 lw=2, color=FG))
    ax.text(3.6, 3.68, "conflict", ha="center", va="bottom", fontsize=10, weight="bold", color=FG)
    ax.text(3.6, 3.22, "same\nnumbers,\ndifferent\nmonths", ha="center", va="top", fontsize=10, color=FG2)

    # recomputed strip
    ax.add_patch(FancyBboxPatch((0.12, 0.12), 6.96, 1.5, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="#eef4fb", ec=BLUE, lw=1.5))
    ax.text(0.26, 1.48, "Recomputed from the authors' source data (66 month pairs, 400 simulations each)",
            ha="left", va="top", fontsize=10, weight="bold", color=BLUE)
    ax.text(0.26, 1.16, "Mar–Oct:  median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
            % (f"{mo[0]:,.0f}", f"{mo[1]:,.0f}", f"{mo[2]:,.0f}", mo[4], mo[5]),
            ha="left", va="top", fontsize=10, color=FG)
    ax.text(0.26, 0.86, "Sep–Nov:  median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
            % (f"{sn[0]:,.0f}", f"{sn[1]:,.0f}", f"{sn[2]:,.0f}", sn[4], sn[5]),
            ha="left", va="top", fontsize=10, color=FG)
    ax.text(0.26, 0.56, "Both pairs: median %.0f spray days. The reported 172,000 [158,000–183,000]\n"
            "and 52%% reduction fit Sep–Nov only." % sn[3], ha="left", va="top", fontsize=10, color=FG2,
            linespacing=1.3)
    # connectors from strip to the panels
    ax.annotate("", xy=(1.65, 1.93), xytext=(1.65, 1.64),
                arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, linestyle=(0, (3, 2))))
    ax.text(1.75, 1.785, "does not match", ha="left", va="center", fontsize=10, color=BLUE)
    ax.annotate("", xy=(5.55, 1.93), xytext=(5.55, 1.64),
                arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
    ax.text(5.65, 1.785, "matches", ha="left", va="center", fontsize=10, color=BLUE)
    save(fig, "fig1")

def fig2(ranked, rank, src, n_show=20):
    rows = ranked[:n_show]
    fig, ax = plt.subplots(figsize=(7.2, 6.4))
    fig.subplots_adjust(left=0.22, right=0.97, top=0.86, bottom=0.09)
    y_rep = 0
    ax.errorbar(REP["ulv2"] / 1e3, y_rep,
                xerr=[[(REP["ulv2"] - REP["ulv2_lo"]) / 1e3], [(REP["ulv2_hi"] - REP["ulv2"]) / 1e3]],
                fmt="o", ms=8, color=ORANGE, lw=2, capsize=0, zorder=3)
    labels = ["Reported best pair\n172 [158, 183]"]
    for r, ((i, j), (m, lo, hi, d)) in enumerate(rows, start=1):
        hl = (i, j) in [(9, 11), (3, 10)]
        c = BLUE if hl else GRAY
        ax.errorbar(m / 1e3, r + 0.5, xerr=[[(m - lo) / 1e3], [(hi - m) / 1e3]], fmt="o",
                    ms=8 if hl else 6, color=c, lw=2 if hl else 1.5, capsize=0, zorder=3)
        labels.append("%d. %s–%s" % (r, MON[i - 1], MON[j - 1]))
        if hl:
            note = ("Pair named in the Abstract" if (i, j) == (3, 10) else "Pair named in Table 2")
            left = (i, j) == (3, 10)   # place the label where the row has free space
            ax.text((lo / 1e3 - 6) if left else (hi / 1e3 + 6), r + 0.5, "%s: %.1f k" % (note, m / 1e3),
                    va="center", ha="right" if left else "left",
                    fontsize=10, color=BLUE)
    ax.set_yticks([0] + [r + 0.5 for r in range(1, n_show + 1)])
    ax.set_yticklabels(labels, fontsize=10)
    for t in ax.get_yticklabels()[1:]:
        t.set_color(FG2)
    ax.get_yticklabels()[0].set_color(ORANGE)
    ax.invert_yaxis()
    ax.axvline(NONE_STATS[0] / 1e3, color=FG2, lw=1)
    ax.text(NONE_STATS[0] / 1e3 - 4, n_show + 0.9, "no spraying\n%.0f k" % (NONE_STATS[0] / 1e3),
            ha="right", va="bottom", fontsize=10, color=FG2)
    ax.set_xlim(0, 400)
    ax.set_ylim(n_show + 1.6, -0.8)
    ax.set_xlabel("Total infections 2000–2010, thousands (median and IQR, 400 simulations)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    fig.text(0.02, 0.975, "Twice-yearly ULV month pairs ranked by median infections (best %d of 66)" % n_show,
             fontsize=11, weight="bold", ha="left", va="top")
    fig.text(0.02, 0.938, "Orange: the paper's reported best twice-yearly result (Abstract and Table 2).\n"
             "Blue: the two named pairs, recomputed from the authors' source data. Gray: other pairs.",
             fontsize=10, color=FG2, ha="left", va="top", linespacing=1.3)
    save(fig, "fig2")

if __name__ == "__main__":
    check_quotes()
    arithmetic()
    rpath = sys.argv[1] if len(sys.argv) > 1 else None
    ranked, rank, src = source_data(rpath)
    fig1(src)
    fig2(ranked, rank, src)
    print("\nFigures written to", HERE)

Abstract reports SF2/SRSF1 decrease in astrocytes; the paper's data show it only in PNG and SPG

PLoS Genet (2023) · doi:10.1371/journal.pgen.1010973 · PMC10553832

Summary

The abstract says that after glial TDP-43 induction "SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes." The paper's own expression data, S8 Table, report a significant decrease only in perineurial glia (PNG; log2 fold change −1.31, p<0.0001) and subperineurial glia (SPG; −0.85, p<0.001). For astrocyte-like glia (ALG), the cell type the paper calls astrocytes, the table gives "n/a", and the Results say outright that TDP-43 induction in astrocytes "does not on its own cause a decrease in SF2/SRSF1 expression." So the abstract names the one cell type in which the body says there is no decrease and leaves out SPG, where there is one. The lifespan rescue by SF2/SRSF1 knockdown (Fig 5) comes from separate experiments and is not affected, but the abstract misstates the only specific transcriptomic result it names.

What the paper reports

Abstract:

Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens the detrimental effects of TDP-43 pathology on lifespan, but extends survival of the glial cells.

Results, section "Knockdown of SF2/SRSF1 in astrocytes or PNG ameliorates the toxicity of TDP-43", first paragraph:

Our differential expression analysis identified SF2/SRSF1 as being down regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, or EG (S8 Table).

Same section, second paragraph:

The fact that SF2/SRSF1 levels are reduced in SPG and PNG, but not other glial cell types, is intriguing given that pan-glial knockdown of SF2/SRSF1 can ameliorate the lifespan effects of pan-glial TDP-43 pathology (Fig 5A) [55]. We observe a substantial impact on lifespan from pathological TDP-43 induction in PNG and ALG (Fig 1), but only see down regulation of SF2/SRSF1 in PNG (S8 Table).

And although TDP-43 induction in astrocytes does not on its own cause a decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by RNAi does ameliorate the effects on lifespan from astrocytic induction of TDP-43 pathology.

Introduction, last paragraph:

We observe that in response to TDP-43 pathology, SF2/SRSF1 is down-regulated in both the PNG and SPG specialized BBB glia.

Discussion, paragraph on the expression profiling:

Notably, the SF2/SRSF1 gene exhibits decreased expression in PNG and SPG, but not other glial cell types (S8 Table).

How the paper identifies astrocytes. Introduction:

the cortex glia (CG), astrocyte-like glia (ALG), and ensheathing glia (EG) provide many of the functional characteristics of mammalian astrocytes and microglia

The strain table (Table 2) lists "R86E01-Gal4 (ALG)". The Fig 5C genotype in the Methods is "Tubulin-Gal80 ts / UAS-SF2-IR; UAS-TDP-43/ R86E01-Gal4", and the Fig 5 legend labels that panel "ALGts + TDP-43 + SF2-IR". The Results describe the same experiment as "the PNG or Astrocyte Gal4 lines" and "TDP-43 induction in either PNG or ALG (Fig 5B and 5C)". In this paper "astrocytes" and ALG are the same cells.

S8 Table (supplementary file pgen.1010973.s011), legend:

SF2 expression was significantly reduced at day 2 post induction of TDP-43 in SPG and PNG day 2.

The table's body, paraphrased from the PDF (row label "SF2 expression (MRN)"; values per group, given here as control → +TDP-43, then log2 fold change and p):

The column headers give n = 3 libraries per group, except n = 2 for the CG and ALG controls. The table does not say whether the per-group values are means, and it does not define "n/a"; the Results' "but not in CG, ALG, or EG" supports reading "n/a" as not significant. The Methods say the analysis used DESeq2 with a Benjamini–Hochberg FDR threshold of p < 0.05.

What the paper's own numbers show

Method. I took the five cell types' per-group SF2/SRSF1 expression values (control and TDP-43) and the reported log2 fold changes and p values from S8 Table. For each cell type I computed the raw log2 ratio of the two group values, treating them as group-level summaries. This is not the DESeq2 estimate, which is model-based and works on per-library counts, so it can only check that the table's group values and its reported fold changes agree; it is not a test. I then listed which cell types the Abstract, Introduction, Results, Discussion and S8 Table each say are decreased, and compared the lists. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
PNG, SF2/SRSF1 change S8 Table; Results 899 → 363 MRN; log2FC −1.31, p<0.0001 log2(363/899) = −1.308; group values fall 59.6% (2^−1.31 = 0.403, a 59.7% fall)
SPG, SF2/SRSF1 change S8 Table; Results 1090 → 594 MRN; log2FC −0.85, p<0.001 log2(594/1090) = −0.876; group values fall 45.5% (2^−0.85 = 0.555, a 44.5% fall). The 0.026 gap from −0.85 is consistent with model-based estimation
ALG (astrocyte-like glia), SF2/SRSF1 change S8 Table; Results 591 → 453 MRN; log2FC n/a, p n/a log2(453/591) = −0.384 (−23.4%); not reported as significant ("n/a", read with the Results' "but not in ... ALG")
CG, SF2/SRSF1 change S8 Table 780 → 727 MRN; n/a log2(727/780) = −0.102 (−6.8%)
EG, SF2/SRSF1 change S8 Table 780 → 669 MRN; n/a log2(669/780) = −0.221 (−14.2%)
Cell types with a significant decrease S8 Table and legend PNG, SPG PNG, SPG
Cell types with a decrease, per the text Introduction; Results; Discussion PNG and SPG (Results and Discussion add: not CG, ALG or EG) agrees with S8 Table
Cell types with a decrease, per the abstract Abstract PNG and astrocytes (ALG) ALG is not significant in S8 Table; SPG, which is significant, is not named

Figure

Abstract versus Results and S8 Table on SF2/SRSF1 in astrocytes

Figure 1. The abstract sentence (left) and the Results sentences and S8 Table entry for the same cell type (right), quoted verbatim from the article as archived in PMC. The abstract names astrocytes, which the paper identifies as ALG, as a cell type in which SF2/SRSF1 decreased. The Results and S8 Table report no significant change in ALG (591 to 453 MRN, log2 fold change "n/a"), and the Results state that TDP-43 induction in astrocytes does not on its own decrease SF2/SRSF1. The Introduction and Discussion give the decrease as PNG and SPG.

SF2/SRSF1 log2 fold change by glial cell type

Figure 2. SF2/SRSF1 log2 fold change after TDP-43 induction (day 2) in each glial cell type. Orange diamonds are the DESeq2 estimates reported in S8 Table and the Results, which exist only for PNG and SPG; the other three cell types are given as "n/a", which the Results describe as not down-regulated. Blue circles are the raw log2 ratio of the S8 Table group values, computed by recompute.py.txt as a consistency check. The vertical line marks no change. The ALG point, which the abstract describes as decreased, is a non-significant −0.38.

Why it matters

The abstract is what most readers and indexing services see, and this sentence is the only specific transcriptomic result the abstract names (it otherwise says only that the authors "identified numerous glial cell-type specific transcriptional changes"). As written, it tells readers that TDP-43 lowers SF2/SRSF1 in the two cell types with the strongest lifespan effects (PNG and astrocytes), and the next sentence's "further knockdown" builds on that. The body's account is different, and the Results call it "intriguing": the decrease is confined to the two blood-brain-barrier glia (PNG and SPG), it is absent in astrocytes, and knockdown in astrocytes still extends lifespan. The Results draw exactly this contrast ("although TDP-43 induction in astrocytes does not on its own cause a decrease..."). A reader who relies on the abstract is told there is a decrease in astrocytes, where the paper reports no significant change, and does not learn about the SPG decrease.

What is not affected: the lifespan extension from SF2/SRSF1 knockdown in pan-glial, PNG and ALG TDP-43 models (Fig 5A–C, log-rank tests), the delayed glial loss with knockdown (Fig 5D–E), and the reaper ablation result in PNG. These are separate experiments and do not depend on the expression data. The title's claim stands.

Explanations considered

"Astrocytes" in the abstract means a different cell type, such as SPG. This does not fit. The paper uses "astrocytes" for ALG throughout: the Results section title is "Knockdown of SF2/SRSF1 in astrocytes or PNG", the experiment uses "the PNG or Astrocyte Gal4 lines" and reports the outcome as "PNG or ALG (Fig 5B and 5C)", and the Fig 5C driver is R86E01-Gal4, listed in Table 2 as the ALG line. SPG is described as a blood-brain-barrier glial type, alongside PNG, and is never called astrocytes.

A different time point or dataset shows an ALG decrease. None found. S8 Table covers one time point (day 2 after induction), the same as the RNA-seq design described in the Results, and the text cites no other SF2/SRSF1 expression data.

The abstract describes a non-significant trend. The ALG group values do fall (raw log2 ratio −0.38, from n = 2 control libraries), but the paper reports this as n/a, and its own Results rule out this reading by saying ALG is not down-regulated and that astrocytic TDP-43 "does not on its own cause a decrease." The abstract's wording ("were found to be decreased", the same wording used for PNG) presents it as a finding.

Text conversion or archiving error. Not supported. The PLOS article page, the Europe PMC full text and the PMC archive have the same abstract sentence, and the S8 Table PDF from the publisher has the values above.

The small SPG discrepancy (−0.876 vs −0.85). A ratio of group values need not equal a DESeq2 model estimate. This gap is small and does not bear on the finding; the PNG value agrees to within 0.002.

A narrower reading of one Results sentence. "...only see down regulation of SF2/SRSF1 in PNG" agrees with the table: it compares only PNG and ALG, the two cell types with the strongest lifespan effects, and SPG is not part of that comparison.

Status

As of 2026-10-23:

Sources: the article as archived in PMC (PMC10553832); journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1010973 and its supplementary file s011; Crossref and Europe PMC records for doi:10.1371/journal.pgen.1010973.

Suggested correction

In the abstract, replace

Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...

with wording that matches the Results and S8 Table, for example:

Notably, SF2/SRSF1 levels were found to be decreased in PNG and SPG, but not in astrocytes. We found that knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...

The word "further" would then apply only to PNG. If the authors prefer to keep the sentence about the two cell types tested in Fig 5, it could instead say that SF2/SRSF1 was decreased in PNG but not in astrocytes.

Recomputation script (python recompute.py.txt)
"""Recompute the SF2/SRSF1 cell-type result of PMC10553832
(PLoS Genet 2023, doi:10.1371/journal.pgen.1010973).

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure designs
(design_A ... design_E, .svg and .png) in ./candidates/.

Inputs are copied from the paper:
  S8 Table (supplementary file pgen.1010973.s011, "SF2 expression across glial cell
  types"): the per-group SF2 expression value (row label "SF2 expression (MRN)") for
  control and +TDP-43 libraries at day 2 after induction, the reported log2 fold change
  and p value. The table does not define "n/a"; the Results' "but not in CG, ALG, or EG"
  supports reading it as not significant. The table does not say whether the per-group
  values are means; they are treated here as group-level summaries.
  Text excerpts: Abstract, Introduction, Results ("Knockdown of SF2/SRSF1 in astrocytes
  or PNG ameliorates the toxicity of TDP-43"), Discussion, as archived in PMC.
The raw log2 ratio of two group values is not the DESeq2 estimate the paper reports
(which is model-based and uses per-library normalized counts); it is shown only as a
check that the reported values and the table's group values agree.
"""
import math
import os
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (from the paper)
# cell type: (n control, MRN control, n TDP-43, MRN TDP-43, reported log2FC, reported p)
S8 = {
    "PNG": (3, 899, 3, 363, -1.31, "<0.0001"),
    "SPG": (3, 1090, 3, 594, -0.85, "<0.001"),
    "CG": (2, 780, 3, 727, None, "n/a"),
    "ALG": (2, 591, 3, 453, None, "n/a"),
    "EG": (3, 780, 3, 669, None, "n/a"),
}
LONG = {
    "PNG": "PNG (perineurial glia)",
    "SPG": "SPG (subperineurial glia)",
    "CG": "CG (cortex glia)",
    "ALG": "ALG (astrocyte-like glia)",
    "EG": "EG (ensheathing glia)",
}
ORDER = ["PNG", "SPG", "CG", "ALG", "EG"]

ABSTRACT = ("Notably, SF2/SRSF1 levels were found to be decreased in both PNG "
            "and in astrocytes.")
RESULTS_1 = ("Our differential expression analysis identified SF2/SRSF1 as being down "
             "regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, "
             "p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, "
             "or EG (S8 Table).")
RESULTS_2 = ("And although TDP-43 induction in astrocytes does not on its own cause a "
             "decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by "
             "RNAi does ameliorate the effects on lifespan from astrocytic induction of "
             "TDP-43 pathology.")

# What each part of the paper says about an SF2/SRSF1 decrease in each cell type.
# "dec" = says decreased, "not" = says not decreased, None = cell type not named.
SAYS = {
    "Abstract": {"PNG": "dec", "SPG": None, "CG": None, "ALG": "dec", "EG": None},
    "Introduction": {"PNG": "dec", "SPG": "dec", "CG": None, "ALG": None, "EG": None},
    "Results": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
    "Discussion": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
}

# ---------------------------------------------------------------- style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
REP = "#eb6834"   # values as reported by the paper
REC = "#2a78d6"   # values recomputed from the paper's own numbers
GRID = "#e4e3df"
PANEL = "#f3f2ee"
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans",
    "font.size": 10,
    "axes.edgecolor": INK2,
    "axes.labelcolor": INK,
    "xtick.color": INK2,
    "ytick.color": INK2,
    "text.color": INK,
    "svg.fonttype": "none",
    "lines.linewidth": 2,
})


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg", facecolor=BG)
    fig.savefig(path_noext + ".png", dpi=200, facecolor=BG)
    plt.close(fig)


def wrap(s, n):
    return "\n".join(textwrap.wrap(s, n))


# ---------------------------------------------------------------- recomputation
print("S8 Table, SF2/SRSF1 expression (MRN) per group, day 2 after induction")
print(f"{'cell type':28s}{'ctrl':>6s}{'TDP43':>7s}{'ratio':>8s}{'raw log2':>10s}"
      f"{'change':>9s}{'reported':>10s}{'p':>9s}")
RAW = {}
for k in ORDER:
    nc, c, nt, t, r, p = S8[k]
    ratio = t / c
    RAW[k] = math.log2(ratio)
    print(f"{LONG[k]:28s}{c:6d}{t:7d}{ratio:8.3f}{RAW[k]:10.3f}{100 * (ratio - 1):8.1f}%"
          f"{('' if r is None else f'{r:.2f}'):>10s}{p:>9s}")

print("\nReported log2 fold changes converted to percent change")
for k in ("PNG", "SPG"):
    r = S8[k][4]
    print(f"  {k}: 2^({r}) = {2 ** r:.3f}  -> {100 * (2 ** r - 1):.1f}%")
print(f"  SPG: reported -0.85 vs raw log2 ratio of group values {RAW['SPG']:.3f} "
      f"(difference {RAW['SPG'] - (-0.85):.3f})")
print(f"  PNG: reported -1.31 vs raw log2 ratio of group values {RAW['PNG']:.3f} "
      f"(difference {RAW['PNG'] - (-1.31):.3f})")

sig = [k for k in ORDER if S8[k][5] != "n/a"]
print("\nCell types with a significant SF2/SRSF1 decrease in S8 Table:", ", ".join(sig))
abstract_types = ["PNG", "ALG"]  # 'astrocytes' = ALG, see writeup
print("Cell types the abstract says are decreased: PNG, astrocytes (= ALG)")
print("Named in the abstract but not significant in S8 Table:",
      ", ".join(k for k in abstract_types if k not in sig))
print("Significant in S8 Table but not named in the abstract:",
      ", ".join(k for k in sig if k not in abstract_types))
print("\nWhat each section says (dec = decreased, not = not decreased, - = not named)")
print(f"{'':14s}" + "".join(f"{k:>6s}" for k in ORDER))
for sec, d in SAYS.items():
    print(f"{sec:14s}" + "".join(f"{(d[k] or '-'):>6s}" for k in ORDER))
print(f"{'S8 Table':14s}" + "".join(f"{('dec' if k in sig else 'n/a'):>6s}" for k in ORDER))


# ================================================================ design A: two text panels
def design_a(path):
    fig = plt.figure(figsize=(W, 4.9), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    L = (0.025, 0.385)
    R = (0.585, 0.975)
    top, bot = 0.875, 0.17
    for (x0, x1) in (L, R):
        ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot,
                                    boxstyle="round,pad=0,rounding_size=0.015",
                                    fc=PANEL, ec=GRID, lw=1))
    ax.text(L[0] + 0.015, top - 0.04, "Abstract", fontsize=12, weight="bold", va="top")
    ax.text(L[0] + 0.015, top - 0.12, wrap("“" + ABSTRACT + "”", 33),
            fontsize=10, va="top", linespacing=1.35)
    ax.text(L[0] + 0.015, bot + 0.16, "Cell types said to decrease:", fontsize=10,
            color=INK2, va="top")
    ax.text(L[0] + 0.015, bot + 0.095, "PNG, astrocytes (ALG)", fontsize=10.5,
            weight="bold", color=INK, va="top")

    ax.text(R[0] + 0.015, top - 0.04, "Results and S8 Table", fontsize=12,
            weight="bold", va="top")
    ax.text(R[0] + 0.015, top - 0.12,
            wrap("“…down regulated in response to TDP-43 expression in PNG "
                 "(-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, "
                 "p < .001), but not in CG, ALG, or EG (S8 Table).”", 35),
            fontsize=10, va="top", linespacing=1.35)
    ax.text(R[0] + 0.015, top - 0.405,
            wrap("“…TDP-43 induction in astrocytes does not on its own cause a "
                 "decrease in SF2/SRSF1 expression…”", 35),
            fontsize=10, va="top", linespacing=1.35)
    ax.text(R[0] + 0.015, bot + 0.16, "S8 Table, ALG: 591 → 453 MRN,", fontsize=10,
            color=INK2, va="top")
    ax.text(R[0] + 0.015, bot + 0.095, "log2FC n/a (not significant)", fontsize=10.5,
            weight="bold", va="top")

    ymid = 0.52
    ax.annotate("", xy=(R[0] - 0.008, ymid), xytext=(L[1] + 0.008, ymid),
                arrowprops=dict(arrowstyle="<|-|>", color=INK, lw=2,
                                shrinkA=0, shrinkB=0, mutation_scale=14))
    ax.text((L[1] + R[0]) / 2, ymid + 0.035, "Conflict:\nastrocytes (ALG)",
            ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.3)
    ax.text((L[1] + R[0]) / 2, ymid - 0.035, "“decreased”\nvs\nnot decreased",
            ha="center", va="top", fontsize=10, color=INK2, linespacing=1.3)
    ax.text(0.025, 0.95,
            "SF2/SRSF1 after TDP-43 induction: what the abstract says vs the paper's data",
            fontsize=11, weight="bold", va="top")
    ax.text(0.025, 0.13,
            "Introduction and Discussion agree with the Results: the decrease is in PNG "
            "and SPG.\nSPG is not named in the abstract.",
            fontsize=10, color=INK2, va="top")
    ax.text(0.025, 0.05, "Quotes verbatim from the article as archived in PMC; "
            "S8 Table = supplementary file s011.", fontsize=10, color=INK2, va="top")
    save(fig, path)


# ================================================================ design B: section x cell-type matrix
def design_b(path):
    secs = list(SAYS) + ["S8 Table"]
    fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
    fig.subplots_adjust(left=0.27, right=0.985, top=0.81, bottom=0.17)
    ax.set_facecolor(BG)
    ax.set_xlim(0, len(secs))
    ax.set_ylim(len(ORDER), 0)
    for j, sec in enumerate(secs):
        for i, k in enumerate(ORDER):
            if sec == "S8 Table":
                nc, c, nt, t, r, p = S8[k]
                txt = f"{r:.2f}\np{p}" if r is not None else "n/a"
                state = "dec" if r is not None else "not"
                col = REP
            else:
                state = SAYS[sec][k]
                txt = {"dec": "decreased", "not": "not\ndecreased", None: "—"}[state]
                col = INK if state else INK2
                if sec == "Abstract" and k == "ALG":
                    txt = "decreased\n(conflict)"
            fc = "#e9e7e1" if state == "dec" else BG
            ax.add_patch(Rectangle((j + 0.04, i + 0.06), 0.92, 0.88, fc=fc, ec=GRID, lw=1))
            ax.text(j + 0.5, i + 0.5, txt, ha="center", va="center", fontsize=10,
                    color=col, linespacing=1.15,
                    weight="bold" if (sec == "S8 Table" and state == "dec")
                    or (sec == "Abstract" and k == "ALG") else "normal")
    # conflict cell
    i = ORDER.index("ALG")
    ax.add_patch(Rectangle((0.04, i + 0.06), 0.92, 0.88, fc="none", ec=INK, lw=2.5))
    ax.set_xticks([j + 0.5 for j in range(len(secs))])
    ax.set_xticklabels(secs, fontsize=10)
    ax.xaxis.tick_top()
    ax.set_yticks([i + 0.5 for i in range(len(ORDER))])
    ax.set_yticklabels([LONG[k] for k in ORDER], fontsize=10)
    ax.tick_params(length=0)
    for s in ax.spines.values():
        s.set_visible(False)
    fig.text(0.015, 0.965, "Where the paper says SF2/SRSF1 decreases after TDP-43 induction",
             fontsize=11, weight="bold", va="top")
    fig.text(0.015, 0.06, "Shaded = says decreased; orange = S8 Table's reported log2 fold "
             "change and p; \u2014 = not named.", fontsize=10, color=INK2)
    fig.text(0.015, 0.015, "Astrocytes in the abstract are the ALG row (the paper's "
             "astrocyte Gal4 line, R86E01, drives ALG).", fontsize=10, color=INK2)
    save(fig, path)


# ================================================================ design C: log2FC dot plot
def design_c(path):
    fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
    fig.subplots_adjust(left=0.27, right=0.72, top=0.79, bottom=0.215)
    ax.set_facecolor(BG)
    ys = {k: i for i, k in enumerate(ORDER)}
    ax.axvline(0, color=INK2, lw=1)
    for k in ORDER:
        y = ys[k]
        ax.plot(RAW[k], y + 0.13, "o", ms=9, color=REC, mec=BG, mew=1.5, zorder=3)
        r = S8[k][4]
        if r is not None:
            ax.plot(r, y - 0.13, "D", ms=8.5, color=REP, mec=BG, mew=1.5, zorder=3)
        lab = (f"reported {r:.2f}, p{S8[k][5]}" if r is not None
               else "reported n/a (not sig.)")
        ax.text(1.03, y, lab, transform=ax.get_yaxis_transform(), va="center",
                fontsize=10, color=INK if r is not None else INK2, clip_on=False)
    ax.set_yticks(range(len(ORDER)))
    ax.set_yticklabels([LONG[k] for k in ORDER])
    ax.set_ylim(len(ORDER) - 0.5, -0.6)
    ax.set_xlim(-1.5, 0.25)
    ax.set_xticks([-1.5, -1.0, -0.5, 0.0])
    ax.set_xlabel("SF2/SRSF1 log2 fold change, TDP-43 vs control (day 2)")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(length=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.text(0.03, 0, "no change", rotation=90, fontsize=10, color=INK2,
            transform=ax.get_xaxis_transform(), ha="left", va="bottom")
    yalg = ys["ALG"]
    ax.annotate("abstract: “decreased”\n(astrocytes)",
                xy=(RAW["ALG"] - 0.04, yalg + 0.13), xytext=(-1.48, yalg + 0.2),
                fontsize=10, weight="bold", va="center",
                arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
    handles = [Line2D([], [], marker="D", ls="", color=REP, ms=8.5,
                      label="reported in S8 Table / Results (DESeq2)"),
               Line2D([], [], marker="o", ls="", color=REC, ms=9,
                      label="recomputed: log2(TDP-43 / control), S8 Table group values")]
    fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
               frameon=False, fontsize=10, ncol=1, handletextpad=0.4)
    fig.text(0.015, 0.975, "SF2/SRSF1 change by glial cell type: only PNG and SPG are "
             "significant", fontsize=11, weight="bold", va="top")
    fig.text(0.015, 0.025, "The recomputed ratio of group values is a consistency check, "
             "not a substitute for the DESeq2 test.", fontsize=10, color=INK2)
    save(fig, path)


# ================================================================ design D: dumbbell of group MRN
def design_d(path):
    fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
    fig.subplots_adjust(left=0.27, right=0.73, top=0.79, bottom=0.215)
    ax.set_facecolor(BG)
    for i, k in enumerate(ORDER):
        nc, c, nt, t, r, p = S8[k]
        ax.plot([t, c], [i, i], color=REP, lw=2, alpha=0.55, zorder=2)
        ax.plot(c, i, "o", ms=9, mfc=BG, mec=REP, mew=2, zorder=3)
        ax.plot(t, i, "o", ms=9, color=REP, mec=REP, zorder=3)
        pctc = 100 * (t / c - 1)
        sigtxt = f"S8: p{p}" if r is not None else "S8: n/a"
        ax.text(1.03, i, f"{pctc:+.0f}%  ", transform=ax.get_yaxis_transform(),
                va="center", ha="left", fontsize=10, color=REC, weight="bold")
        ax.text(1.17, i, sigtxt, transform=ax.get_yaxis_transform(), va="center",
                ha="left", fontsize=10, color=INK if r is not None else INK2)
    ax.set_yticks(range(len(ORDER)))
    ax.set_yticklabels([LONG[k] for k in ORDER])
    ax.set_ylim(len(ORDER) - 0.5, -0.6)
    ax.set_xlim(0, 1200)
    ax.set_xlabel("SF2/SRSF1 expression (MRN), S8 Table group values")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.tick_params(length=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ia = ORDER.index("ALG")
    ax.annotate("abstract: “decreased”", xy=(453, ia), xytext=(40, ia + 0.42),
                fontsize=10, weight="bold", va="center",
                arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
    handles = [Line2D([], [], marker="o", ls="", mfc=BG, mec=REP, mew=2, ms=9,
                      label="control (reported)"),
               Line2D([], [], marker="o", ls="", color=REP, ms=9,
                      label="+TDP-43 (reported)"),
               Line2D([], [], ls="", marker="$\\%$", color=REC, ms=10,
                      label="change (recomputed)")]
    fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
               frameon=False, fontsize=10, ncol=3, handletextpad=0.3, columnspacing=1.2)
    fig.text(0.015, 0.975, "SF2/SRSF1 expression, control vs TDP-43, by glial cell type",
             fontsize=11, weight="bold", va="top")
    fig.text(0.015, 0.025, "Axis starts at zero. Controls: n = 2 libraries for CG and ALG, "
             "3 otherwise; TDP-43: n = 3.", fontsize=10, color=INK2)
    save(fig, path)


# ================================================================ design E: sentence revision
def runs(ax, x, y, parts, fs=10.5, fig_w=W):
    """Draw one line of text runs in axes-fraction coordinates.
    parts = [(text, dict(color=..., weight=..., strike=bool, under=bool))]."""
    fig = ax.figure
    fig.set_dpi(200)  # measure at the PNG resolution (Agg glyph advances depend on dpi)
    r = fig.canvas.get_renderer()
    for txt, st in parts:
        wt = st.get("weight", "normal")
        t = ax.text(x, y, txt.replace(" ", "\u00a0"), fontsize=fs,
                    color=st.get("color", INK), weight=wt, va="baseline")
        bb = t.get_window_extent(renderer=r)
        x1 = x + bb.width / (fig.dpi * fig_w)
        if st.get("strike"):
            ax.plot([x, x1], [y + 0.010, y + 0.010], color=st.get("color", INK), lw=1.8)
        if st.get("under"):
            ax.plot([x, x1], [y - 0.014, y - 0.014], color=st.get("color", INK), lw=1.8)
        x = x1
    return x


def design_e(path):
    fig = plt.figure(figsize=(W, 3.8), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    fig.canvas.draw()
    ax.text(0.025, 0.95, "The abstract sentence, and what the paper's own results support",
            fontsize=11, weight="bold", va="top")
    ax.text(0.025, 0.80, "As published (Abstract)", fontsize=10, color=INK2)
    runs(ax, 0.025, 0.72, [("“…SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
                           ("astrocytes", {"strike": True, "weight": "bold"}),
                           (".”", {})])
    ax.text(0.025, 0.58, "Consistent with Results, Introduction, Discussion and S8 Table",
            fontsize=10, color=INK2)
    runs(ax, 0.025, 0.50, [("SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
                           ("SPG", {"under": True, "weight": "bold"}),
                           (", not in astrocytes.", {"under": True, "weight": "bold"})])
    ax.plot([0.025, 0.975], [0.40, 0.40], color=GRID, lw=1)
    ax.text(0.025, 0.375, "S8 Table. Orange: reported MRN per group (control → TDP-43), log2FC and p.\n"
            "Blue: recomputed raw log2 ratio of the two group values.", fontsize=10, color=INK2,
            va="top")
    cols = [0.025, 0.215, 0.405, 0.575, 0.80]
    for x, k in zip(cols, ORDER):
        nc, c, nt, t, r, p = S8[k]
        ax.text(x, 0.245, k + (" (astrocytes)" if k == "ALG" else ""), fontsize=10,
                weight="bold", va="top")
        ax.text(x, 0.18, f"{c} → {t}", fontsize=10, color=REP, va="top")
        ax.text(x, 0.115, (f"{r:.2f}, p{p}" if r is not None else "n/a"), fontsize=10,
                color=REP, va="top")
        ax.text(x, 0.05, f"raw {RAW[k]:+.2f}", fontsize=10, color=REC, va="top")
    save(fig, path)


design_a(os.path.join(CAND, "design_A_two_panels"))
design_b(os.path.join(CAND, "design_B_section_matrix"))
design_c(os.path.join(CAND, "design_C_log2fc_dots"))
design_d(os.path.join(CAND, "design_D_expression_dumbbell"))
design_e(os.path.join(CAND, "design_E_sentence_revision"))
# The write-up uses design A as fig1 and design C as fig2.
design_a(os.path.join(HERE, "fig1"))
design_c(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/design_A..E (.svg, .png)")

Table 2 odds ratios for haplogroup B4 and other zero-cell rows cannot come from the counts

PLoS Genet (2013) · doi:10.1371/journal.pgen.1003696 · PMC3744426

Summary

The paper reports that mitochondrial haplogroup B4 is less common in biliary atresia (BA) than in controls, with an odds ratio (OR) of 0.71 (95% CI 0.66–0.77), and an age- and sex-adjusted OR of 0.82 (0.71–0.85) in the Abstract. Its own Table 2 shows that none of the 71 BA patients and 23 of the 200 controls carry B4. With an empty cell the sample odds ratio is 0. The exact 95% CI is 0 to 0.45, and adding 0.5 to each cell gives 0.053 (0.003–0.881). The printed 0.71 (0.66–0.77) instead equals, to two decimals, a different statistic: the proportion of B4-negative participants who are controls divided by the proportion of B4-positive participants who are controls, 0.714 (0.660–0.772). Three other zero-cell rows of Table 2 (M8&Z, Y and D) show the same pattern. The direction of the B4 association and its p-value (Fisher exact p = 0.0009) stand, but the effect size and its precision are misreported for the paper's headline genetic finding, and the adjusted estimate cannot be finite for these counts.

What the paper reports

Results, "Association of BA with specific mitochondrial haplogroups", second paragraph:

Evaluation of haplogroup distributions revealed that the haplogroup B4 was less prevalent in patients with BA than in the control group (odds ratio [OR], 0.71; 95% confidence interval [CI], 0.66–0.77; p = 0.001).

The same paragraph, adjusted analysis:

Multivariate logistic-regression analysis of haplogroups associated with BA with adjustment for age and sex, showing the risk for haplogroup B4 was p = 0.007, OR 0.82, 95%CI 0.71–0.85

Abstract:

The haplogroup B4 and E prevalence were significantly lower and higher respectively, in the patients with BA than in the controls (odds ratios, 0.82 [p = 0.007] and 7.36 [p = 0.032] respectively) in multivariate logistic-regression analysis.

Table 2, "Association between biliary atresia (BA) and the mitochondrial haplogroups". The columns are BA positive | BA negative | control positive | control negative | OR | 95% CI | p:

B4 | 0 | 71 | 23 | 177 | 0.71 | 0.66–0.77 | 0.001*

M8&Z | 0 | 71 | 5 | 195 | 1.36 | 1.27–1.47 | 0.331

Y | 0 | 71 | 1 | 199 | 1.36 | 1.26–1.46 | 1.000

D | 2 | 69 | 0 | 200 | 3.90 | 3.18–4.78 | 0.068

Z | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.06 | 0.456

C | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.07 | 1.456

Table 1, "Descriptive data of studied population". The columns are BA | control (children) | control (adult):

haplogroup B4 | 0 | 5 | 18

Materials and Methods, "Subjects", and "Statistical analysis":

The genetic background was similar in both control groups and was put together for statistical calculation.

Frequencies of all mitochondrial haplogroups in the BA patients and controls were tested for independency using Pearson chi-square statistics and Fisher's exact test as appropriate.

The statistical analyses were performed using the Statistical Package for Social Science (SPSS, version 12) software package.

The conclusions that rest on the B4 result. Author Summary:

The prevalence of haplogroup B4 and E was significantly lower and higher respectively, in the patients with BA than in the controls.

Discussion, first paragraph:

We found that whereas the haplogroup B4 was significantly less likely to occur in the patients with BA, the haplogroup E was more likely to occur in the patients with BA than in the healthy controls.

Conclusion:

The use of the cybrid models helped to establish the association of haplogroups B4 and E with BA

What the paper's own numbers show

Method. I first checked the B4 counts against Table 1: 5 child and 18 adult controls carry B4, so 23 of the 200 pooled controls, and 0 of 71 BA patients do. I then recomputed every row of Table 2 from its four counts: the sample odds ratio with a Woolf (log-Wald) CI, the odds ratio with 0.5 added to each cell (Haldane–Anscombe; for a single binary exposure this equals the Firth-penalised logistic estimate, which the script confirms), the exact conditional odds ratio and CI for B4, Fisher's exact test and Pearson's chi-square test. For the rows that did not reproduce, I tried simple ratios of proportions (with Katz log CIs) to find a statistic that does match the printed values. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
B4 carriers, BA vs controls Table 1, Table 2 0/71 vs 23/200 (5 + 18) consistent: 23 = 5 + 18; 177 = 200 − 23
B4 odds ratio Table 2; Results 0.71 sample OR (0 × 177)/(71 × 23) = 0; exact conditional MLE 0
B4 95% CI Table 2; Results 0.66–0.77 exact conditional CI 0–0.45; with 0.5 added to each cell, OR 0.053 (0.003–0.881)
B4 CI width on the log scale Table 2 0.154 5.63 for the 0.5-corrected OR (36 times wider)
Statistic that does match 0.71 (0.66–0.77) Table 2 — P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) = 0.714 (0.660–0.772)
B4 p Table 2; Results 0.001 Fisher exact two-sided 0.0009 (matches); Pearson 0.0028; Yates 0.0062
B4 adjusted OR Abstract; Results 0.82 (0.71–0.85), p = 0.007 no finite maximum-likelihood estimate: all 23 B4 carriers are controls, so the B4 coefficient is quasi-completely separated with or without age and sex in the model. The printed CI is also off-centre on the log scale (geometric midpoint 0.777, not 0.82)
M8&Z Table 2 OR 1.36 (1.27–1.47) sample OR 0 (0/71 vs 5/200); 0.5-corrected 0.249 (0.014–4.552). Matching statistic: P(control | positive)/P(control | negative) = 1.364 (1.269–1.467)
Y Table 2 OR 1.36 (1.26–1.46) sample OR 0 (0/71 vs 1/200); 0.5-corrected 0.930 (0.037–23.092). Matching statistic: P(control | positive)/P(control | negative) = 1.357 (1.263–1.457)
D Table 2 OR 3.90 (3.18–4.78), p = 0.068 sample OR infinite (2/71 vs 0/200); 0.5-corrected 14.424 (0.684–304.150). Matching statistic: P(BA | positive)/P(BA | negative) = 3.899 (3.181–4.778). The printed CI excludes 1 while the printed p (Fisher 0.0679) does not reach 0.05
C p Table 2 1.456 Fisher 0.456; row Z, with identical counts, prints 0.456. A probability cannot exceed 1
The 14 rows without a zero cell Table 2 ORs and CIs all 14 ORs reproduce to two decimals and the CIs to within rounding, e.g. E 11.88 (1.30–108.16), Fisher p 0.0177 vs 0.018
E adjusted OR (side note) Abstract; Results 7.36 (1.22–94.6) CI geometric midpoint 10.74, not 7.36; cannot be checked further without individual data

The ratios of proportions that match the zero-cell rows are not oriented the same way: for B4 the B4-negative group is the numerator, whereas for M8&Z, Y and D the carrier group is the numerator. In the opposite orientation M8&Z would give 0.733 (0.682–0.788). For the two rows whose p-values differ from Fisher's test, F (0.324) and D5 (0.372), the printed values equal Pearson's chi-square p (0.3240 and 0.3723), which the Methods allow ("as appropriate"), so I do not count them as errors.

Figure

Haplogroup B4: the published odds ratio and adjusted odds ratio (orange) beside three estimates recomputed from the Table 2 counts (blue), on one log axis

Figure 1. Haplogroup B4, 0 of 71 BA patients vs 23 of 200 controls, on one logarithmic ratio axis with the null value 1 marked by a thin vertical line. Orange marks the values the paper prints: the "OR" 0.71 (0.66–0.77) of Table 2 and the Results, and the adjusted "OR" 0.82 (0.71–0.85) of the Abstract and Results. Blue marks values recomputed from the Table 2 counts. The sample odds ratio is 0, which a log axis cannot show, so its exact 95% CI is drawn as an arrow from 0.45 towards zero. The odds ratio with 0.5 added to each cell is 0.053, with a CI (0.003–0.881) spanning more than two orders of magnitude. The bottom row is the ratio of the proportion of controls among B4-negative participants to that among B4-positive participants, 0.714 (0.660–0.772), which reproduces the printed 0.71 (0.66–0.77). The printed intervals are narrow enough to sit almost inside their markers. All values come from Table 2 or from recompute.py.txt.

Zero-cell rows of Table 2: printed values (orange) vs 0.5-corrected odds ratios and matching ratios of proportions (blue)

Figure 2. The four rows of Table 2 with an empty cell (B4, M8&Z, Y and D), on one logarithmic axis with the null value 1 marked. For each row, the top orange marker is the printed "OR" with its 95% CI, the blue diamond is the odds ratio with 0.5 added to each cell with its Woolf CI, and the hollow blue square is the ratio of proportions that matches the printed value. In every row the printed value and interval coincide with the ratio of proportions, not with any odds ratio, and the printed intervals are one to two orders of magnitude narrower than the counts allow. For M8&Z and Y the printed value lies above 1 although no BA patient carries the haplogroup.

Why it matters

B4 is the paper's headline genetic result. It appears in the title, the Abstract, the Author Summary, the Discussion and the Conclusion, and it is the reason the in vitro work compares a B4 cybrid with an E cybrid. The printed crude OR of 0.71 describes a 29% reduction in odds with a tight interval (0.66–0.77). The counts show something different in both size and certainty: no BA patient carried B4, so the data are compatible with anything from no carriers at all to an odds ratio of about 0.45 (exact CI), or 0.003 to 0.88 with a 0.5 correction. A reader using 0.71 (0.66–0.77) in a meta-analysis or a power calculation would carry over an effect measure of the wrong kind with a standard error about 36 times too small on the log scale.

The adjusted estimate in the Abstract, OR 0.82 (0.71–0.85), p = 0.007, is affected more seriously. With no B4 carrier among the cases, the logistic-regression likelihood keeps increasing as the B4 odds ratio goes to 0 (the script prints the profile log-likelihood at 0.1, 0.01 and 0.001), and adding age and sex as covariates cannot change that. Standard software then reports either no estimate or a very large negative coefficient with a very large standard error, not a finite OR with a narrow interval. Table 1 also shows that the BA patients are aged 0.1–11 years and the adult controls 18–60 years, so age alone separates the cases from three-quarters of the controls; any age-adjusted model of these data is fragile for that reason too.

What survives: the direction of the B4 association and its crude significance (Fisher exact p = 0.0009, matching the printed 0.001). The E association (crude OR 11.88) recomputes correctly, as do the other 13 rows without a zero cell. The cybrid experiments do not depend on Table 2's effect sizes. The finding is therefore a misreported key effect estimate and an adjusted estimate that cannot exist as stated, not an overturned conclusion. The errors in the M8&Z, Y and D rows matter less on their own, but the M8&Z and Y values point the wrong way (above 1 with no exposed cases), and the D interval contradicts its own p-value.

Explanations considered

Status

Suggested correction

  1. Replace the B4 effect estimate in Table 2 and the Results with an estimate that suits a zero cell, for example the exact conditional OR (0; 95% CI 0–0.45) or a 0.5-corrected or Firth OR (0.053; 95% CI 0.003–0.88), and state the method. Alternatively, report the carrier proportions (0/71 vs 23/200) with Fisher's exact p = 0.0009.
  2. Withdraw or replace the adjusted B4 estimate in the Abstract and Results (OR 0.82, 0.71–0.85, p = 0.007). If an adjusted analysis is kept, use a method that handles separation (Firth-penalised or exact logistic regression) and report its estimate and interval. Recheck the adjusted E estimate (7.36, 1.22–94.6), whose interval is not centred on it on the log scale.
  3. Correct the OR and CI columns of Table 2 for rows M8&Z, Y and D in the same way, and correct the p-value for row C from 1.456 to 0.456.
  4. State in the Methods which test produced each p-value in Table 2 (Fisher's exact test or Pearson's chi-square).
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 effect estimates of PMC3744426
(PLoS Genet 2013, doi:10.1371/journal.pgen.1003696).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs (candidates/)
next to this file.

Inputs are copied from the article as archived in PMC (identical to the publisher PDF):
  Table 1: B4 carriers by group (BA 0; child controls 5; adult controls 18)
  Table 2: BA positive / BA negative / control positive / control negative, OR, 95% CI, p
  Results text and Abstract: adjusted (age, sex) ORs for B4 and E
"""
import os

import numpy as np
from scipy.stats import chi2_contingency, fisher_exact
from scipy.stats.contingency import odds_ratio
from scipy.special import expit

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964

# ------------------------------------------------------------------ inputs (Table 2)
# name: (BA+, BA-, Ctl+, Ctl-, reported OR, reported CI low, high, reported p)
T2 = {
    "M10": (2, 69, 4, 196, 1.42, 0.25, 7.93, 0.654),
    "M7": (11, 60, 32, 168, 0.96, 0.45, 2.03, 1.000),
    "M8": (5, 66, 10, 190, 1.44, 0.48, 4.37, 0.549),
    "E": (4, 67, 1, 199, 11.88, 1.31, 108.16, 0.018),
    "G": (1, 70, 10, 190, 0.27, 0.03, 2.16, 0.298),
    "M9": (4, 67, 4, 196, 2.93, 0.71, 12.02, 0.213),
    "M8&Z": (0, 71, 5, 195, 1.36, 1.27, 1.47, 0.331),
    "Z": (1, 70, 1, 199, 2.84, 0.18, 46.06, 0.456),
    "C": (1, 70, 1, 199, 2.84, 0.18, 46.07, 1.456),
    "B4": (0, 71, 23, 177, 0.71, 0.66, 0.77, 0.001),
    "B5": (2, 69, 14, 186, 0.39, 0.09, 1.74, 0.253),
    "F": (13, 58, 48, 152, 0.71, 0.36, 1.41, 0.324),
    "A": (4, 67, 13, 187, 0.86, 0.27, 2.73, 1.000),
    "Y": (0, 71, 1, 199, 1.36, 1.26, 1.46, 1.000),
    "D": (2, 69, 0, 200, 3.90, 3.18, 4.78, 0.068),
    "D4": (6, 65, 24, 176, 0.68, 0.27, 1.73, 0.512),
    "D5": (4, 67, 18, 182, 0.60, 0.20, 1.85, 0.372),
    "N9": (2, 69, 7, 193, 0.80, 0.16, 3.94, 1.000),
}
TABLE1_B4 = {"BA": 0, "control children": 5, "control adults": 18}
N_BA, N_CTL = 71, 200
ADJ = {"B4": (0.82, 0.71, 0.85, 0.007), "E": (7.36, 1.22, 94.6, 0.032)}  # Abstract / Results


# ------------------------------------------------------------------ helpers
def woolf(a, b, c, d):
    """Sample OR ad/bc with Woolf (log-Wald) CI; None if a cell is zero."""
    if 0 in (a, b, c, d):
        return None
    o = a * d / (b * c)
    s = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    return o, o * np.exp(-Z * s), o * np.exp(Z * s)


def haldane(a, b, c, d):
    return woolf(a + .5, b + .5, c + .5, d + .5)


def ratio_of_proportions(x1, n1, x2, n2):
    """(x1/n1)/(x2/n2) with Katz log CI."""
    r = (x1 / n1) / (x2 / n2)
    s = np.sqrt(1 / x1 - 1 / n1 + 1 / x2 - 1 / n2)
    return r, r * np.exp(-Z * s), r * np.exp(Z * s)


def firth_or(a, b, c, d, iters=200):
    """Firth-penalised logistic regression of BA on carrier status (individual rows)."""
    x = np.r_[np.ones(a), np.zeros(b), np.ones(c), np.zeros(d)]
    y = np.r_[np.ones(a + b), np.zeros(c + d)]
    X = np.c_[np.ones_like(x), x]
    beta = np.zeros(2)
    for _ in range(iters):
        p = expit(X @ beta)
        W = p * (1 - p)
        info = X.T @ (X * W[:, None])
        inv = np.linalg.inv(info)
        h = W * np.einsum("ij,jk,ik->i", X, inv, X)
        step = inv @ (X.T @ (y - p + h * (0.5 - p)))
        beta = beta + step
        if np.max(np.abs(step)) < 1e-12:
            break
    return float(np.exp(beta[1]))


def fmt(t, k=2):
    return f"{t[0]:.{k}f} ({t[1]:.{k}f}–{t[2]:.{k}f})"


# ------------------------------------------------------------------ 1. B4 consistency with Table 1
print("== 1. B4 counts")
print("Table 1 B4 carriers:", TABLE1_B4, "-> controls", 5 + 18, "of", N_CTL,
      "; BA", 0, "of", N_BA, "; B4-negative controls", N_CTL - 23)

# ------------------------------------------------------------------ 2. B4 odds ratio, recomputed
a, b, c, d = T2["B4"][:4]
print("\n== 2. B4 (BA 0/71 vs controls 23/200)")
print("sample OR (0*177)/(71*23) =", a * d / (b * c))
cond = odds_ratio([[a, b], [c, d]])
ci = cond.confidence_interval()
print(f"exact conditional MLE OR = {cond.statistic:.3f}, exact 95% CI {ci.low:.3f}–{ci.high:.3f}")
B4_HAL = haldane(a, b, c, d)
print("Haldane-Anscombe (+0.5) OR, Woolf CI =", fmt(B4_HAL, 3))
print(f"Firth-penalised logistic OR (no covariates) = {firth_or(a, b, c, d):.4f}  (equals the +0.5 OR)")
print(f"Fisher exact two-sided p = {fisher_exact([[a, b], [c, d]])[1]:.4f}   (reported 0.001)")
print(f"Pearson chi-square p (no correction) = {chi2_contingency([[a, b], [c, d]], correction=False)[1]:.4f}; "
      f"Yates p = {chi2_contingency([[a, b], [c, d]], correction=True)[1]:.4f}")
B4_RR = ratio_of_proportions(d, b + d, c, a + c)  # P(control | B4-neg) / P(control | B4-pos)
print("P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) =", fmt(B4_RR, 3),
      " <- reported 'OR' 0.71 (0.66–0.77)")
print("width of reported CI on log scale:", round(np.log(0.77 / 0.66), 3),
      "; width of +0.5 OR CI:", round(np.log(B4_HAL[2] / B4_HAL[1]), 3),
      "; ratio:", round(np.log(B4_HAL[2] / B4_HAL[1]) / np.log(0.77 / 0.66), 1))
print(f"+0.5 OR to 4 dp = {B4_HAL[0]:.4f}")

# ------------------------------------------------------------------ 3. adjusted (age, sex) estimates
print("\n== 3. adjusted estimates (Abstract, Results)")
for k, (o, lo, hi, p) in ADJ.items():
    print(f"{k}: reported OR {o} ({lo}–{hi}); geometric centre of CI = {np.sqrt(lo * hi):.3f}")
print("B4: every B4-positive participant is a control, so the B4 coefficient in any logistic model "
      "of BA is quasi-completely separated; the likelihood keeps rising as OR -> 0, whatever "
      "covariates are added. Profile of the log-likelihood (no covariates) for OR = 0.1, 0.01, 0.001:")
for orv in (0.1, 0.01, 0.001):
    # intercept for B4-negative fixed at its MLE log(71/177); B4-positive group: 23 controls, 0 cases
    p_pos = expit(np.log(71 / 177) + np.log(orv))
    ll = 71 * np.log(71 / 248) + 177 * np.log(177 / 248) + 23 * np.log(1 - p_pos)
    print(f"   OR={orv:<6} loglik={ll:.4f}")
print("Table 1 age ranges: BA 0.1–11 y; child controls 0.3–16 y; adult controls 18–60 y")
print("adult controls (all older than every BA patient):", 148, "of", N_CTL, "=", f"{148 / N_CTL:.0%}")

# ------------------------------------------------------------------ 4. every Table 2 row
print("\n== 4. all Table 2 rows: reported vs recomputed")
print(f"{'row':5s} {'reported OR (CI)':22s} {'Woolf OR (CI)':22s} {'+0.5 OR (CI)':24s} "
      f"{'rep p':>6s} {'Fisher':>7s} {'Pearson':>7s}")
REC = {}
for k, (a, b, c, d, ro, rl, rh, rp) in T2.items():
    w = woolf(a, b, c, d)
    h = haldane(a, b, c, d)
    pf = fisher_exact([[a, b], [c, d]])[1]
    pp = chi2_contingency([[a, b], [c, d]], correction=False)[1] if min(a, b, c, d) >= 0 else np.nan
    REC[k] = dict(woolf=w, hal=h, fisher=pf, pearson=pp)
    ws = fmt(w) if w else "0 or inf (zero cell)"
    print(f"{k:5s} {ro:.2f} ({rl:.2f}–{rh:.2f}){'':5s} {ws:22s} {fmt(h, 3):24s} "
          f"{rp:6.3f} {pf:7.4f} {pp:7.4f}")

print("\nzero-cell rows: the printed value equals a ratio of proportions")
ZERO = {
    "B4": ("P(control | negative) / P(control | positive)", ratio_of_proportions(177, 248, 23, 23)),
    "M8&Z": ("P(control | positive) / P(control | negative)", ratio_of_proportions(5, 5, 195, 266)),
    "Y": ("P(control | positive) / P(control | negative)", ratio_of_proportions(1, 1, 199, 270)),
    "D": ("P(BA | positive) / P(BA | negative)", ratio_of_proportions(2, 2, 69, 269)),
}
for k, (lab, r) in ZERO.items():
    ro, rl, rh = T2[k][4:7]
    print(f"{k:5s} reported {ro:.2f} ({rl:.2f}–{rh:.2f})   {lab} = {fmt(r, 3)}   "
          f"+0.5 OR = {fmt(REC[k]['hal'], 3)}")
print("M8&Z opposite orientation: P(control|neg)/P(control|pos) =",
      fmt(ratio_of_proportions(195, 266, 5, 5), 3))
print("D: reported CI 3.18–4.78 excludes 1, reported p = 0.068; Fisher p =", round(REC["D"]["fisher"], 4))
print("C and Z have identical counts (1/70 vs 1/199): reported p 1.456 and 0.456; Fisher p =",
      round(REC["C"]["fisher"], 4))
print("F and D5 reported p match Pearson chi-square without correction:",
      round(REC["F"]["pearson"], 4), round(REC["D5"]["pearson"], 4))
print("E: Woolf OR", fmt(REC["E"]["woolf"]), "Fisher p", round(REC["E"]["fisher"], 4))
print("F row OR (13*152)/(58*48) =", round(13 * 152 / (58 * 48), 3), "with CI", fmt(REC["F"]["woolf"]))
nonzero_ok = [k for k in T2 if REC[k]["woolf"] and abs(REC[k]["woolf"][0] - T2[k][4]) < 0.006]
print("rows without a zero cell whose OR reproduces to 2 dp:", len(nonzero_ok), "of",
      sum(1 for k in T2 if REC[k]["woolf"]))

# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
    "xtick.color": INK2, "ytick.color": INK, "axes.edgecolor": INK2, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def clean(ax):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)


# ---- candidate A: B4 dot-and-interval, all estimates on one log axis
def design_a(path):
    XMIN, XMAX = 0.001, 3
    rows = [
        ("Table 2, Results:\n“OR” 0.71 (0.66–0.77)", 0.71, 0.66, 0.77, "rep"),
        ("Abstract, Results (adjusted):\n“OR” 0.82 (0.71–0.85)", 0.82, 0.71, 0.85, "rep"),
        (f"Odds ratio (0×177)/(71×23) = 0,\nexact 95% CI 0–{ci.high:.2f}",
         None, None, ci.high, "rec0"),
        ("Odds ratio, 0.5 added to cells:\n" + fmt(B4_HAL, 3), *B4_HAL, "rec"),
        ("Share of controls, B4− / B4+:\n" + fmt(B4_RR, 3),
         *B4_RR, "rec"),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 3.9))
    ys = np.arange(len(rows))[::-1]
    for y, (lab, o, lo, hi, kind) in zip(ys, rows):
        col = ORANGE if kind == "rep" else BLUE
        if kind == "rec0":
            ax.annotate("", xy=(XMIN, y), xytext=(hi, y),
                        arrowprops=dict(arrowstyle="-|>", color=col, lw=2, mutation_scale=14))
            ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
            ax.text(XMIN * 1.15, y + 0.2, "estimate 0 (off the log axis)", color=INK2, fontsize=10,
                    va="bottom")
            continue
        ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
        for e in (lo, hi):
            ax.plot([e, e], [y - .14, y + .14], color=col, lw=2, zorder=3)
        mk = "o" if kind == "rep" else "D"
        ax.plot(o, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.text(0.93, ys[0] + 0.5, "no association", color=INK2, fontsize=10, va="center", ha="right")
    ax.set_xscale("log")
    ax.set_xlim(XMIN, XMAX)
    ax.set_ylim(-0.6, len(rows) - 0.3)
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_xticks([0.001, 0.01, 0.1, 0.5, 1, 2])
    ax.set_xticklabels(["0.001", "0.01", "0.1", "0.5", "1", "2"])
    ax.set_xlabel("Ratio, log scale (B4: 0/71 BA vs 23/200 controls)")
    clean(ax)
    ax.plot([], [], "o-", color=ORANGE, ms=8, label="reported in the paper")
    ax.plot([], [], "D-", color=BLUE, ms=8, label="recomputed from Table 2 counts")
    fig.legend(loc="upper left", bbox_to_anchor=(0.01, 0.99), ncol=2, frameon=False, fontsize=10)
    fig.subplots_adjust(left=0.36, right=0.97, top=0.88, bottom=0.14)
    save(fig, path)


# ---- candidate B: Cleveland dot plot over every Table 2 row
def design_b(path):
    order = sorted(T2, key=lambda k: T2[k][4])
    fig, ax = plt.subplots(figsize=(7.2, 5.6))
    for i, k in enumerate(order):
        ro = T2[k][4]
        w, h = REC[k]["woolf"], REC[k]["hal"]
        zero = w is None
        rec = h[0] if zero else w[0]
        ax.plot([min(ro, rec), max(ro, rec)], [i, i], color=GRID if not zero else INK2,
                lw=2, zorder=2)
        ax.plot(ro, i, "o", color=ORANGE, ms=11, mec=BG, mew=1.5, zorder=4)
        if zero:
            ax.plot(rec, i, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=5)
            ax.text(max(ro, rec) * 1.25, i, "zero cell", color=INK2, fontsize=10, va="center")
        else:
            ax.plot(rec, i, "D", color=BLUE, ms=5.5, zorder=6)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.set_xscale("log")
    ax.set_xlim(0.03, 40)
    ax.set_yticks(range(len(order)))
    ax.set_yticklabels(order)
    ax.set_ylim(-0.7, len(order) - 0.3)
    ax.set_xticks([0.05, 0.1, 0.5, 1, 2, 5, 10, 20])
    ax.set_xticklabels(["0.05", "0.1", "0.5", "1", "2", "5", "10", "20"])
    ax.set_xlabel("Odds ratio, BA vs controls (log scale; 1 = no association)")
    clean(ax)
    ax.plot([], [], "o", color=ORANGE, ms=11, label="“OR” printed in Table 2")
    ax.plot([], [], "D", color=BLUE, ms=5.5, label="OR recomputed from the row's counts (inside the orange dot when they agree)")
    ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2,
            label="zero-cell row: OR with 0.5 added to each cell")
    ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


# ---- shared box helpers for the diagram designs
def box(ax, x, y, w, h, text, edge, fs=10.5, ha="left", weight="normal", fill=BG):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.015",
                                fc=fill, ec=edge, lw=1.6))
    tx = x + 0.02 if ha == "left" else x + w / 2
    ax.text(tx, y + h / 2, text, ha=ha, va="center", fontsize=fs, color=INK, weight=weight,
            linespacing=1.45, family="DejaVu Sans")


def arrow(ax, p, q, col=INK2, text=None, toff=(0, 0.03)):
    ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=14, color=col, lw=1.6))
    if text:
        ax.text((p[0] + q[0]) / 2 + toff[0], (p[1] + q[1]) / 2 + toff[1], text, ha="center",
                va="bottom", fontsize=10, color=INK2)


def canvas(h):
    fig = plt.figure(figsize=(7.2, h))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    return fig, ax


# ---- candidate C: calculation-flow diagram for the B4 row
def design_c(path):
    fig, ax = canvas(4.4)
    ax.text(0.03, 0.95, "Haplogroup B4, Table 2 counts", fontsize=11, weight="bold", va="center")
    box(ax, 0.03, 0.60, 0.30, 0.28,
        "              B4+     B4−\nBA              0       71\ncontrols      23     177",
        INK2, fs=10.5)
    box(ax, 0.40, 0.66, 0.57, 0.22,
        f"Odds ratio = (0 × 177) / (71 × 23) = 0\nexact 95% CI 0–{ci.high:.2f}\n"
        f"with 0.5 added to each cell: {fmt(B4_HAL, 3)}", BLUE, fs=10.5)
    box(ax, 0.40, 0.36, 0.57, 0.22,
        "Proportion of controls, B4− / B4+\n"
        "= (177/248) / (23/23) = " + fmt(B4_RR, 3), BLUE)
    arrow(ax, (0.335, 0.77), (0.395, 0.77))
    arrow(ax, (0.335, 0.66), (0.395, 0.47))
    box(ax, 0.40, 0.05, 0.57, 0.20,
        "Printed in Table 2 and Results as the odds ratio:\n“OR” 0.71, 95% CI 0.66–0.77, p = 0.001",
        ORANGE)
    ax.add_patch(FancyArrowPatch((0.695, 0.355), (0.695, 0.255), arrowstyle="<|-|>",
                                 mutation_scale=14, color=INK2, lw=1.6))
    ax.text(0.71, 0.305, "same value to two decimals", fontsize=10, color=INK2, va="center")
    ax.text(0.03, 0.45, "blue = recomputed from\nthe paper's counts\n\norange = reported\nin the paper",
            fontsize=10, color=INK2, va="top")
    save(fig, path)


# ---- candidate D: text panels, Table 2 rows vs what the counts give
def design_d(path):
    fig, ax = canvas(4.6)
    ax.text(0.03, 0.955, "Table 2 (as published)", fontsize=11, weight="bold", va="center",
            color=INK)
    ax.text(0.46, 0.955, "Recomputed from the same counts", fontsize=11, weight="bold",
            va="center", color=INK)
    lines = [
        ("B4", "0/71 vs 23/200", "“OR” 0.71 (0.66–0.77)",
         "OR 0; ratio of proportions 0.714 (0.66–0.77)"),
        ("M8&Z", "0/71 vs 5/200", "“OR” 1.36 (1.27–1.47)",
         "OR 0; ratio of proportions 1.364 (1.27–1.47)"),
        ("Y", "0/71 vs 1/200", "“OR” 1.36 (1.26–1.46)",
         "OR 0; ratio of proportions 1.357 (1.26–1.46)"),
        ("D", "2/71 vs 0/200", "“OR” 3.90 (3.18–4.78)",
         "OR infinite; ratio of proportions 3.899 (3.18–4.78)"),
        ("C", "1/71 vs 1/200", "p = 1.456",
         "Fisher p = %.3f (row Z, same counts: 0.456)" % REC["C"]["fisher"]),
    ]
    top = 0.86
    hgt = 0.135
    for i, (k, cnt, rep, rec) in enumerate(lines):
        y = top - i * (hgt + 0.03) - hgt
        box(ax, 0.03, y, 0.33, hgt, f"{k}   {cnt}\n{rep}", ORANGE, fs=10.5)
        box(ax, 0.46, y, 0.51, hgt, rec.replace("; ", ";\n").replace(" (row Z", "\n(row Z"),
            BLUE, fs=10.5)
        arrow(ax, (0.365, y + hgt / 2), (0.455, y + hgt / 2))
    ax.text(0.03, 0.025, "Counts: carriers among 71 BA patients vs among 200 controls.",
            fontsize=10, color=INK2, ha="left")
    save(fig, path)


# ---- candidate E: intervals for every zero-cell row on one log axis
def design_e(path):
    rows = ["B4", "M8&Z", "Y", "D"]
    fig, ax = plt.subplots(figsize=(7.2, 4.6))
    yt, yl = [], []
    for i, k in enumerate(rows):
        base = (len(rows) - 1 - i) * 3.4
        ro, rl, rh = T2[k][4:7]
        h = REC[k]["hal"]
        r = ZERO[k][1]
        trio = [(base + 1.0, ro, rl, rh, ORANGE, "o", BLUE),
                (base + 0.0, h[0], h[1], h[2], BLUE, "D", BLUE),
                (base - 1.0, r[0], r[1], r[2], BLUE, "s", BLUE)]
        for y, o, lo, hi, col, mk, _ in trio:
            ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
            ax.plot(o, y, mk, color=col, ms=8, mec=BG, mew=1.5, zorder=4,
                    mfc=col if mk != "s" else BG)
            if mk == "s":
                ax.plot(o, y, mk, color=col, ms=8, mfc=BG, mew=2, zorder=4)
        a_, b_, c_, d_ = T2[k][:4]
        yt.append(base)
        yl.append(f"{k}\n{a_}/71 vs {c_}/200")
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.set_xscale("log")
    ax.set_xlim(0.002, 400)
    ax.set_yticks(yt)
    ax.set_yticklabels(yl, fontsize=10)
    ax.set_ylim(-1.8, (len(rows) - 1) * 3.4 + 1.8)
    ax.set_xticks([0.01, 0.1, 1, 10, 100])
    ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
    ax.set_xlabel("Ratio, BA vs controls (log scale; 1 = no association)")
    clean(ax)
    ax.plot([], [], "o-", color=ORANGE, ms=8, label="“OR” (95% CI) printed in Table 2")
    ax.plot([], [], "D-", color=BLUE, ms=8, label="OR with 0.5 added to each cell")
    ax.plot([], [], "s-", color=BLUE, ms=8, mfc=BG, mew=2,
            label="ratio of proportions matching the printed value")
    ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


design_a(os.path.join(CAND, "A_b4_intervals"))
design_b(os.path.join(CAND, "B_all_rows_dotplot"))
design_c(os.path.join(CAND, "C_b4_calculation_flow"))
design_d(os.path.join(CAND, "D_zero_cell_text_panels"))
design_e(os.path.join(CAND, "E_zero_cell_intervals"))
# provisional choice for the write-up: A as fig1, E as fig2
design_a(os.path.join(HERE, "fig1"))
design_e(os.path.join(HERE, "fig2"))
print("\nfigures written to", HERE)

Abstract gives the earlier cohort's 70% accuracy as this trial's result; recomputed value is 63%

PLoS Med (2019) · doi:10.1371/journal.pmed.1002813 · PMC6546206

Summary

The abstract of this multi-institutional validation trial (MULTI-IMPROD, 338 men analysed) reports that "the accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75)". The NPV of 71/75 belongs to this trial. The accuracy of 113/161, however, is the figure printed in Table 3 for the earlier single-centre IMPROD trial (161 men). From the paper's own Table 2 counts, accuracy in the 338-man trial is (142 + 71)/338 = 213/338 = 63.0% (Wilson 95% CI 57.8%–68.0%). Table 3 prints this as 63%, but with the count 217/338, which is 64.2%. Overall accuracy is part of the trial's stated primary outcome, so the abstract overstates a primary-outcome result by 7.2 percentage points, by a value outside its 95% CI. It also hides the fall in accuracy between the earlier trial and the validation. The paper's headline conclusion, a high NPV, does not depend on this figure and stands.

What the paper reports

Abstract, Methods and findings (the primary outcome and the cohort):

The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value—of IMPROD bpMRI for clinically significant PCa (SPCa), which was defined as a Gleason score ≥ 3 + 4 (Gleason grade group 2 or higher). In total, 338 (338/364, 93%) prospectively enrolled men completed the trial.

Abstract, Methods and findings, the next sentence:

The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively.

Methods, "Study end points":

The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, NPV, and positive predictive value—of IMPROD bpMRI for SPCa, which was defined as a Gleason score ≥ 3 + 4.

Table 3, title and rows (columns: Measure | MULTI-IMPROD trial | IMPROD trial):

Comparison between the current trial (MULTI-IMPROD trial) and the pre-validation cohort (IMPROD trial).

Patient cohort in total | 338 | 161

Sensitivity* | 97% (142/146) [93%–99%] | 95% (79/83) [88%–98%]

Specificity* | 37% (71/192) [31%–44%] | 44% (34/78) [33%–55%]

NPV* | 95% (71/75) [87%–98%] | 90% (34/38) [76%–96%]

PPV* | 54% (141/263) [48%–60%] | 64% (79/123) [55%–72%]

Accuracy* | 63% (217/338) | 70% (113/161)

Table 3, footnote:

*Sensitivity, specificity, NPV, PPV, and accuracy values are based on binary classification (IMPROD bpMRI Likert score 1–2 versus 3–5) for predicting prostate cancer with Gleason score ≥ 3 + 4.

Table 2, title (the table gives men per Likert score 1–5 by biopsy outcome; its cells are used below):

Prevalence of prostate cancer and clinically significant prostate cancer in different IMPROD bpMRI Likert score groups based on a cohort of 338 men.

Results, "Diagnostic accuracy", first paragraph (relevant to a secondary finding below):

SPCa was diagnosed in 5% (2/34), 5% (2/41), 12% (8/66), 39% (23/59), and 88% (121/138) of men with IMPROD bpMRI Likert scores of 1, 2, 3, 4, and 5, respectively (Table 2).

What the paper's own numbers show

Method. Table 2 gives, for each Likert score from 1 to 5, the number of men with no cancer, Gleason 3+3, Gleason 3+4 and Gleason > 3+4. I checked that each column sums to its printed total (338 in all). Using the Table 3 footnote's definitions (test positive = Likert 3–5; reference positive = SPCa, Gleason ≥ 3+4), I built the 2 × 2 table for the 338 men. From it I computed sensitivity, specificity, NPV, PPV and overall accuracy, (TP + TN)/N, each with a Wilson 95% CI, and compared them with Table 3 and the abstract. I did the same for the earlier IMPROD trial from its Table 3 counts. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
2 × 2 cells, this trial Table 2 (cell counts by Likert score) TP = 8 + 23 + 111 = 142, FN = 2 + 2 = 4, FP = 263 − 142 = 121, TN = 75 − 4 = 71; N = 338
Sensitivity, this trial Table 3 97% (142/146) [93%–99%] 142/146 = 97.3% [93.2–98.9]; matches
Specificity, this trial Table 3 37% (71/192) [31%–44%] 71/192 = 37.0% [30.5–44.0]; matches (lower bound differs by under one point)
NPV, this trial Table 3; Abstract 95% (71/75) [87%–98%] 71/75 = 94.7% [87.1–97.9]; matches
PPV, this trial Table 3 54% (141/263) [48%–60%] 142/263 = 54.0% [48.0–59.9]; percentage matches, numerator printed one too low (141/263 = 53.6%)
Overall accuracy, this trial Table 3 63% (217/338) (142 + 71)/338 = 213/338 = 63.0% [57.8–68.0]; percentage matches, but 217/338 = 64.2%
Overall accuracy, as given in the abstract Abstract 70% (113/161) 113/161 = 70.2%. Denominator 161 is not this trial's N (338); 70.2% lies outside the 57.8–68.0% CI of the recomputed value
Overall accuracy, earlier IMPROD trial Table 3, IMPROD column 70% (113/161) (79 + 34)/161 = 113/161 = 70.2% [62.7–76.7]; identical to the abstract figure
NPV, earlier IMPROD trial Table 3, IMPROD column 90% (34/38) 34/38 = 89.5%
Gap between abstract accuracy and this trial's accuracy derived — 70.2% − 63.0% = 7.2 percentage points
SPCa in Likert 5 (secondary finding) Results, "Diagnostic accuracy" 88% (121/138) Table 2: 37 + 74 = 111/138 = 80.4%. With 121 the per-score SPCa counts would total 156, not the 146 used throughout

The abstract's single sentence therefore combines two cohorts: its NPV (71/75) comes from the 338 men of this trial, and its accuracy (113/161) from the 161 men of the earlier trial. Accuracy for this trial, from its own cells, is 63.0%, which is the percentage Table 3 prints.

Figure

The abstract sentence draws its accuracy from the earlier trial's Table 3 column and its NPV from this trial's column

Figure 1. Where each number in the abstract sentence comes from. Top: the two consecutive abstract sentences, quoted verbatim; the accuracy figure (orange outline, as reported) and the NPV figure (blue outline) are boxed. Bottom: the two columns of Table 3, with each printed value and the value recomputed from that column's own counts (TP + TN over the cohort N for accuracy; TN over test negatives for NPV). The abstract's accuracy, 70% (113/161), is the earlier single-centre cohort's value; its NPV, 95% (71/75), is this trial's. The recomputed accuracy for this trial's 338 men is 213/338 = 63.0%. All numbers are from the paper's Abstract, Table 2 and Table 3, or from recompute.py.txt.

Overall accuracy: abstract value against the value recomputed for this trial, with 95% CIs

Figure 2. Overall accuracy for SPCa on one 0–100% axis. The hollow orange dot is the abstract's reported value (70%; 113/161 = 70.2%). The blue dot and line are the accuracy recomputed for this trial's 338 men from Table 2 (213/338 = 63.0%, Wilson 95% CI 57.8–68.0%, also shaded). The gray dot and line are the earlier IMPROD trial (113/161 = 70.2%, 62.7–76.7%), shown for context: the abstract value coincides with it and falls outside the CI for this trial.

Why it matters

Explanations considered

  1. The abstract deliberately quotes the earlier cohort. Does not account for it. The preceding sentence introduces "338 (338/364, 93%) prospectively enrolled men", and the same sentence pairs the accuracy with this trial's NPV (71/75) using "respectively". Nothing in the abstract tells the reader that 113/161 comes from the earlier IMPROD trial (NCT01864135).
  2. A different definition, threshold or subgroup. Does not account for it. The Table 3 footnote states that all five metrics use the same cut (Likert 1–2 versus 3–5) and the same reference (Gleason ≥ 3+4). No subgroup of the 338 men with 161 members is reported; 161 is the total of the earlier cohort in Table 3. The 2 × 2 cells rebuilt from Table 2 reproduce Table 3's sensitivity, specificity and NPV and their CIs, and give an accuracy of 213/338 = 63.0%.
  3. Alternative definitions of SPCa in the supplements. Does not account for it. S1 Table and S2 Table (two alternative SPCa definitions) give MULTI-IMPROD accuracies of 62% (210/338) and 48% (163/338). In both, accuracy equals TP + TN over 338, and neither is 70% or uses 161.
  4. Rounding. Does not account for it. 70% against 63% is 7.2 percentage points.
  5. A text-conversion error in the PMC copy. Ruled out. The publisher's HTML and PDF print the same abstract sentence and the same Table 3 values; the PMC and Europe PMC XML carry each value in its own correctly aligned table cell.
  6. Is Table 3's 217/338 the true count, with 63% the slip? No. 217 is inconsistent with the table's own sensitivity and specificity numerators (142 + 71 = 213) and with 63% (217/338 = 64.2%). The percentage is right and the numerator is off by 4. Likewise the PPV numerator 141 should be 142 (TP), with no effect on the printed 54%.
  7. Likert 5 count of 121. Table 2 gives 37 + 74 = 111 men with SPCa among 138 at Likert 5, and only 111 makes the per-score counts sum to the 146 SPCa cases used throughout. The figure 121 does appear elsewhere in the paper as the number of SPCa cases detected by targeted biopsy (121/338 and 121/263), but that is a different quantity with different denominators, so it does not reconcile 121/138.

Status

Suggested correction

  1. Abstract, Methods and findings: replace "The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively." with "The accuracy and NPV of IMPROD bpMRI for SPCa were 63% (213/338) and 95% (71/75) (95% CI 87%–98%), respectively." If a CI for accuracy is wanted, the Wilson 95% CI is 58%–68%. If the authors wish to compare with the earlier trial, say so explicitly (for example, "compared with 70% (113/161) in the pre-validation IMPROD trial").
  2. Table 3, MULTI-IMPROD column: change "Accuracy 63% (217/338)" to "63% (213/338)" and "PPV 54% (141/263)" to "54% (142/263)".
  3. Results, "Diagnostic accuracy", first paragraph: change "88% (121/138)" to "80% (111/138)" for Likert score 5, as in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy figures of PMC6546206
(PLoS Med 2019, doi:10.1371/journal.pmed.1002813, the MULTI-IMPROD trial).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
  fig1.svg/png   provenance diagram: which cohort each number in the abstract sentence comes from
  fig2.svg/png   accuracy: abstract value vs the value recomputed for the 338-man cohort, with 95% CIs
  candidates/cand1..cand5.svg/png   five alternative figure designs for review
                                    (cand1 = fig1, cand2 = fig2)

Inputs are copied from the paper as archived in PMC:
  Table 2: men per IMPROD bpMRI Likert score (1..5) by biopsy outcome, MULTI-IMPROD cohort (n = 338)
  Table 3: diagnostic metrics for MULTI-IMPROD (n = 338) and the earlier IMPROD trial (n = 161)
Test positive = Likert 3-5; reference positive = Gleason score >= 3+4 (SPCa), as the Table 3 footnote states.
"""
import os
from math import sqrt

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from scipy.stats import beta  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (from the paper)
# Table 2, columns = Likert 1, 2, 3, 4, 5
NO_CANCER = [31, 33, 42, 19, 6]
GG1 = [1, 6, 16, 17, 21]          # Gleason 3+3
GG2 = [2, 2, 4, 13, 37]           # Gleason 3+4
GG3P = [0, 0, 4, 10, 74]          # Gleason > 3+4
TOTAL = [34, 41, 66, 59, 138]

# Table 3, as printed: (numerator, denominator, printed %)
T3_MULTI = {"Sensitivity": (142, 146, 97), "Specificity": (71, 192, 37), "NPV": (71, 75, 95),
            "PPV": (141, 263, 54), "Accuracy": (217, 338, 63)}
T3_IMPROD = {"Sensitivity": (79, 83, 95), "Specificity": (34, 78, 44), "NPV": (34, 38, 90),
             "PPV": (79, 123, 64), "Accuracy": (113, 161, 70)}
T3_MULTI_CI = {"Sensitivity": (93, 99), "Specificity": (31, 44), "NPV": (87, 98), "PPV": (48, 60)}

# Abstract, Methods and findings
ABS_ACC = (113, 161, 70)
ABS_NPV = (71, 75, 95)
ABS_NPV_CI = (87, 98)
# Results, "Diagnostic accuracy", paragraph 1 (Likert 5)
TEXT_L5 = (121, 138, 88)
TB_SPCA = 121                     # Results, "Comparison of TB with SB": TB detected SPCa in 121/338

# ------------------------------------------------------------------ helpers
Z = 1.959964


def wilson(a, n):
    p = a / n
    c = (p + Z * Z / (2 * n)) / (1 + Z * Z / n)
    h = Z * sqrt(p * (1 - p) / n + Z * Z / (4 * n * n)) / (1 + Z * Z / n)
    return c - h, c + h


def exact(a, n):
    lo = beta.ppf(0.025, a, n - a + 1) if a > 0 else 0.0
    hi = beta.ppf(0.975, a + 1, n - a) if a < n else 1.0
    return lo, hi


def pct(a, n):
    return 100.0 * a / n


def line(label, a, n, ci=True):
    s = f"  {label:<40s} {a:>4d}/{n:<4d} = {pct(a, n):5.1f}%"
    if ci:
        w = wilson(a, n)
        e = exact(a, n)
        s += f"   Wilson [{100*w[0]:.1f}-{100*w[1]:.1f}]   Clopper-Pearson [{100*e[0]:.1f}-{100*e[1]:.1f}]"
    print(s)


# ------------------------------------------------------------------ 1. Table 2 consistency
print("1. Table 2 (MULTI-IMPROD, n = 338)")
for i in range(5):
    assert NO_CANCER[i] + GG1[i] + GG2[i] + GG3P[i] == TOTAL[i]
N = sum(TOTAL)
SPCA = [GG2[i] + GG3P[i] for i in range(5)]
print(f"  rows sum to column totals; N = {N}; SPCa per Likert 1..5 = {SPCA}, total {sum(SPCA)}")
for i in range(5):
    line(f"SPCa, Likert {i+1}", SPCA[i], TOTAL[i], ci=False)

# ------------------------------------------------------------------ 2. 2x2 table, MULTI-IMPROD
TP = sum(SPCA[2:])
FN = sum(SPCA[:2])
FP = sum(TOTAL[2:]) - TP
TN = sum(TOTAL[:2]) - FN
assert TP + FN + FP + TN == N == 338
print(f"\n2. MULTI-IMPROD 2x2 (Likert 3-5 positive, SPCa reference): TP={TP} FN={FN} FP={FP} TN={TN} N={N}")
line("Sensitivity TP/(TP+FN)", TP, TP + FN)
line("Specificity TN/(TN+FP)", TN, TN + FP)
line("NPV TN/(TN+FN)", TN, TN + FN)
line("PPV TP/(TP+FP)", TP, TP + FP)
line("Accuracy (TP+TN)/N", TP + TN, N)
ACC_M = (TP + TN, N)
print("  As printed in Table 3, MULTI-IMPROD column:")
for k, (a, n, p) in T3_MULTI.items():
    print(f"    {k:<12s} printed {p}% ({a}/{n}); {a}/{n} = {pct(a, n):.1f}%")
print(f"  Accuracy numerator printed 217, recomputed {TP+TN}: difference {217-(TP+TN)}")
print(f"  PPV numerator printed 141, recomputed {TP}: difference {TP-141}")

# ------------------------------------------------------------------ 3. IMPROD trial column
TP2, FN2 = 79, 83 - 79
TN2, FP2 = 34, 78 - 34
N2 = TP2 + FN2 + TN2 + FP2
assert N2 == 161 and TP2 + FP2 == 123 and TN2 + FN2 == 38
print(f"\n3. IMPROD trial (Table 3 right-hand column): TP={TP2} FN={FN2} FP={FP2} TN={TN2} N={N2}")
line("Accuracy (TP+TN)/N", TP2 + TN2, N2)
line("NPV", TN2, TN2 + FN2)
line("Specificity", TN2, TN2 + FP2)
line("PPV", TP2, TP2 + FP2)

# ------------------------------------------------------------------ 4. The abstract sentence
print("\n4. Abstract: 'accuracy and NPV ... were 70% (113/161) and 95% (71/75) (95% CI 87%-98%)'")
print(f"  113/161 = {pct(113,161):.1f}% = IMPROD trial accuracy ({TP2}+{TN2} = {TP2+TN2}; N = {N2})")
print(f"  71/75  = {pct(71,75):.1f}% = MULTI-IMPROD NPV (Wilson {100*wilson(71,75)[0]:.0f}-{100*wilson(71,75)[1]:.0f}%)")
lo, hi = wilson(*ACC_M)
print(f"  MULTI-IMPROD accuracy {ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%, Wilson 95% CI {100*lo:.1f}-{100*hi:.1f}%;"
      f" 70.2% inside CI? {lo <= 113/161 <= hi}")
print(f"  Overstatement: {pct(113,161) - pct(*ACC_M):.1f} percentage points")
lo2, hi2 = wilson(113, 161)
print(f"  IMPROD accuracy Wilson CI {100*lo2:.1f}-{100*hi2:.1f}%")

# ------------------------------------------------------------------ 5. Side finding (Likert 5)
print("\n5. Results text, Likert 5: printed 88% (121/138)")
print(f"  121/138 = {pct(121,138):.1f}%; Table 2 gives {GG2[4]}+{GG3P[4]} = {SPCA[4]}/138 = {pct(SPCA[4],138):.1f}%")
print(f"  With 121 the Likert-group SPCa counts would sum to {sum(SPCA[:4]) + 121}, not {sum(SPCA)}")
print(f"  121 equals the TB-detected SPCa count ({TB_SPCA}/338 = {pct(TB_SPCA,338):.1f}%; {TB_SPCA}/263 = {pct(TB_SPCA,263):.1f}%)")

# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRAY, GRID = "#eb6834", "#2a78d6", "#8a8984", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "lines.linewidth": 1.5, "svg.fonttype": "path",
})
W = 7.2


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def box(ax, x, y, w, h, edge=GRAY, lw=1.2, fill=BG):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                fc=fill, ec=edge, lw=lw, transform=ax.transAxes))


def arrow(ax, p0, p1, color, rad=0.0, lw=1.5):
    ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle="-|>", mutation_scale=13,
                                 color=color, lw=lw, connectionstyle=f"arc3,rad={rad}"))


def blank(h):
    fig = plt.figure(figsize=(W, h))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_axis_off()
    return fig, ax


m_acc = f"{ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%"


# ---------------------------------------------------------- candidate 1: provenance diagram (fig1)
def segments(fig, ax, x, y, parts, fs=10.5):
    """Place text pieces left to right; parts = [(text, edgecolor or None)]. Returns x-centres."""
    r = fig.canvas.get_renderer()
    inv = ax.transAxes.inverted()
    centres = []
    for txt, ec in parts:
        kw = dict(fontsize=fs, va="top", transform=ax.transAxes)
        if ec:
            kw.update(weight="bold", bbox=dict(boxstyle="round,pad=0.22", fc=BG, ec=ec, lw=2))
        t = ax.text(x, y, txt, **kw)
        bb = t.get_window_extent(r)
        (x0, _), (x1, _) = inv.transform([(bb.x0, bb.y0), (bb.x1, bb.y1)])
        centres.append((x0 + x1) / 2)
        x = x1 + (0.012 if ec else 0.0)
        if ec:
            x += 0.004
    return centres


def cand1():
    fig, ax = blank(4.7)
    box(ax, 0.02, 0.68, 0.96, 0.29, edge=INK2)
    ax.text(0.04, 0.945, "Abstract (Methods and findings)", fontsize=10.5, weight="bold", va="top")
    ax.text(0.04, 0.875, "\u201cIn total, 338 (338/364, 93%) prospectively enrolled men completed the trial.",
            fontsize=10.5, va="top")
    ax.text(0.04, 0.815, "The accuracy and NPV of IMPROD bpMRI for SPCa were", fontsize=10.5, va="top")
    c = segments(fig, ax, 0.04, 0.745, [("70% (113/161)", ORANGE), (" and  ", None), ("95% (71/75)", BLUE),
                                         (" (95% CI 87%\u201398%), respectively.\u201d", None)])
    acc_x, npv_x = c[0], c[2]
    panels = [
        (0.52, "Table 3, MULTI-IMPROD column", "this trial: 4 centres, n = 338",
         [("NPV", "95% (71/75)", f"recomputed {TN}/{TN+FN} = {pct(TN,TN+FN):.1f}%", True),
          ("Accuracy", "63% (217/338)", f"recomputed ({TP}+{TN})/{N} = {pct(*ACC_M):.1f}%", False)]),
        (0.02, "Table 3, IMPROD trial column", "earlier single-centre cohort, n = 161",
         [("NPV", "90% (34/38)", f"recomputed {TN2}/{TN2+FN2} = {pct(TN2,TN2+FN2):.1f}%", False),
          ("Accuracy", "70% (113/161)", f"recomputed ({TP2}+{TN2})/{N2} = {pct(TP2+TN2,N2):.1f}%", True)]),
    ]
    for x0, title, sub, rows in panels:
        box(ax, x0, 0.10, 0.46, 0.40, edge=INK2)
        ax.text(x0 + 0.02, 0.47, title, fontsize=10.5, weight="bold", va="top")
        ax.text(x0 + 0.02, 0.415, sub, fontsize=10, color=INK2, va="top")
        for j, (k, pr, rc, hit) in enumerate(rows):
            y = 0.335 - 0.115 * j
            ax.text(x0 + 0.02, y, f"{k}:  {pr}", fontsize=10.5, va="top", weight="bold" if hit else "normal")
            ax.text(x0 + 0.02, y - 0.05, rc, fontsize=10, va="top", color=INK2)
    arrow(ax, (acc_x, 0.69), (acc_x, 0.505), ORANGE)
    arrow(ax, (npv_x, 0.69), (0.70, 0.505), BLUE, rad=0.1)
    ax.text(acc_x + 0.02, 0.59, "accuracy: from the earlier\ntrial (n = 161)", fontsize=10, ha="left", va="center")
    ax.text(0.97, 0.59, "NPV: from this trial\n(n = 338)", fontsize=10, ha="right", va="center")
    ax.text(0.02, 0.045, f"Accuracy of this trial's 338 men, from its own cells: {m_acc}. "
            f"(Table 3 prints 63%, but 217/338 = {pct(217,338):.1f}%.)", fontsize=10, color=INK2, va="center")
    return fig


# ---------------------------------------------------------- candidate 2: dot-and-interval (fig2)
def cand2():
    from matplotlib.lines import Line2D
    rows = [  # label, a, n, kind
        ("Abstract, presented as\nthis trial: \u201c70% (113/161)\u201d", 113, 161, "rep"),
        ("This trial (n = 338),\nrecomputed (142 + 71)/338", ACC_M[0], ACC_M[1], "rec"),
        ("Earlier IMPROD trial\n(n = 161), Table 3", 113, 161, "ctx"),
    ]
    col = {"rep": ORANGE, "rec": BLUE, "ctx": GRAY}
    fig, ax = plt.subplots(figsize=(W, 2.9))
    ys = list(range(len(rows)))[::-1]
    lo_m, hi_m = wilson(*ACC_M)
    ax.axvspan(100 * lo_m, 100 * hi_m, color=BLUE, alpha=0.08, lw=0)
    for y, (lab, a, n, kind) in zip(ys, rows):
        p = pct(a, n)
        c = col[kind]
        if kind == "rep":
            ax.plot(p, y, "o", ms=10, mfc=BG, mec=c, mew=2.2)
            ax.text(p + 2.5, y, f"70% as printed (113/161 = {p:.1f}%)", va="center", fontsize=10)
        else:
            lo, hi = wilson(a, n)
            ax.plot([100 * lo, 100 * hi], [y, y], color=c, lw=2, solid_capstyle="round")
            ax.plot(p, y, "o", ms=9, color=c)
            ax.text(100 * hi + 1.5, y, f"{p:.1f}%  [{100*lo:.1f}\u2013{100*hi:.1f}]", va="center", fontsize=10)
    ax.text(100 * lo_m + 0.5, len(rows) - 0.45, "95% CI, this trial", fontsize=10, color=INK2, va="top", ha="right")
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
    ax.set_xlim(0, 100)
    ax.set_ylim(-0.55, len(rows) - 0.4)
    ax.set_xlabel("Overall accuracy for SPCa, %  (Likert 3\u20135 vs 1\u20132; lines: Wilson 95% CI)", fontsize=10)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=ORANGE, mew=2, label="reported in the abstract"),
         Line2D([], [], marker="o", ls="-", ms=9, color=BLUE, lw=2, label="recomputed, this trial"),
         Line2D([], [], marker="o", ls="-", ms=9, color=GRAY, lw=2, label="earlier cohort (context)")]
    ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.02, 1.0), frameon=False, fontsize=10, ncol=3,
              handletextpad=0.4, columnspacing=1.2)
    return fig


# ---------------------------------------------------------- candidate 3: all metrics, both cohorts
def cand3():
    metrics = [("Sensitivity", (TP, TP + FN), (TP2, TP2 + FN2)),
               ("Specificity", (TN, TN + FP), (TN2, TN2 + FP2)),
               ("NPV", (TN, TN + FN), (TN2, TN2 + FN2)),
               ("PPV", (TP, TP + FP), (TP2, TP2 + FP2)),
               ("Accuracy", ACC_M, (TP2 + TN2, N2))]
    fig, ax = plt.subplots(figsize=(W, 3.4))
    ys = list(range(len(metrics)))[::-1]
    for y, (k, m, i) in zip(ys, metrics):
        pm, pi = pct(*m), pct(*i)
        ax.plot([pm, pi], [y, y], color=GRID, lw=2, zorder=1)
        ax.plot(pi, y, "s", ms=8, color=GRAY, zorder=2)
        ax.plot(pm, y, "o", ms=9, color=BLUE, zorder=3)
    # abstract values
    ax.plot(pct(*ABS_NPV[:2]), ys[2], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
    ax.plot(pct(*ABS_ACC[:2]), ys[4], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
    ax.annotate("abstract NPV 95% (71/75)\n= this trial's value", (pct(71, 75), ys[2]), xytext=(72, ys[2] - 0.45),
                fontsize=10, color=INK, ha="right", va="center",
                arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
    ax.annotate("abstract accuracy 70% (113/161)\n= the earlier trial's value", (pct(113, 161), ys[4]),
                xytext=(84, ys[4] + 0.55), fontsize=10, color=INK, ha="left", va="center",
                arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
    ax.set_yticks(ys)
    ax.set_yticklabels([m[0] for m in metrics], color=INK)
    ax.set_xlim(0, 100)
    ax.set_ylim(-0.6, len(metrics) - 0.3)
    ax.set_xlabel("%  (Likert 3–5 vs 1–2, SPCa = Gleason ≥ 3+4)")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    from matplotlib.lines import Line2D
    h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="MULTI-IMPROD (this trial, n = 338), recomputed"),
         Line2D([], [], marker="s", ls="", ms=8, color=GRAY, label="IMPROD trial (earlier, n = 161), Table 3"),
         Line2D([], [], marker="o", ls="", ms=12, mfc="none", mec=ORANGE, mew=2, label="value in the abstract")]
    ax.legend(handles=h, loc="lower left", bbox_to_anchor=(0, 1.0), frameon=False, fontsize=10, ncol=2)
    return fig


# ---------------------------------------------------------- candidate 4: calculation flow
def cand4():
    fig, ax = blank(4.6)
    top = [
        (0.02, "1. Table 2 (n = 338)", ["SPCa, Likert 1\u20135:", "  " + ", ".join(map(str, SPCA)),
                                       "men, Likert 1\u20135:", "  " + ", ".join(map(str, TOTAL))], INK2),
        (0.355, "2. Cut at Likert \u2265 3", [f"TP {TP}    FN {FN}", f"FP {FP}    TN {TN}",
                                            "reproduces Table 3 sens,", "spec, NPV and their CIs"], INK2),
        (0.69, "3. Accuracy", [f"({TP} + {TN}) / {N}", f"= {TP+TN}/{N} = {pct(TP+TN,N):.1f}%",
                                          "Wilson 95% CI", f"{100*wilson(TP+TN,N)[0]:.1f}\u2013{100*wilson(TP+TN,N)[1]:.1f}%"], BLUE),
    ]
    for x0, t, body, c in top:
        box(ax, x0, 0.56, 0.29, 0.41, edge=c, lw=2.2 if c == BLUE else 1.2)
        ax.text(x0 + 0.02, 0.935, t, weight="bold", va="top", fontsize=10.5)
        for j, b in enumerate(body):
            ax.text(x0 + 0.02, 0.85 - 0.068 * j, b, va="top", fontsize=10,
                    color=INK if c == BLUE or j % 2 else INK2)
    arrow(ax, (0.312, 0.765), (0.353, 0.765), INK2)
    arrow(ax, (0.647, 0.765), (0.688, 0.765), INK2)
    bottom = [
        (0.355, "Table 3 prints", ["63% (217/338)", f"but 217/338 = {pct(217,338):.1f}%",
                                   f"count {217-(TP+TN)} too high"]),
        (0.69, "Abstract prints", ["70% (113/161)", f"= earlier trial's {TP2}+{TN2}",
                                    f"{pct(113,161)-pct(TP+TN,N):.1f} points higher"]),
    ]
    for x0, t, body in bottom:
        box(ax, x0, 0.13, 0.29, 0.33, edge=ORANGE, lw=2.2)
        ax.text(x0 + 0.02, 0.425, t, weight="bold", va="top", fontsize=10.5)
        for j, b in enumerate(body):
            ax.text(x0 + 0.02, 0.34 - 0.07 * j, b, va="top", fontsize=10, color=INK if j == 0 else INK2)
    arrow(ax, (0.80, 0.56), (0.55, 0.465), INK2, rad=-0.1)
    arrow(ax, (0.835, 0.56), (0.835, 0.465), INK2)
    ax.text(0.02, 0.40, "compare step 3\nwith the printed\nvalues (orange)", fontsize=10, color=INK2, va="top")
    ax.text(0.02, 0.04, "Blue: recomputed from the paper's own Table 2 counts.  Orange: values as printed.",
            fontsize=10, color=INK2, va="bottom")
    return fig


# ---------------------------------------------------------- candidate 5: the two 2x2 tables
def cand5():
    fig, axs = plt.subplots(1, 2, figsize=(W, 3.6))
    for ax, (title, tp, fn, fp, tn, col, note) in zip(axs, [
        ("MULTI-IMPROD (this trial), n = 338", TP, FN, FP, TN, BLUE, f"accuracy {TP+TN}/{TP+FN+FP+TN} = {pct(TP+TN,TP+FN+FP+TN):.1f}%"),
        ("IMPROD trial (earlier), n = 161", TP2, FN2, FP2, TN2, GRAY, f"accuracy {TP2+TN2}/{N2} = {pct(TP2+TN2,N2):.1f}%")]):
        ax.set_xlim(0, 2)
        ax.set_ylim(0, 2.75)
        ax.set_axis_off()
        cells = {(0, 1): ("TP", tp), (1, 1): ("FP", fp), (0, 0): ("FN", fn), (1, 0): ("TN", tn)}
        for (cx, cy), (k, v) in cells.items():
            diag = k in ("TP", "TN")
            ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
                                        fc=col if diag else BG, ec=col, lw=2, alpha=0.18 if diag else 1))
            ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
                                        fc="none", ec=col, lw=2))
            ax.text(cx + 0.5, cy + 0.6, k, ha="center", fontsize=10, color=INK2)
            ax.text(cx + 0.5, cy + 0.3, str(v), ha="center", fontsize=13, weight="bold")
        ax.text(0.5, 2.03, "SPCa", ha="center", fontsize=10, color=INK2)
        ax.text(1.5, 2.03, "no SPCa", ha="center", fontsize=10, color=INK2)
        ax.text(1.0, 2.35, title, ha="center", fontsize=10.5, weight="bold")
        ax.text(1.0, -0.12, note, ha="center", va="top", fontsize=10.5)
    axs[0].text(-0.05, 1.5, "Likert 3–5", ha="right", va="center", fontsize=10, color=INK2)
    axs[0].text(-0.05, 0.5, "Likert 1–2", ha="right", va="center", fontsize=10, color=INK2)
    fig.text(0.5, -0.12, "Abstract, for this trial: “accuracy … 70% (113/161)” — the right-hand "
             "table's figure, not the left-hand table's.", ha="center", fontsize=10.5, color=INK,
             bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=2))
    fig.subplots_adjust(wspace=0.25)
    return fig


save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_provenance"))
save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_interval"))
save(cand3(), os.path.join(CAND, "cand3_all_metrics"))
save(cand4(), os.path.join(CAND, "cand4_calculation_flow"))
save(cand5(), os.path.join(CAND, "cand5_two_by_two"))
print("\nWrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png) in", HERE)

The abstract's univariate diabetes odds ratio is inverted, and its p-value disagrees with Fig 5

PLoS Med (2019) · doi:10.1371/journal.pmed.1002961 · PMC6822721

Summary

The abstract and the Results say that type 2 diabetes is a risk factor for being a tuberculosis transmitter. They report the univariate result as "odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003". Fig 5 of the same paper gives the counts behind this comparison: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes. It prints OR 5.14 (0.90–36.45) and p = 0.03 for them. Recomputed from those counts, the OR for transmitters vs others is 5.24 (exact 95% CI 0.92–37.08) and the two-sided Fisher p is 0.032. The text's 0.19 (0.02–1.10) is that estimate with the groups swapped (0.191, 0.027–1.089). So the text gives an OR pointing the opposite way from its own word "enriched", and a p-value ten times smaller than the counts support. This is the only risk factor named in the abstract. At the univariate level, the counts support a borderline association (p = 0.032, exact CI just including 1), not one with p < 0.003.

What the paper reports

Abstract, "Methods and findings":

Our data suggest that type 2 diabetes mellitus is a risk factor associated with being a transmitter (odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003).

Results, section "Identification of transmitters allows association of risk factors to transmission", second paragraph:

Transmitters were significantly enriched in diabetic patients in both univariate (Fisher’s exact test; OR 0.19 [95% CI 0.02–1.10], p < 0.003) and multivariate (logistic regression; OR 23.77 [95% CI 2.53–339.69], p < 0.009) statistical analyses.

The same section, first paragraph, defines the comparison:

We combined likely transmitter cases together with the index cases predicted by TransPhylo (n = 23) and compared them to the other clustered cases (n = 61).

Methods, "Statistical analysis":

We selected the index cases and the samples with higher than 0.6 posterior probability of being transmitters as predicted by TransPhylo (23 transmitters compared to the remaining 84 clustered cases), with sensitivity analysis of the later threshold in S1 Text and S1 Table. Then, we computed the odds ratio (OR) and 95% confidence intervals (Fisher’s exact test) to explore epidemiological variables associated with being a transmitter.

Fig 5, Diabetes row. Fig 5 is an image, so this row was read from the publisher's figure file (journal.pmed.1002961.g005). Its columns are All Patients (n=84), Transmitter (n=23), Other samples (n=61), Odd ratio (95% IC), p-value*:

Diabetes | 8 (9%) | 5 (22%) | 3 (5%) | 5.14 (0.90-36.45) | 0.03

Fig 5 caption:

The figure shows estimated odd ratios for each risk factor tested. *Fisher’s exact test. Comparisons were made between transmitter cases and the rest of the clustered samples.

Discussion, fourth paragraph:

Still, our data suggest that certain risk and epidemiological factors are enriched among the transmitters, while others are depleted.

What the paper's own numbers show

Method. I took the diabetes counts from Fig 5 and rebuilt the 2×2 table: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes, 8 of 84 in all. For transmitters vs others, and again with the groups swapped, I computed the conditional maximum-likelihood OR with its central exact (Cornfield) 95% CI. These are the quantities R's fisher.test prints. I also computed the two-sided Fisher exact p. As a separate check that does not use the counts, I back-calculated a log-Wald p from the text's own interval, 0.02–1.10. I searched every 23-vs-61 table with more diabetes among transmitters for tables that would give Fisher p < 0.003. I did the same Wald back-calculation for the multivariate result, which I cannot recompute without individual data. Every number below is printed by recompute.py.txt (scipy).

Quantity Location Reported Recomputed from the paper's numbers
Diabetic transmitters Fig 5 5 (22%) of 23 5/23 = 21.7%
Diabetic other clustered cases Fig 5 3 (5%) of 61 3/61 = 4.9%
OR, transmitters vs others Fig 5 5.14 (0.90–36.45) conditional MLE 5.24 (exact 95% CI 0.92–37.08); sample OR 5.37. The median-unbiased estimate is 5.15, closer to the printed 5.14
OR as given in the text Abstract; Results 0.19 (0.02–1.10) Groups swapped (others vs transmitters): 0.191 (0.027–1.089). Equivalently 1/5.14 = 0.195, 1/36.45 = 0.027, 1/0.90 = 1.11
Direction implied by the text's OR Abstract; Results "risk factor", "significantly enriched" With others at 4.9%, an OR of 0.19 would mean 0.97% of transmitters had diabetes (0.22 of 23 people), not 21.7%
Fisher exact p Fig 5 0.03 two-sided 0.0324; one-sided 0.0324
Fisher exact p Abstract; Results p < 0.003 0.0324 from the Fig 5 counts, about ten times larger
p compatible with the text's own CI Abstract; Results p < 0.003 log-Wald p from 0.19 (0.02–1.10) ≈ 0.10. A central exact 95% CI that includes 1 means each one-sided exact p is at least 0.025, so the two-sided Fisher p is at least 0.025
Counts needed for Fisher p < 0.003 derived — With 3 of 61 diabetic others, at least 8 of 23 transmitters (Fig 5: 5). With 5 diabetic transmitters, 0 of 61 others (Fig 5: 3)
Multivariate logistic OR Results 23.77 (2.53–339.69), p < 0.009 Not recomputable. Wald p from the CI: 0.011 using 23.77, or 0.007 using the CI's geometric midpoint (29.32). Consistent with the reported p and with OR > 1
Size of the comparison group Methods vs Results, Fig 5 "remaining 84" vs n = 61 23 + 61 = 84, so 84 is the total, not the comparison group

Figure

Reported and recomputed odds ratios for diabetes, transmitters vs other clustered cases

Odds ratios for diabetes with 95% confidence intervals on a log scale; the thin vertical line marks OR = 1 (no association). Orange: values printed in the paper (abstract and Results text, Fig 5, and the multivariate model). Blue: values recomputed from Fig 5's counts (5/23 transmitters vs 3/61 other clustered cases), once with transmitters as the index group and once with the groups swapped. The text's univariate interval coincides with the swapped-group recomputation. Fig 5's interval coincides with the transmitters-vs-others recomputation. The p-value printed next to the text's interval (p < 0.003) matches neither.

Reported and recomputed p-values for the univariate diabetes comparison

The same comparison on a p-value scale (log scale); the thin vertical line marks 0.05. Orange: the text's "p < 0.003" (the arrow shows that the bound is an upper bound) and Fig 5's p = 0.03. Blue: the two-sided Fisher p recomputed from Fig 5's counts (0.0324), and the log-Wald p implied by the text's own interval 0.02–1.10 (≈ 0.10). The shaded region (p < 0.025) is excluded for a Fisher test whose exact 95% CI includes 1. The text's value lies inside it.

Why it matters

The diabetes result is the only risk factor for transmission named in the abstract. It also serves as the example for the abstract's wider claim that "factors associated with likely transmitters are different to those of simply being in a transmission cluster". As printed, the univariate statistics cannot all be right together:

A reader who takes the abstract at face value gets a wrong direction and a strength of evidence about ten times greater than the data support. Recomputed from the paper's own counts, the univariate result is OR 5.24, two-sided Fisher p = 0.032, with an exact CI (0.92–37.08) that just includes 1. That is a borderline association resting on 5 diabetic transmitters, not a result with p < 0.003. The text describes the univariate analysis as showing transmitters "significantly enriched". That description holds only at p = 0.032, and the matching exact interval does not exclude 1.

What is not affected:

Explanations considered

Two further observations, which do not affect the claim:

Status

Suggested correction

In the Abstract and in the Results ("Identification of transmitters..."), the univariate result should be reported in the same direction as the sentence and as Fig 5. Using Fig 5's own values, the replacement would be "odds ratio 5.14 [95% CI 0.90–36.45], p = 0.03". From the Fig 5 counts, the conditional maximum-likelihood estimate is "OR 5.24 [95% CI 0.92–37.08], p = 0.032".

Because that interval includes 1, the Results' "significantly enriched ... in both univariate ... and multivariate" analyses could be reworded. For example: diabetes was more frequent among transmitters (5/23 vs 3/61; Fisher's exact p = 0.03, OR 5.14, 95% CI 0.90–36.45) and was significantly associated in the multivariate model.

In the Methods, "the remaining 84 clustered cases" should read "the remaining 61 clustered cases (84 in total)".

The authors hold the individual data and could confirm these values, and the Fig 5 HIV row, against it.

Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the diabetes risk-factor statistics in PMC6822721
(PLoS Med 2019, doi:10.1371/journal.pmed.1002961) from the paper's own Fig 5 counts,
and regenerate the figures in this folder.

Run:  python recompute.py.txt
Needs numpy, scipy, matplotlib.

Inputs (all from the version of record):
  Fig 5, Diabetes row: transmitters 5 (22%) of n = 23; other clustered samples 3 (5%) of n = 61;
      odds ratio 5.14 (0.90-36.45); Fisher's exact p = 0.03
  Abstract and Results text: OR 0.19 [95% CI 0.02-1.10], p < 0.003 (Fisher's exact test)
  Results text, multivariate logistic regression: OR 23.77 [95% CI 2.53-339.69], p < 0.009
"""
import os
import numpy as np
from scipy.stats import fisher_exact, nchypergeom_fisher as NCH, norm
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs
TX_DIAB, TX_N = 5, 23        # Fig 5, transmitters
OT_DIAB, OT_N = 3, 61        # Fig 5, other clustered samples
FIG5 = dict(or_=5.14, lo=0.90, hi=36.45, p=0.03)
TEXT = dict(or_=0.19, lo=0.02, hi=1.10, p=0.003)      # p printed as "p < 0.003"
MULTI = dict(or_=23.77, lo=2.53, hi=339.69, p=0.009)  # p printed as "p < 0.009"

# ---------------------------------------------------------------- helpers
def margins(a, b, c, d):
    return a + b + c + d, a + b, a + c   # total, row-1 total, column-1 total

def cond_mle(a, b, c, d):
    """Conditional maximum-likelihood OR (the estimate R's fisher.test prints)."""
    M, n, N = margins(a, b, c, d)
    return np.exp(brentq(lambda t: NCH.mean(M, n, N, np.exp(t)) - a, -20, 20))

def exact_ci(a, b, c, d, alpha=0.05):
    """Central exact (Cornfield) CI, as printed by R's fisher.test."""
    M, n, N = margins(a, b, c, d)
    lo = np.exp(brentq(lambda t: NCH.sf(a - 1, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
    hi = np.exp(brentq(lambda t: NCH.cdf(a, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
    return lo, hi

def median_unbiased(a, b, c, d):
    M, n, N = margins(a, b, c, d)
    f = lambda t: (NCH.cdf(a, M, n, N, np.exp(t)) - 0.5 * NCH.pmf(a, M, n, N, np.exp(t))) - 0.5
    return np.exp(brentq(f, -20, 20))

def wald_p_from_ci(est, lo, hi):
    se = (np.log(hi) - np.log(lo)) / (2 * norm.ppf(0.975))
    z = np.log(est) / se
    return se, z, 2 * norm.sf(abs(z))

def r(x, k=3):
    return float(np.round(x, k))

# ---------------------------------------------------------------- numbers
print("=== Fig 5 diabetes counts ===")
a, b = TX_DIAB, TX_N - TX_DIAB
c, d = OT_DIAB, OT_N - OT_DIAB
print(f"transmitters diabetic {a}/{TX_N} = {a/TX_N:.1%}; other clustered {c}/{OT_N} = {c/OT_N:.1%}; "
      f"all {a+c}/{TX_N+OT_N} = {(a+c)/(TX_N+OT_N):.1%}")
print(f"group sizes: {TX_N} + {OT_N} = {TX_N+OT_N} (Fig 5 'All Patients (n=84)')")

print("\n=== Transmitters vs others (direction stated in the text: 'enriched') ===")
sample_or = (a * d) / (b * c)
mle = cond_mle(a, b, c, d)
lo, hi = exact_ci(a, b, c, d)
mue = median_unbiased(a, b, c, d)
p2 = fisher_exact([[a, b], [c, d]])[1]
p_hi = fisher_exact([[a, b], [c, d]], alternative="greater")[1]
print(f"sample OR = {sample_or:.3f}")
print(f"conditional MLE OR = {mle:.3f}, exact 95% CI {lo:.3f}-{hi:.3f}")
print(f"median-unbiased OR = {mue:.3f}  (Fig 5 prints 5.14)")
print(f"Fisher exact p: two-sided = {p2:.4f}; one-sided (greater) = {p_hi:.4f}; doubled one-sided = {min(1,2*p_hi):.4f}")
print(f"exact CI includes 1: {lo < 1 < hi}")

print("\n=== Reversed coding: others vs transmitters ===")
mle_r = cond_mle(c, d, a, b)
lo_r, hi_r = exact_ci(c, d, a, b)
print(f"conditional MLE OR = {mle_r:.3f}, exact 95% CI {lo_r:.3f}-{hi_r:.3f}  (text: 0.19 [0.02-1.10])")
print(f"reciprocals of Fig 5: 1/{FIG5['or_']} = {1/FIG5['or_']:.3f}; 1/{FIG5['hi']} = {1/FIG5['hi']:.4f}; "
      f"1/{FIG5['lo']} = {1/FIG5['lo']:.3f}")
print(f"reciprocals of text: 1/{TEXT['or_']} = {1/TEXT['or_']:.2f}; 1/{TEXT['hi']} = {1/TEXT['hi']:.3f}; "
      f"1/{TEXT['lo']} = {1/TEXT['lo']:.0f}")

print("\n=== What the text's OR would imply ===")
odds_ot = c / d
odds_tx_implied = TEXT["or_"] * odds_ot
pct_implied = odds_tx_implied / (1 + odds_tx_implied)
print(f"with others at {c}/{OT_N} ({c/OT_N:.1%}), OR 0.19 implies {pct_implied:.2%} of transmitters diabetic "
      f"= {pct_implied*TX_N:.2f} of {TX_N} people; Fig 5 reports {a}/{TX_N} = {a/TX_N:.1%}")

print("\n=== p-values compatible with the text's CI ===")
se_t, z_t, p_t = wald_p_from_ci(TEXT["or_"], TEXT["lo"], TEXT["hi"])
print(f"log-Wald back-calculation from 0.19 (0.02-1.10): se = {se_t:.3f}, z = {z_t:.3f}, p = {p_t:.3f}")
print("an exact central 95% CI that includes 1 means each one-sided exact p >= 0.025,")
print("so Fisher's two-sided p (which contains a full tail) is >= 0.025; text p < 0.003 cannot hold")

print("\n=== Which 23 vs 61 tables would give Fisher p < 0.003 with more diabetes in transmitters? ===")
hits = []
for x in range(TX_N + 1):
    for y in range(OT_N + 1):
        if x / TX_N > y / OT_N:
            pp = fisher_exact([[x, TX_N - x], [y, OT_N - y]])[1]
            if pp < 0.003:
                hits.append((x, y, pp))
min_x_given_y3 = min(x for x, y, _ in hits if y == OT_DIAB)
print(f"with 3 diabetic others, transmitters would need at least {min_x_given_y3} of {TX_N} diabetic "
      f"(Fig 5: {TX_DIAB}); with {TX_DIAB} diabetic transmitters, others would need "
      f"{max(y for x, y, _ in hits if x == TX_DIAB)} of {OT_N}")

print("\n=== Multivariate result (reported only; not recomputable) ===")
se_m, z_m, p_m = wald_p_from_ci(MULTI["or_"], MULTI["lo"], MULTI["hi"])
gm = np.exp((np.log(MULTI["lo"]) + np.log(MULTI["hi"])) / 2)
_, _, p_m2 = wald_p_from_ci(gm, MULTI["lo"], MULTI["hi"])
print(f"Wald from CI with reported point 23.77: se = {se_m:.3f}, p = {p_m:.4f}; "
      f"with CI geometric midpoint {gm:.2f}: p = {p_m2:.4f} (reported p < 0.009)")

print("\n=== Non-clustered comparison sentence (observation only) ===")
for k in (5, 6):
    print(f"{k}/{TX_N} = {k/TX_N:.1%}", end="; ")
print("text says transmitters 27% diabetic in that comparison")

# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 2.0, 9

def save(fig, name, also=None):
    for ext in ("svg", "png"):
        kw = {"dpi": 200} if ext == "png" else {}
        fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
        if also:
            fig.savefig(os.path.join(HERE, f"{also}.{ext}"), bbox_inches="tight", **kw)
    plt.close(fig)

def fmt(x):
    return f"{x:.3g}" if x < 1 else (f"{x:.2f}" if x < 100 else f"{x:.0f}")

# ---------------------------------------------------------------- A: OR dot-and-interval
def cand_A():
    rows = [  # label, est, lo, hi, color, kind, p-label
        ("Abstract & Results text\n(univariate, Fisher)", TEXT["or_"], TEXT["lo"], TEXT["hi"], ORANGE, "Reported",
         "0.19 (0.02–1.10)   p < 0.003"),
        ("Recomputed, others vs transmitters\n(reversed coding)", mle_r, lo_r, hi_r, BLUE, "Recomputed",
         f"{mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f})   p = {p2:.3f}"),
        ("Fig 5, Diabetes row\n(univariate, Fisher)", FIG5["or_"], FIG5["lo"], FIG5["hi"], ORANGE, "Reported",
         "5.14 (0.90–36.45)   p = 0.03"),
        (f"Recomputed, transmitters vs others\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", mle, lo, hi, BLUE, "Recomputed",
         f"{mle:.2f} ({lo:.2f}–{hi:.2f})   p = {p2:.3f}"),
        ("Results text\n(multivariate logistic)", MULTI["or_"], MULTI["lo"], MULTI["hi"], ORANGE, "Reported",
         "23.77 (2.53–339.69)   p < 0.009"),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 4.3))
    ys = np.arange(len(rows))[::-1]
    for y, (lab, e, l, h, col, kind, pl) in zip(ys, rows):
        ax.plot([l, h], [y, y], color=col, lw=LW, solid_capstyle="butt")
        ax.plot([e], [y], "o", color=col, ms=MS, mec=BG, mew=1.5,
                label=kind if kind not in ax.get_legend_handles_labels()[1] else None)
        ax.text(h * 1.25, y, pl, va="center", fontsize=10, color=INK)
    ax.axvline(1, color=INK2, lw=0.8, zorder=0)
    ax.set_xscale("log")
    ax.set_xlim(0.01, 20000)
    ax.set_xticks([0.01, 0.1, 1, 10, 100, 1000])
    ax.set_xticklabels(["0.01", "0.1", "1", "10", "100", "1000"])
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-0.7, len(rows) - 0.3)
    ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Odds ratio for diabetes, 95% CI (log scale)")
    ax.text(0.85, len(rows) - 0.45, "← fewer diabetic transmitters", ha="right", fontsize=10, color=INK2)
    ax.text(1.18, len(rows) - 0.45, "more diabetic transmitters →", ha="left", fontsize=10, color=INK2)
    ax.legend(loc="lower left", frameon=False, fontsize=10, handletextpad=0.3)
    return fig

# ---------------------------------------------------------------- B: p-value scale
def cand_B():
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    ax.axvspan(1e-4, 0.025, color="#f1f0ec", zorder=0)
    ax.text(1.2e-4, -0.55, "Shaded: Fisher p below 0.025, impossible when the exact\n95% CI of the same test includes 1 (as 0.02–1.10 does)",
            fontsize=10, color=INK2, va="center")
    rows = [
        ("Abstract & Results text", TEXT["p"], ORANGE, "p < 0.003", True),
        ("Fig 5, Diabetes row", FIG5["p"], ORANGE, "p = 0.03", False),
        (f"Recomputed Fisher, two-sided\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", p2, BLUE, f"p = {p2:.4f}", False),
        ("Recomputed log-Wald p implied\nby the text's CI 0.02–1.10", p_t, BLUE, f"p ≈ {p_t:.2f}", False),
    ]
    ys = np.arange(len(rows))[::-1]
    seen = set()
    for y, (lab, p, col, pl, less) in zip(ys, rows):
        kind = "Reported" if col == ORANGE else "Recomputed"
        ax.plot([p], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, zorder=3,
                label=None if kind in seen else kind)
        seen.add(kind)
        if less:
            ax.annotate("", xy=(p / 3.5, y), xytext=(p / 1.15, y),
                        arrowprops=dict(arrowstyle="-|>", color=col, lw=LW))
        ax.text(p * 1.35, y + 0.02, pl, va="center", fontsize=10)
    ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
    ax.text(0.053, 3.45, "0.05", fontsize=10, color=INK2)
    ax.set_xscale("log")
    ax.set_xlim(1e-4, 1)
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-1.0, len(rows) - 0.3)
    ax.grid(axis="x", color=GRID, lw=0.6)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("p-value for diabetes, transmitters vs other clustered cases (log scale)")
    ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
    ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
    return fig

# ---------------------------------------------------------------- C: text vs figure panels
def box(ax, x, y, w, h, title, body, edge, fc="white"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc=fc, ec=edge, lw=1.5))
    ax.text(x + 0.02, y + h - 0.025, title, fontsize=10.5, weight="bold", va="top")
    ax.text(x + 0.02, y + h - 0.085, body, fontsize=10, va="top", color=INK, linespacing=1.4)

def cand_C():
    fig = plt.figure(figsize=(7.2, 5.0)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
    ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    box(ax, 0.0, 0.545, 0.45, 0.445, "Abstract & Results text (printed)",
        "\"type 2 diabetes mellitus is a risk\nfactor associated with being a\ntransmitter\"\n"
        "\"Transmitters were significantly\nenriched in diabetic patients\"\n\n"
        "OR 0.19 [95% CI 0.02–1.10]\np < 0.003", ORANGE)
    box(ax, 0.55, 0.545, 0.45, 0.445, "Fig 5, Diabetes row (printed)",
        "Transmitters (n = 23):  5 (22%)\nOther samples (n = 61):  3 (5%)\n\n"
        "Odds ratio 5.14 (0.90–36.45)\np = 0.03", ORANGE)
    ax.add_patch(FancyArrowPatch((0.45, 0.77), (0.55, 0.77), arrowstyle="<|-|>", mutation_scale=14,
                                 color=INK2, lw=LW))
    ax.text(0.5, 0.80, "conflict", ha="center", fontsize=10, color=INK2)
    box(ax, 0.0, 0.27, 1.0, 0.245, "Recomputed: the direction is reversed",
        f"0.19 (0.02–1.10) is the reciprocal of Fig 5: 1/5.14 = {1/FIG5['or_']:.3f},\n"
        f"1/36.45 = {1/FIG5['hi']:.3f}, 1/0.90 = {1/FIG5['lo']:.2f}. Others vs transmitters: {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f}).\n"
        "An OR below 1 means fewer diabetic transmitters, not \"enriched\".", BLUE, fc="#eef4fc")
    box(ax, 0.0, 0.0, 1.0, 0.245, "Recomputed: the text's p-value is ten times too small",
        f"Fisher exact two-sided p from the Fig 5 counts = {p2:.4f}\n"
        "(Fig 5 prints 0.03; the text prints p < 0.003). An exact 95% CI that\n"
        "includes 1, as 0.02–1.10 does, rules out a Fisher p below 0.025.", BLUE, fc="#eef4fc")
    return fig

# ---------------------------------------------------------------- D: what OR 0.19 would imply
def cand_D():
    fig, ax = plt.subplots(figsize=(7.2, 2.9))
    rows = [
        ("Other clustered cases\n(Fig 5)", c / OT_N * 100, ORANGE, f"{c}/{OT_N} = {c/OT_N:.1%}"),
        ("Transmitters\n(Fig 5)", a / TX_N * 100, ORANGE, f"{a}/{TX_N} = {a/TX_N:.1%}"),
        ("Transmitters, if the text's\nOR 0.19 were correct", pct_implied * 100, BLUE,
         f"{pct_implied:.1%} (≈ {pct_implied*TX_N:.1f} of {TX_N} people)"),
    ]
    ys = np.arange(len(rows))[::-1]
    seen = set()
    for y, (lab, v, col, txt) in zip(ys, rows):
        kind = "Reported" if col == ORANGE else "Recomputed (implied by OR 0.19 and the 4.9% in others)"
        ax.plot([0, v], [y, y], color=col, lw=LW, alpha=0.5)
        ax.plot([v], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, label=None if kind in seen else kind)
        seen.add(kind)
        ax.text(v + 0.7, y, txt, va="center", fontsize=10)
    ax.set_xlim(0, 30)
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-0.6, len(rows) - 0.4)
    ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Share with diabetes (%)")
    ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
    return fig

# ---------------------------------------------------------------- E: calculation flow
def cand_E():
    fig = plt.figure(figsize=(7.2, 4.8)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
    ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    def b(x, y, w, h, title, body, edge, fc="white"):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                    fc=fc, ec=edge, lw=1.5))
        ax.text(x + w / 2, y + h - 0.03, title, ha="center", va="top", fontsize=10.5, weight="bold")
        ax.text(x + w / 2, y + h - 0.095, body, ha="center", va="top", fontsize=10, linespacing=1.35)
    def arrow(p, q):
        ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, color=INK2, lw=1.6))
    b(0.29, 0.79, 0.42, 0.20, "Fig 5 counts (as printed)",
      f"Transmitters: {a} of {TX_N} diabetic\nOther clustered: {c} of {OT_N} diabetic", ORANGE)
    b(0.0, 0.43, 0.44, 0.22, "Transmitters vs others",
      f"recomputed OR {mle:.2f} ({lo:.2f}–{hi:.2f})\nFig 5 prints 5.14 (0.90–36.45)", BLUE, fc="#eef4fc")
    b(0.56, 0.43, 0.44, 0.22, "Others vs transmitters",
      f"recomputed OR {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f})\ntext prints 0.19 (0.02–1.10)", BLUE, fc="#eef4fc")
    b(0.29, 0.04, 0.42, 0.25, "Fisher exact, two-sided",
      f"recomputed p = {p2:.4f}\nFig 5 prints 0.03\ntext prints p < 0.003", BLUE, fc="#eef4fc")
    arrow((0.36, 0.79), (0.22, 0.65)); arrow((0.64, 0.79), (0.78, 0.65)); arrow((0.5, 0.79), (0.5, 0.29))
    ax.text(0.27, 0.73, "matches Fig 5", fontsize=10, color=INK2, ha="right", va="center")
    ax.text(0.73, 0.73, "matches the text's OR, but\nthe text says \"enriched\"",
            fontsize=10, color=INK2, ha="left", va="center")
    ax.text(0.515, 0.36, "matches Fig 5, not the text", fontsize=10, color=INK2, ha="left")
    ax.text(0.0, 0.0, "Orange outline: as printed.  Blue: recomputed in this script.", fontsize=10, color=INK2)
    return fig

save(cand_A(), "A_or_intervals", also="fig1")
save(cand_B(), "B_p_values", also="fig2")
save(cand_C(), "C_text_vs_fig5")
save(cand_D(), "D_implied_share")
save(cand_E(), "E_calculation_flow")
print("\nfigures written to", HERE, "and", CAND)

Abstract labels the 39.65 million prevented CVD events as deaths; Results report 13.86 million deaths

PLoS Med (2022) · doi:10.1371/journal.pmed.1004111 · PMC9725126

Summary

This microsimulation study compares hypertension screening scenarios in low- and middle-income countries. Its abstract states that almost 40 million (39,650,363) HTN-induced deaths over 10 years could be prevented by increasing screening coverage and completion (scenario S4). The Results give that same number, 39,650,363 (14.2%), as prevented CVD events, and give S4 prevented deaths as 13,856,507 (13.7%); the Author summary agrees ("almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"). The same abstract sentence also places 13,856,507 beside the 0.6% result for improved test accuracy (S2). By the paper's own percentages, 0.6% corresponds to roughly 0.61 million deaths. So the abstract's mortality headline is about 2.9 times the Results figure, and its absolute death count for the accuracy scenario is about 23 times what the Results imply. The version of record carries this text, and no correction notice was found (as of 2026-10-23).

What the paper reports

Abstract, "Methods and findings":

The study showed that while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years (13,856,507 [9,382,742; 17,395,833]), almost 40 million (39,650,363 [31,34,233, 49,298,921], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths could be prevented by increasing coverage and completion of a screening event in the same time frame.

Abstract, the next sentence:

Doubling the coverage only would still prevent 3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events 10 years after the rollout of the program.

Author summary, "What did the researchers do and find?":

While improving test accuracy avoids only 0.6% of HTN-induced deaths over 10 years, scaling up test coverage and completion can avoid almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths.

Results, paragraph 3 (CVD events, Fig 3, scenario S4):

In fact, it leads to a reduction of over 7 million (7,031,120; 26%) CVD events in year 10 compared to current standard of care (scenario S0). In sum, this equals 39,650,363 (14.2%) prevented CVD events over the period of 10 years.

Results, paragraph 4 (deaths, Fig 4):

Whereas survival does not improve much by increasing the test accuracy with 0.6% of HTN-induced deaths avoided over 10 years, we observe a high impact by doubling the coverage of the first screen (S3) with 6.6%. In year 10 alone, more than 2 million more lives (2,392,109) can be saved by additionally increasing completing probability in scenario S4, equalling to a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years.

Discussion, paragraph 1:

Whereas the improvement of the test accuracy had only a small potential impact on CVD events and related mortality over 10 years (reduction of 0.5% and 0.6%, respectively), by doubling of the coverage of a first screen, 6.6% of HTN-induced deaths could be avoided over a period of 10 years, and by increasing screen completion in addition, even 13.7% of deaths would not occur.

Discussion, paragraph 2:

Moreover, according to a review on available data on HTN in the literature [30], in 2015, an estimated 7.5 million deaths were attributable to systolic blood pressure in LMICs. This number coincides with the number of deaths in our model for the baseline (S0).

Methods, "Simulation study design", which treats events and deaths as separate endpoints:

We investigated the effect of blood pressure screening on the total number of CVD events and the resulting deaths within the population.

What the paper's own numbers show

Method. Each prevented count in the Results comes with its relative reduction against the base case S0. Dividing the count by that share gives the 10-year S0 total the paper is working from: S0 CVD events = 39,650,363 / 0.142 and S0 deaths = 13,856,507 / 0.137. These implied totals let us check each abstract number. For each one we ask which outcome and which percentage it fits. We also ask what S0 total the abstract's own label and percentage would require. The printed percentages are rounded to 0.1 points, so the implied totals are accurate to within about 1%. That is far smaller than any discrepancy below. All arithmetic is in recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
S4 prevented CVD events, 10 y Results para. 3; Author summary 39,650,363 (14.2%); "almost 40 million" events Implied S0 CVD events = 279,227,908 (27.9 M per year). Cross-check: year-10 figures imply 27.04 M (S4) and 27.31 M (S3) S0 events in year 10
S4 prevented deaths, 10 y Results para. 4; Author summary; Discussion 13,856,507 (13.7%); "14 million (13.7%)" deaths Implied S0 deaths = 101,142,387 (10.11 M per year); deaths/events in S0 = 0.362
S4 prevented deaths, 10 y Abstract 39,650,363, "12.7% [9.9, 15.8]" of HTN-induced deaths 2.86 times the Results figure; 25,793,856 more deaths than the Results report
S0 deaths implied by the abstract's S4 sentence Abstract (39,650,363 at 12.7%) 312,207,583 deaths (31.2 M per year). That is 1.118 times the implied S0 CVD events, so more deaths than events; and 4.16 times the 7.5 M per year the Discussion says the baseline matches (Results reading: 1.35 times)
S4 relative reduction Abstract vs Results 12.7% vs 14.2% (events) and 13.7% (deaths) 39,650,363 / 0.127 = 312.2 M matches neither implied denominator (279.2 M or 101.1 M); gaps of 1.5 and 1.0 points
S2 (improved accuracy) prevented deaths, 10 y Abstract 0.6% "(13,856,507 [9,382,742; 17,395,833])" 0.6% × 101,142,387 = 606,854 deaths. 13,856,507 / 101,142,387 = 0.137, the S4 share. Read as 0.6%, 13,856,507 would need 2.31 billion S0 deaths (22.8 times the implied baseline)
Lower bound of the S4 interval Abstract "31,34,233" 7 digits; a value near 31 million needs 8. In the publisher's JATS XML, "31" and "34" are tagged as citations of references 31 and 34
S3 prevented CVD events Abstract vs Results para. 3 "3,304,212 million" vs 3,304,212 Read literally, the abstract says 3.30 × 10^12; the Results value is 3,304,212

The abstract's mortality sentence takes the S4 CVD-event count and labels it as deaths. It then moves the S4 death count next to the S2 accuracy percentage. The Author summary, Results and Discussion all agree with one another; only the abstract departs from them.

Figure

Abstract excerpts beside the Results and Author summary excerpts that give the same two numbers different meanings

Figure 1. Left, in orange: the two clauses of the abstract's "Methods and findings" sentence, quoted verbatim. Right: the Results paragraphs (counted from the first Results paragraph as 1) and Author summary that report the same numbers. The upper connector links 39,650,363. The abstract calls it HTN-induced deaths (12.7%), while Results paragraph 3 calls it prevented CVD events (14.2%). Recomputed (blue), it is 2.86 times the S4 death count given in Results paragraph 4. The lower connector links 13,856,507. The abstract places it beside the 0.6% improved-accuracy result (S2), while Results paragraph 4 gives it as the S4 death total (13.7%). Recomputed from the paper's implied baseline of 101.1 million S0 deaths over 10 years, the S2 figure is 0.61 million. The Author summary (bottom right) gives about 40 million CVD events and 14 million deaths for S4, which agrees with the Results.

Deaths prevented over 10 years: abstract values versus Results and recomputed values

Figure 2. Deaths prevented over 10 years relative to S0, in millions, on a linear axis starting at zero. Orange: the counts the abstract attaches to each scenario's death result. For S4 (top), the open blue marker is the count in Results paragraph 4 (13.86 M); the abstract's 39.65 M is 2.9 times larger. For S2 (bottom), the filled blue marker is 0.6% of the S0 deaths implied by the Results (0.61 M); the abstract's 13.86 M is 22.8 times larger. The interval the abstract prints with 13,856,507 (9.38–17.40 M) excludes the S2 value implied by the Results.

Why it matters

The abstract's "Methods and findings" sentence is the paper's headline mortality estimate, and it is the figure most likely to be quoted. It says almost 40 million deaths could be prevented under S4. The paper's own Results put this at about 13.9 million, so the abstract overstates deaths averted by about 25.8 million. The same sentence gives the improved-accuracy scenario about 13.9 million averted deaths in absolute terms, when its 0.6% corresponds to about 0.6 million. A reader of the abstract alone would therefore overestimate the absolute mortality benefit of both interventions. The ratio between them would also be off: the abstract's numbers give about 2.9 to 1 (39.65 M vs 13.86 M), while the Results imply about 23 to 1 (13.86 M vs 0.61 M).

The direction and ranking of the scenarios do not change. Coverage plus completion still clearly outperforms improved accuracy, and the Author summary, Results and Discussion state the percentages consistently. The paper's qualitative conclusion, that screening completion and coverage matter more than test accuracy, is therefore unaffected. The error concerns the absolute numbers in the abstract and, for S4, its percentage. The abstract's 12.7% [9.9, 15.8] cannot be matched to any figure in the main text, so it is not clear what that interval refers to.

Explanations considered

Status

Suggested correction

In the abstract's "Methods and findings", align the mortality sentence with the Results. For example: "while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years, 13,856,507 [9,382,742; 17,395,833] (13.7%) of the HTN-induced deaths and almost 40 million CVD events (39,650,363 [lower bound; 49,298,921], i.e., 14.2%) could be prevented by increasing coverage and completion of a screening event in the same time frame." Here "lower bound" stands for the correct lower credible bound, which is not printed anywhere in the article. The authors should also confirm:

  1. The absolute number of deaths avoided under S2 (about 0.6 million by the paper's own percentages), with its credible interval.
  2. Which percentage and interval belong to the S4 CVD-event reduction (the Results give 14.2%; the abstract's 12.7% [9.9, 15.8] matches no reported figure).
  3. The S4 lower credible bound, garbled as "31,34,233".
  4. "3,304,212 million" corrected to "3,304,212" CVD events (S3, year 10).
Recomputation script (python recompute.py.txt)
"""Recompute the abstract-vs-Results figures of PMC9725126
(PLoS Med 2022, doi:10.1371/journal.pmed.1004111).

Run:  python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.

All inputs are copied from the article as archived in PMC (identical in the
publisher's JATS XML):
  Abstract (Methods and findings), Author summary, Results paragraphs 3-4 (Fig 3 events, Fig 4 deaths),
  Discussion paragraphs 1-2.
Scenario labels: S0 base case, S2 improved accuracy, S3 increased coverage,
S4 increased coverage + improved completion. All totals are over 10 years.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs
# Results paragraph 3 (CVD events, Fig 3)
S4_EVENTS_10Y = 39_650_363          # "39,650,363 (14.2%) prevented CVD events over the period of 10 years"
S4_EVENTS_PCT = 0.142
S4_EVENTS_Y10 = 7_031_120           # "reduction of over 7 million (7,031,120; 26%) CVD events in year 10"
S4_EVENTS_Y10_PCT = 0.26
S3_EVENTS = 3_304_212               # "3,304,212 prevented CVD events (12.1%) ... 10 years after a rollout"
S3_EVENTS_PCT = 0.121
# Results paragraph 4 (deaths, Fig 4)
S4_DEATHS_10Y = 13_856_507          # "a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years"
S4_DEATHS_PCT = 0.137
S2_DEATHS_PCT = 0.006               # "0.6% of HTN-induced deaths avoided over 10 years"
S3_DEATHS_PCT = 0.066               # "(S3) with 6.6%"
# Abstract
ABS_S4_DEATHS = 39_650_363          # "almost 40 million (39,650,363 [...], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths"
ABS_S4_PCT = 0.127
ABS_S4_LOWER_TEXT = "31,34,233"     # printed lower bound of that interval
ABS_S4_UPPER = 49_298_921
ABS_S2_DEATHS = 13_856_507          # "improving test accuracy avoids 0.6% ... (13,856,507 [9,382,742; 17,395,833])"
ABS_S2_CI = (9_382_742, 17_395_833)
ABS_S3_TEXT_MILLION = 3_304_212     # "3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events"
# Author summary: "almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"
SUMMARY_EVENTS_M, SUMMARY_DEATHS_M, SUMMARY_DEATHS_PCT = 40, 14, 0.137
# Discussion paragraph 2: 7.5 million deaths attributable to SBP in LMICs (2015),
# which "coincides with the number of deaths in our model for the baseline (S0)"
DISC_ANNUAL_DEATHS = 7.5e6

M = 1e6
print("=== 1. Denominators implied by the Results (baseline S0, 10 years) ===")
base_events = S4_EVENTS_10Y / S4_EVENTS_PCT
base_deaths = S4_DEATHS_10Y / S4_DEATHS_PCT
print(f"S0 CVD events = 39,650,363 / 0.142 = {base_events:,.0f}  ({base_events / 10 / M:.1f} M per year on average)")
print(f"S0 deaths     = 13,856,507 / 0.137 = {base_deaths:,.0f}  ({base_deaths / 10 / M:.2f} M per year on average)")
print(f"S0 deaths / S0 CVD events = {base_deaths / base_events:.3f}")
print(f"cross-check, year 10: S4 7,031,120 / 0.26 = {S4_EVENTS_Y10 / S4_EVENTS_Y10_PCT:,.0f} S0 events in year 10; "
      f"S3 3,304,212 / 0.121 = {S3_EVENTS / S3_EVENTS_PCT:,.0f}")
print(f"Author summary rounding: 39,650,363 -> {S4_EVENTS_10Y / M:.1f} M ('almost {SUMMARY_EVENTS_M} million' events); "
      f"13,856,507 -> {S4_DEATHS_10Y / M:.1f} M ('{SUMMARY_DEATHS_M} million', {SUMMARY_DEATHS_PCT:.1%} deaths)")

print("\n=== 2. Abstract S4 sentence: 39,650,363 labelled as deaths ===")
ratio = ABS_S4_DEATHS / S4_DEATHS_10Y
diff = ABS_S4_DEATHS - S4_DEATHS_10Y
print(f"abstract deaths / Results deaths = 39,650,363 / 13,856,507 = {ratio:.2f}")
print(f"difference = {diff:,} deaths")
implied_deaths_abs = ABS_S4_DEATHS / ABS_S4_PCT
print(f"if 39,650,363 were deaths at 12.7%: implied S0 deaths = {implied_deaths_abs:,.0f} "
      f"({implied_deaths_abs / 10 / M:.1f} M per year)")
print(f"  -> exceeds the implied S0 CVD events ({base_events:,.0f}) by a factor {implied_deaths_abs / base_events:.3f}")
print(f"  -> per-year value vs Discussion's 7.5 M: x{implied_deaths_abs / 10 / DISC_ANNUAL_DEATHS:.2f}; "
      f"Results reading: x{base_deaths / 10 / DISC_ANNUAL_DEATHS:.2f}")
print(f"abstract 12.7% vs Results 14.2% (events) and 13.7% (deaths): "
      f"gaps {100 * (S4_EVENTS_PCT - ABS_S4_PCT):.1f} and {100 * (S4_DEATHS_PCT - ABS_S4_PCT):.1f} points")
print(f"39,650,363 / 0.127 = {S4_EVENTS_10Y / ABS_S4_PCT:,.0f} matches neither {base_events:,.0f} nor {base_deaths:,.0f}")

print("\n=== 3. Abstract S2 sentence: 13,856,507 beside 0.6% ===")
s2_deaths = S2_DEATHS_PCT * base_deaths
print(f"0.6% x {base_deaths:,.0f} = {s2_deaths:,.0f} deaths (S2, implied)")
print(f"13,856,507 / {base_deaths:,.0f} = {S4_DEATHS_10Y / base_deaths:.3f} (the S4 share, 13.7%)")
print(f"13,856,507 as 0.6% would imply S0 deaths = {ABS_S2_DEATHS / S2_DEATHS_PCT:,.0f} "
      f"({ABS_S2_DEATHS / S2_DEATHS_PCT / base_deaths:.1f} x the Results-implied baseline)")
print(f"abstract S2 count / implied S2 count = {ABS_S2_DEATHS / s2_deaths:.1f}")
print(f"for reference, S3 deaths at 6.6% = {S3_DEATHS_PCT * base_deaths:,.0f}")
print(f"abstract interval printed with 13,856,507: {ABS_S2_CI[0]:,}-{ABS_S2_CI[1]:,}; "
      f"its lower end is {ABS_S2_CI[0] / s2_deaths:.1f} x the implied S2 point value")

print("\n=== 4. Typesetting / unit slips in the same abstract paragraph ===")
digits = len(ABS_S4_LOWER_TEXT.replace(",", ""))
print(f"lower bound '{ABS_S4_LOWER_TEXT}': {digits} digits (a value near 31 million needs 8); "
      f"upper bound {ABS_S4_UPPER:,}")
print(f"'3,304,212 million' read literally = {ABS_S3_TEXT_MILLION * M:,.0f}; Results value {S3_EVENTS:,}")

# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


# fig1: abstract vs Results/Author summary, two text panels with connectors
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")


def box(x, y, w, h, edge, title, body, title_col):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                facecolor="white", edgecolor=edge, linewidth=1.5))
    ax.text(x + 1.5, y + h - 1.8, title, fontsize=10, fontweight="bold", color=title_col, va="top")
    ax.text(x + 1.5, y + h - 7.0, body, fontsize=10, color=INK, va="top", linespacing=1.35)


ax.text(21, 96, "Abstract (Methods and findings)", ha="center", fontsize=11, fontweight="bold", color=ORANGE)
ax.text(79, 96, "Results and Author summary", ha="center", fontsize=11, fontweight="bold", color=INK)

LX, RX, W = 1.5, 59.5, 39
box(LX, 64, W, 27, ORANGE, "S4 sentence",
    "“almost 40 million (39,650,363\n[31,34,233, 49,298,921], i.e.,\n12.7% [9.9, 15.8]) of the\n"
    "HTN-induced deaths could be\nprevented”", ORANGE)
box(RX, 64, W, 27, INK2, "Results paragraph 3 (S4)",
    "“In sum, this equals 39,650,363\n(14.2%) prevented CVD events\nover the period of 10 years.”", INK)
box(LX, 30, W, 27, ORANGE, "S2 sentence",
    "“while improving test accuracy\navoids 0.6% of HTN-induced\ndeaths over 10 years\n"
    "(13,856,507 [9,382,742;\n17,395,833])”", ORANGE)
box(RX, 30, W, 27, INK2, "Results paragraph 4 (S4)",
    "“equalling to a total of\n13,856,507 (13.7%) of the\nHTN-induced deaths over the\n10 years.”", INK)
box(RX, 3, W, 20, INK2, "Author summary (S4)",
    "“almost 40 million HTN-induced\nCVD events and 14 million\n(13.7%) related deaths”", INK)


def connector(y, line1, line2, line3):
    ax.annotate("", xy=(RX - 0.6, y), xytext=(LX + W + 0.6, y),
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
    ax.text(50, y + 1.5, line1, ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
    ax.text(50, y - 1.5, line2, ha="center", va="top", fontsize=10, color=INK2)
    ax.text(50, y - 9.0, line3, ha="center", va="top", fontsize=10, color=BLUE)


connector(80, "same number", "deaths vs\nCVD events", f"recomputed:\n{ratio:.2f}× the\nS4 deaths")
connector(46, "same number", "S2 (0.6%) vs\nS4 (13.7%)", f"recomputed:\nS2 = {s2_deaths / M:.2f} M\ndeaths")
ax.text(21, 18, f"Abstract quotes in orange.\nBlue: recomputed from the\npaper's own Results numbers\n"
        f"(S0 deaths = 13,856,507 / 0.137\n= {base_deaths / M:.1f} M over 10 years).",
        ha="center", va="center", fontsize=10, color=INK2, linespacing=1.35)
save(fig, "fig1")

# fig2: deaths prevented over 10 years, abstract vs Results / recomputed
fig, ax = plt.subplots(figsize=(7.2, 3.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
rows = [("S2: improved test accuracy", ABS_S2_DEATHS, s2_deaths, "recomputed: 0.6% × S0 deaths"),
        ("S4: coverage + completion", ABS_S4_DEATHS, S4_DEATHS_10Y, "Results / Author summary")]
for i, (lab, rep, alt, altlab) in enumerate(rows):
    y = i
    ax.plot([alt / M, rep / M], [y, y], color=GRID, lw=2, zorder=1)
    ax.scatter(rep / M, y, s=90, color=ORANGE, zorder=3, label="Abstract" if i == 0 else None)
    if i == 0:
        ax.scatter(alt / M, y, s=90, color=BLUE, zorder=3, label="Recomputed: 0.6% of S0 deaths")
    else:
        ax.scatter(alt / M, y, s=90, facecolor="white", edgecolor=BLUE, linewidth=2, zorder=3,
                   label="Results paragraph 4")
    ax.text(rep / M, y + 0.2, f"{rep / M:.2f} M", ha="center", va="bottom", color=ORANGE, fontsize=10)
    ax.text(alt / M, y + 0.2, f"{alt / M:.2f} M", ha="center", va="bottom", color=BLUE, fontsize=10)
    ax.text((rep + alt) / 2 / M, y - 0.3, f"×{rep / alt:.1f}", ha="center", va="top", color=INK2, fontsize=10)
ax.set_yticks([0, 1])
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_ylim(-0.7, 1.55)
ax.set_xlim(0, 45)
ax.set_xlabel("HTN-induced deaths prevented over 10 years vs S0 (millions)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.88))
fig.legend(loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)

Abstract reverses the direction of the Alpha versus B.1.177 viral-burden difference reported in Table 1

PLoS Pathog (2023) · doi:10.1371/journal.ppat.1011461 · PMC10449197

Summary

The Abstract states that, among unvaccinated people with no known prior exposure, viral burden was "44% lower among Alpha variant infections" than among B.1.177 infections, and one Results sentence says the same. The paper's own coefficient points the other way. In Table 1, B.1.177 has a Beta score of +1.00 Ct against the Alpha reference, so B.1.177 samples had higher Ct values. Under the paper's stated rule (3 Ct = 10-fold change in viral load), Alpha burden is 10^(1/3) = 2.15 times that of B.1.177: 115% higher, or equivalently B.1.177 is 54% lower. The Results subsection heading, the sensitivity analysis and the Discussion ("115% higher among Alpha") all agree with this. The Abstract's first quantitative finding therefore has the wrong sign and the wrong size, and the publisher correction issued in October 2023 does not address it. A second sentence, in the sensitivity analysis, reverses the Delta versus Alpha direction among vaccinated people, and several smaller conversions in the text also disagree with Table 1.

What the paper reports

Abstract:

By analysing the adjusted Ct values using partial least squares regression, we found that among unvaccinated individuals with no known prior exposure, viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177.

Results, subsection headed "Among unvaccinated individuals with no known prior exposure, viral burden was higher among Alpha compared to B.1.177 infections":

For samples sequenced at Oxford, Ct values in this group were higher for B.1.177 samples (Beta score = 1.00, VIP = 1.05) compared to Alpha. This corresponds to a 44% reduction in viral loads with Alpha.

Table 1, block "Variant amongst unvaccinated individuals with no known prior exposure" (columns: Oxford N | Beta score (viral load factor change*) | VIP):

Ref = Alpha (Oxford), Delta (Northumbria) | 3526

B.1.177 | 658 | 1.00 (2.15) | 1.05

Table 1 caption footnote:

*based upon a Ct value decrease of 3 being equivalent to a 10-fold increase in viral load [34].

Results, sensitivity analysis (subsection "The association between novel variants and higher viral burden is robust to assumptions about the shape of the within-host viral trajectory"):

It is noteworthy that the VIP value remained greater than unity across plausible parameter combinations, providing support for the observation that viral burden is higher in infections with the Alpha-variant compared to B.1.177, among unvaccinated individuals with no known prior exposure.

The second paragraph of the same subsection, on Delta versus Alpha among vaccinated individuals:

This analysis therefore provides support for the finding that samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals. In summary, our sensitivity analysis supports the finding that both the shift from B.1.177 to Alpha and the shift from Alpha to Delta was associated with an increase in viral burden amongst the population (mostly unvaccinated and mostly vaccinated, respectively) that prevailed at each stage.

Discussion, third paragraph:

Among individuals with no known prior immunity, viral burden was, on average, 115% higher among Alpha-variant compared to B.1.177 samples.

The same paragraph and the Abstract also give the other conversions checked below:

We found that viral burden decreased over time and was higher among older individuals (13% and 17% higher for every 10 year age increase, in the samples sequenced in Oxford and Northumbria, respectively).

Vaccination reduced viral burden, with the average reduction being 75% and 30% amongst the samples sequenced at Oxford and Northumbria, respectively.

Abstract:

Vaccination reduced viral burden by 67%, and among vaccinated individuals, viral burden was 286% higher among Delta variant, compared to Alpha variant, infections.

Results, subsection "Viral burden is higher among older individuals":

Based upon a 3-point decrease in Ct value being equivalent to a 10-fold increase in viral load (see [34]), these age effects equate to an approximate 14% and 17% increase in viral load for every 10 year age increase, respectively.

What the paper's own numbers show

Method. A Beta score is the difference in (epidemiologically adjusted) Ct between a category and its reference. The paper converts it to a viral-load factor with the rule in the Table 1 footnote and the Results: a Ct decrease of 3 is a 10-fold increase, so factor (category / reference) = 10^(−Beta/3). I applied this rule to all 27 Beta scores in Table 1 and compared the result with the factor printed beside each one. I then converted every percentage in the text back to the Beta it implies. The PLS model itself cannot be re-run: the authors' public repository holds MATLAB code but no Ct data. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Ct(B.1.177) − Ct(Alpha), unvaccinated, Oxford Table 1; Results Beta = 1.00 ("Ct values ... were higher for B.1.177") B.1.177 has higher Ct, so lower viral burden
Viral load, Alpha / B.1.177 Table 1 factor 2.15 10^(1.00/3) = 2.154
Alpha relative to B.1.177 Discussion "115% higher among Alpha" +115.4% (matches)
Alpha relative to B.1.177 Abstract; Results sentence "44% lower among Alpha"; "44% reduction in viral loads with Alpha" +115.4%: opposite direction
B.1.177 relative to Alpha (the reverse reading) — −53.6%, not −44%
Beta implied by "44% lower among Alpha" derived — 3·log10(0.56) = −0.76, opposite sign to the printed +1.00
Table 1 factors that follow 10^(−Beta/3) Table 1, all rows 27 printed factors 26 of 27 match to two decimals; the exception is this B.1.177 row, where 0.46 is expected and 2.15 = 1/0.464 is printed

The other conversions in the text were checked the same way. These are smaller and do not change a conclusion:

Quantity Location Reported Recomputed
Vaccinated vs no known prior exposure, Oxford (Beta 1.44) Table 1; Abstract; Results; Discussion 0.33; "67%" reduction (Abstract, Results); "75%" (Discussion) factor 0.331, a 66.9% reduction; the Discussion's 75% does not match
Vaccinated vs no known prior exposure, Northumbria (Beta 0.34) Table 1; Discussion 0.77; "30%" reduction factor 0.770, a 23.0% reduction
Delta vs Alpha among vaccinated, Oxford (Beta −1.37) Table 1; Abstract; Results; Discussion 2.86; "286% higher" factor 2.862, which is 186% higher (286% of Alpha's level)
Age per 10 years, Oxford (Beta −0.016 per year) Table 1; Results; Discussion 1.13; "14%" (Results); "13%" (Discussion) factor 1.131, +13.1%
Age per 10 years, Northumbria (Beta −0.021 per year) Table 1; Abstract; Results; Discussion 1.17; "17%" factor 1.175, +17.5% (consistent)
Direction of Delta vs Alpha among vaccinated, Oxford (Beta −1.37) Results, sensitivity-analysis subsection, second paragraph "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" factor 2.862: Delta higher. Table 1, the Abstract ("286% higher among Delta"), the Results subsection on vaccinated individuals, the Discussion and the next sentence of the same paragraph ("the shift from Alpha to Delta was associated with an increase in viral burden") all say higher

Figure

Why it matters

The Alpha versus B.1.177 comparison is the first quantitative finding in the Abstract, which is the part of the paper most readers and indexing services see. As printed, it tells readers that the Alpha variant was associated with lower within-host viral burden than its predecessor among unvaccinated people. The paper's coefficient, its Table 1 factor, its section heading, its sensitivity analysis and its Discussion all report the opposite: Alpha was associated with about 2.15-fold (115%) higher burden. A reader who relies on the Abstract, or who cites the Results sentence, would carry forward the reversed direction and a magnitude (44%) that no conversion of the published coefficient under the paper's rule produces.

The paper's broader narrative, that each new variant brought higher viral burden in the population then prevailing (B.1.177 to Alpha among the mostly unvaccinated, Alpha to Delta among the mostly vaccinated), is supported by its own analysis and survives once the Abstract is corrected. The Abstract's closing summary ("is influenced by the interplay of vaccination status and viral variant") states no direction and is not affected. The secondary discrepancies matter less. The sensitivity-analysis sentence that Delta had "a lower viral burden" than Alpha among vaccinated people contradicts the summary sentence that immediately follows it and every other statement of that result, so a careful reader can resolve it, but a reader quoting that sentence alone would carry the reversed direction. In addition, "286% higher" for a 2.86-fold factor overstates the Delta effect by 100 percentage points in the Abstract, and the Discussion's vaccination reductions (75% and 30%) are larger than the Table 1 values (67% and 23%).

Explanations considered

Status

Suggested correction

  1. Abstract: replace "viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177" with a statement consistent with Table 1, for example "viral burden was 115% higher (2.15-fold) among Alpha variant infections, compared to those with the predecessor strain, B.1.177".
  2. Results: change "This corresponds to a 44% reduction in viral loads with Alpha" to "This corresponds to a 115% increase in viral loads with Alpha (equivalently, a 54% reduction with B.1.177)".
  3. Table 1: print the B.1.177 factor in the same orientation as the other rows (0.46, B.1.177 relative to Alpha), or footnote that 2.15 is Alpha relative to B.1.177.
  4. Abstract, Results and Discussion: describe the Delta versus Alpha effect as "186% higher" or "2.86-fold", not "286% higher".
  5. Results, sensitivity-analysis subsection, second paragraph: change "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" to "... had a higher viral burden ...".
  6. Discussion: change the vaccination reductions from 75% and 30% to 67% and 23%, and align the Oxford age effect (14% in the Results, 13% in the Discussion; 13.1% from Table 1).
Recomputation script (python recompute.py.txt)
"""Recompute the Alpha vs B.1.177 viral-burden statements of PMC10449197
(PLoS Pathog 2023, doi:10.1371/journal.ppat.1011461).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
five candidate figure designs (candidates/cand_A..E.svg/png) next to this file, for the user
to choose from. No design is embedded until one is chosen:

      python recompute.py.txt --pick A,E    # writes fig1 = cand_A, fig2 = cand_E and prints captions

The candidates/ folder is review material only and is not part of what goes to the journal.

Inputs are copied from the paper as archived in PMC (Table 1 and the text).
Conversion rule stated in the paper (Table 1 footnote; Results, "Viral burden is higher
among older individuals"): a Ct decrease of 3 is a 10-fold increase in viral load, so a
Beta score b (Ct of the category minus Ct of the reference) corresponds to a viral-load
factor (category / reference) of 10 ** (-b / 3).
"""
import math
import os
import sys
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2


def factor(beta):
    """Viral-load factor (category / reference) for a Ct difference beta."""
    return 10 ** (-beta / 3)


# ------------------------------------------------------------------ inputs (Table 1)
# (label, site, beta as printed, multiplier to the printed unit, printed factor)
TABLE1 = [
    ("Sample date (per year)", "Oxford", 0.008, 1, 0.99),
    ("Sample date (per year)", "Northumbria", 0.013, 1, 0.99),
    ("Age (per 10 years)", "Oxford", -0.016, 10, 1.13),
    ("Age (per 10 years)", "Northumbria", -0.021, 10, 1.17),
    ("Known prior exposure", "Oxford", 1.56, 1, 0.30),
    ("Known prior exposure", "Northumbria", -0.12, 1, 1.10),
    ("Vaccinated", "Oxford", 1.44, 1, 0.33),
    ("Vaccinated", "Northumbria", 0.34, 1, 0.77),
    ("B.1.177 vs Alpha, unvaccinated", "Oxford", 1.00, 1, 2.15),
    ("Delta vs Alpha, unvaccinated", "Oxford", -0.01, 1, 1.01),
    ("BA.1 vs Delta, unvaccinated", "Northumbria", -0.95, 1, 2.07),
    ("B.1.177 vs Alpha, vaccinated", "Oxford", 1.82, 1, 0.25),
    ("Delta vs Alpha, vaccinated", "Oxford", -1.37, 1, 2.86),
    ("BA.1 vs Delta, vaccinated", "Northumbria", -0.59, 1, 1.57),
    ("B.1.177 vs Alpha, prior exposure", "Oxford", 5.17, 1, 0.02),
    ("Delta vs Alpha, prior exposure", "Oxford", -3.05, 1, 10.39),
    ("BA.1 vs Delta, prior exposure", "Northumbria", -0.35, 1, 1.31),
    ("AstraZeneca vs Pfizer", "Oxford", -0.19, 1, 1.16),
    ("AstraZeneca vs Pfizer", "Northumbria", -0.10, 1, 1.08),
    (">=2 doses vs 1", "Oxford", -0.09, 1, 1.07),
    (">=2 doses vs 1", "Northumbria", 0.07, 1, 0.95),
    ("Other ethnicities vs White", "Oxford", 0.24, 1, 0.83),
    ("Other ethnicities vs White", "Northumbria", 0.46, 1, 0.70),
    ("Male vs female", "Oxford", -0.11, 1, 1.09),
    ("Male vs female", "Northumbria", 0.50, 1, 0.68),
    ("Health care worker", "Oxford", 0.44, 1, 0.71),
    ("Health care worker", "Northumbria", 0.28, 1, 0.81),
]
BETA_B177 = 1.00          # Table 1: B.1.177 vs Alpha (ref), unvaccinated, no known prior exposure, Oxford
ABSTRACT_PCT_LOWER = 44   # Abstract and Results: "44% lower among Alpha" / "44% reduction ... with Alpha"
DISCUSSION_PCT_HIGHER = 115

# ------------------------------------------------------------------ 1. main comparison
print("== 1. B.1.177 vs Alpha, unvaccinated, no known prior exposure (Oxford) ==")
b177_over_alpha = factor(BETA_B177)
alpha_over_b177 = 1 / b177_over_alpha
print(f"Beta (Ct B.1.177 - Ct Alpha)            = {BETA_B177:+.2f}")
print(f"B.1.177 / Alpha viral load = 10^(-1/3)   = {b177_over_alpha:.4f}")
print(f"Alpha / B.1.177 viral load = 10^(+1/3)   = {alpha_over_b177:.4f}  (Table 1 prints 2.15)")
print(f"Alpha relative to B.1.177                = {100 * (alpha_over_b177 - 1):+.1f}%  (Discussion: +115%)")
print(f"B.1.177 relative to Alpha                = {100 * (b177_over_alpha - 1):+.1f}%")
abs_ratio = 1 - ABSTRACT_PCT_LOWER / 100
# Alpha/B.1.177 = 0.56  ->  Ct(Alpha) - Ct(B.1.177) = -3*log10(0.56)  ->  beta = 3*log10(0.56)
beta_implied_abs = 3 * math.log10(abs_ratio)
print(f"Abstract: Alpha / B.1.177                = {abs_ratio:.2f}")
print(f"  implied Beta (Ct B.1.177 - Ct Alpha)   = {beta_implied_abs:+.3f}  (Table 1: +1.00)")
beta_implied_disc = -3 * math.log10(1 / (1 + DISCUSSION_PCT_HIGHER / 100))
print(f"Discussion: Alpha / B.1.177              = {1 + DISCUSSION_PCT_HIGHER / 100:.2f}; implied Beta = {beta_implied_disc:+.3f}")
beta_b177_44_lower = -3 * math.log10(abs_ratio)
print(f"Beta needed for 'B.1.177 44% lower than Alpha' = {beta_b177_44_lower:+.3f}")

print("\n-- Conversions that might give 44% from Beta = 1.00 --")
cands = [
    ("3 Ct per 10-fold, B.1.177 lower than Alpha (paper's rule)", 1 - 10 ** (-1 / 3)),
    ("3.3 Ct per 10-fold", 1 - 10 ** (-1 / 3.3)),
    ("1 Ct = 2-fold", 1 - 2 ** -1),
    ("4 Ct per 10-fold", 1 - 10 ** (-1 / 4)),
]
for lab, v in cands:
    print(f"  {lab:58s} -> {100 * v:.1f}% lower")
print("  (none of these makes Alpha the lower one; only the 4-Ct rule gives ~44%)")
print("  Other text percentages under a 4-Ct rule (check whether the paper used it):")
print(f"    Vaccinated Oxford, Beta 1.44   -> {100 * (1 - 10 ** (-1.44 / 4)):.1f}% reduction (text: 67%)")
print(f"    Delta vs Alpha vacc, Beta -1.37 -> factor {10 ** (1.37 / 4):.2f} (Table 1: 2.86)")
print(f"    Age Northumbria, -0.21 per 10y  -> {100 * (10 ** (0.21 / 4) - 1):.1f}% (text: 17%)")
# preprint (bioRxiv v1, 2022-12-02) abstract: Ct "0.94 lower among Alpha ... compared [to] B.1.177"
pre = 0.94
print(f"\nPreprint abstract: Ct 0.94 lower with Alpha -> Alpha/B.1.177 = {10 ** (pre / 3):.2f} "
      f"(+{100 * (10 ** (pre / 3) - 1):.0f}%); B.1.177 {100 * (1 - 10 ** (-pre / 3)):.1f}% lower")

# ------------------------------------------------------------------ 2. Table 1 consistency
print("\n== 2. Table 1: printed factor vs 10^(-Beta/3) for every entry ==")
rows = []
for lab, site, b, mult, printed in TABLE1:
    calc = factor(b * mult)
    recip = abs(printed - round(1 / calc, 2)) <= 0.011
    ok = abs(printed - round(calc, 2)) <= 0.011
    rows.append((lab, site, b * mult, calc, printed, ok))
    flag = "matches" if ok else ("RECIPROCAL" if recip else "MISMATCH")
    print(f"  {lab:34s} {site:11s} beta={b * mult:+.3f}  10^(-b/3)={calc:7.3f}  printed={printed:6.2f}  {flag}")
n_ok = sum(r[5] for r in rows)
print(f"  {n_ok} of {len(rows)} entries match 10^(-Beta/3) to 2 decimals; the exception is "
      f"B.1.177 vs Alpha, unvaccinated (expected {factor(BETA_B177):.2f}, printed 2.15 = 1/{factor(BETA_B177):.3f})")

# ------------------------------------------------------------------ 3. other text percentages
print("\n== 3. Other percentages in the text ==")
pcts = [
    ("Vaccination, Oxford (reduction)", 1.44, "Abstract/Results 67%; Discussion 75%", "red"),
    ("Vaccination, Northumbria (reduction)", 0.34, "Discussion 30%", "red"),
    ("Delta vs Alpha, vaccinated, Oxford", -1.37, "Abstract/Results/Discussion '286% higher'", "inc"),
    ("Age per 10 y, Oxford", -0.16, "Results 14%; Discussion 13%", "inc"),
    ("Age per 10 y, Northumbria", -0.21, "Abstract/Results/Discussion 17%", "inc"),
]
for lab, b, txt, kind in pcts:
    f = factor(b)
    if kind == "red":
        print(f"  {lab:38s} factor {f:.3f} -> {100 * (1 - f):.1f}% reduction   [{txt}]")
    else:
        print(f"  {lab:38s} factor {f:.3f} -> {100 * (f - 1):.1f}% higher      [{txt}]")

print("\n== 4. Direction of the Delta vs Alpha effect among vaccinated (Oxford) ==")
f_delta = factor(-1.37)
print(f"  Beta (Ct Delta - Ct Alpha) = -1.37 -> Delta / Alpha viral load = {f_delta:.3f} (Table 1: 2.86)")
print(f"  Delta is {'HIGHER' if f_delta > 1 else 'LOWER'} than Alpha by {100 * (f_delta - 1):.0f}%; "
      "the sensitivity-analysis sentence says Delta had a 'lower viral burden'")


# ================================================================== figures
def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def wrap(s, n):
    return "\n".join(textwrap.wrap(s, n))


def box(ax, x, y, w, h, edge, lw=1.4, fc="white"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc=fc, ec=edge, lw=lw, transform=ax.transAxes))


def fig_A():
    """Two text panels joined by an annotated connector."""
    fig = plt.figure(figsize=(W, 5.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.text(0.03, 0.965, "Alpha vs B.1.177: unvaccinated, no known prior exposure (Oxford)",
            fontsize=11.5, weight="bold", va="top")
    LQ, RQ = "\u201c", "\u201d"

    def panel(x, title, items, verdict):
        box(ax, x, 0.29, 0.41, 0.6, ORANGE)
        t = ax.text(x + 0.02, 0.865, title, fontsize=10, weight="bold", va="top")
        fig.canvas.draw()  # keep every panel title inside its box
        bb = t.get_window_extent().transformed(ax.transAxes.inverted())
        assert bb.x1 < x + 0.41 - 0.01, f"panel title overflows: {title}"
        y = 0.79
        for where, txt in items:
            ax.text(x + 0.02, y, where, fontsize=10, va="top", color=INK2)
            body = wrap(txt, 36)
            ax.text(x + 0.02, y - 0.045, body, fontsize=10, va="top", color=INK)
            y -= 0.045 + 0.042 * (body.count("\n") + 1) + 0.025
        ax.text(x + 0.02, 0.325, verdict, fontsize=10.5, weight="bold", color=ORANGE)

    panel(0.03, "Abstract and a Results sentence", [
        ("Abstract", LQ + "viral burden was 44% lower among Alpha variant infections, compared to those "
         "with the predecessor strain, B.1.177" + RQ),
        ("Results", LQ + "This corresponds to a 44% reduction in viral loads with Alpha." + RQ),
    ], "States: Alpha LOWER, by 44%")
    panel(0.56, "Table 1, heading, Discussion", [
        ("Table 1, B.1.177 row (ref = Alpha)", "Beta score 1.00 (2.15)"),
        ("Results subsection heading", LQ + "viral burden was higher among Alpha compared to B.1.177 "
         "infections" + RQ),
        ("Discussion", LQ + "viral burden was, on average, 115% higher among Alpha-variant compared to "
         "B.1.177 samples" + RQ),
    ], "States: Alpha HIGHER, by 115%")
    ax.add_patch(FancyArrowPatch((0.44, 0.55), (0.56, 0.55), arrowstyle="<|-|>", mutation_scale=14,
                                 color=INK, lw=2, transform=ax.transAxes))
    ax.text(0.5, 0.58, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold")
    # recomputation strip
    box(ax, 0.03, 0.03, 0.94, 0.21, BLUE, fc="#f3f7fd")
    ax.text(0.05, 0.215, "Recomputed with the paper's rule (3 Ct = 10-fold change):", fontsize=10.5,
            weight="bold", va="top", color=INK)
    ax.text(0.05, 0.15, f"Beta = +1.00 Ct  \u2192  Alpha / B.1.177 = 10^(1/3) = {alpha_over_b177:.2f}  \u2192  "
            f"Alpha {100 * (alpha_over_b177 - 1):.0f}% higher\n(equivalently B.1.177 "
            f"{100 * (1 - b177_over_alpha):.0f}% lower).  '44% lower with Alpha' would need Beta = "
            f"{beta_implied_abs:+.2f}.", fontsize=10, va="top", color=BLUE, linespacing=1.5)
    return fig


def fig_B():
    """One log axis: the Alpha/B.1.177 ratio implied by each statement."""
    fig, ax = plt.subplots(figsize=(W, 3.3))
    items = [
        ("Abstract: “44% lower among Alpha”", abs_ratio, ORANGE, "o"),
        ("Results: “44% reduction ... with Alpha”", abs_ratio, ORANGE, "o"),
        ("Discussion: “115% higher among Alpha”", 1 + DISCUSSION_PCT_HIGHER / 100, ORANGE, "o"),
        ("Table 1: Beta = +1.00 Ct, via 10^(Beta/3)", alpha_over_b177, BLUE, "D"),
    ]
    ys = list(range(len(items)))[::-1]
    for (lab, v, c, m), y in zip(items, ys):
        ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
        ax.text(v * (1.06 if v > 1 else 0.94), y, f"{v:.2f}", va="center",
                ha="left" if v > 1 else "right", fontsize=10, color=INK)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.set_xscale("log"); ax.set_xlim(0.35, 3.5)
    ax.minorticks_off()
    ax.set_xticks([0.4, 0.5, 0.7, 1, 1.5, 2, 3]); ax.set_xticklabels(["0.4", "0.5", "0.7", "1", "1.5", "2", "3"])
    ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items]); ax.set_ylim(-0.6, len(items) - 0.4)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0, labelcolor=INK)
    ax.set_xlabel("Viral load, Alpha / B.1.177 (log scale)")
    ax.text(0.97, len(items) - 0.45, "← Alpha lower", ha="right", fontsize=10, color=INK2, va="top")
    ax.text(1.03, len(items) - 0.45, "Alpha higher →", ha="left", fontsize=10, color=INK2, va="top")
    ax.plot([], [], "o", color=ORANGE, ms=9, label="stated in the paper")
    ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from Table 1 Beta")
    ax.legend(loc="lower center", bbox_to_anchor=(0.35, 1.0), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    return fig


def fig_C():
    """Calculation-flow diagram."""
    fig = plt.figure(figsize=(W, 3.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")

    def node(x, y, w, h, txt, edge, color=INK, bold=False, fc="white"):
        box(ax, x, y, w, h, edge, fc=fc)
        ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10, color=color,
                weight="bold" if bold else "normal")

    def arrow(p, q):
        ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.6,
                                     transform=ax.transAxes))
    node(0.02, 0.62, 0.24, 0.26, "Table 1\nB.1.177 vs Alpha (ref)\nBeta = +1.00 Ct", ORANGE)
    node(0.37, 0.62, 0.26, 0.26, "Paper's rule:\n3 Ct = 10-fold\nfactor = 10^(-Beta/3)", INK2)
    node(0.73, 0.62, 0.25, 0.26, f"Alpha / B.1.177\n= 10^(1/3) = {alpha_over_b177:.2f}", BLUE, color=BLUE, bold=True)
    arrow((0.26, 0.75), (0.37, 0.75)); arrow((0.63, 0.75), (0.73, 0.75))
    node(0.55, 0.12, 0.2, 0.3, f"Alpha\n{100 * (alpha_over_b177 - 1):.0f}% higher\n(Discussion: 115%,\nmatches)",
         BLUE, color=BLUE)
    node(0.78, 0.12, 0.2, 0.3, f"B.1.177\n{100 * (1 - b177_over_alpha):.0f}% lower\nthan Alpha", BLUE, color=BLUE)
    arrow((0.80, 0.62), (0.68, 0.42)); arrow((0.88, 0.62), (0.88, 0.42))
    node(0.02, 0.12, 0.42, 0.3, f"Abstract and Results: Alpha 44% lower\n= Alpha / B.1.177 of {abs_ratio:.2f}\n"
         f"would need Beta = {beta_implied_abs:+.2f} Ct\n(opposite sign to Table 1)", ORANGE, color=INK)
    ax.text(0.495, 0.27, "≠", fontsize=22, ha="center", va="center", color=INK)
    ax.text(0.02, 0.965, "From the Table 1 coefficient to a percentage", fontsize=11.5, weight="bold", va="top")
    return fig


def fig_D():
    """Ct-difference number line: the Beta each statement implies."""
    fig, ax = plt.subplots(figsize=(W, 2.9))
    items = [
        ("Abstract and Results: “44% lower among Alpha”", beta_implied_abs, BLUE, "D",
         "implied by the text"),
        ("Discussion: “115% higher among Alpha”", beta_implied_disc, BLUE, "D", "implied by the text"),
        ("Table 1: Beta score as printed", BETA_B177, ORANGE, "o", "printed"),
    ]
    ys = [2, 1, 0]
    for (lab, v, c, m, _), y in zip(items, ys):
        ax.plot([0, v], [y, y], color=c, lw=2, zorder=2)
        ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, zorder=3)
        ax.text(v + (0.06 if v > 0 else -0.06), y + 0.22, f"{v:+.2f}", ha="left" if v > 0 else "right",
                fontsize=10, color=INK)
    ax.axvline(0, color=INK2, lw=0.8)
    ax.set_xlim(-1.5, 1.5); ax.set_ylim(-0.6, 2.7)
    ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items])
    ax.tick_params(axis="y", length=0, labelcolor=INK)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.set_xlabel("Ct(B.1.177) − Ct(Alpha), Ct units")
    ax.text(-0.05, -0.5, "← Alpha lower burden", ha="right", fontsize=10, color=INK2)
    ax.text(0.05, -0.5, "Alpha higher burden →", ha="left", fontsize=10, color=INK2)
    ax.plot([], [], "o", color=ORANGE, ms=9, label="printed in Table 1")
    ax.plot([], [], "D", color=BLUE, ms=9, label="Beta implied by the text")
    ax.legend(loc="lower center", bbox_to_anchor=(0.3, 1.0), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    return fig


def fig_E():
    """Every Table 1 entry: printed factor vs 10^(-Beta/3)."""
    fig, ax = plt.subplots(figsize=(W, 4.6))
    lo, hi = 0.015, 15
    ax.plot([lo, hi], [lo, hi], color=BLUE, lw=2, zorder=1, label="printed = 10^(−Beta/3)  (paper's rule)")
    xs = [r[3] for r in rows]; ys = [r[4] for r in rows]
    okx = [x for x, r in zip(xs, rows) if r[5]]; oky = [y for y, r in zip(ys, rows) if r[5]]
    ax.plot(okx, oky, "o", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3,
            label=f"Table 1 entries ({n_ok} of {len(rows)} on the line)")
    bad = [r for r in rows if not r[5]]
    for r in bad:
        ax.plot([r[3]], [r[4]], "o", ms=11, mfc="none", mec=ORANGE, mew=2.2, zorder=4)
        ax.plot([r[3]], [r[3]], "D", ms=9, color=BLUE, mec=BG, mew=1.2, zorder=4)
        ax.annotate("", xy=(r[3], r[4] * 0.93), xytext=(r[3], r[3] * 1.08),
                    arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
        ax.text(r[3] * 0.8, r[4], f"B.1.177 vs Alpha, unvaccinated\nprinted {r[4]:.2f}", ha="right", va="center",
                fontsize=10)
        ax.text(r[3] * 1.2, r[3] * 0.9, f"expected {r[3]:.2f}", ha="left", va="top", fontsize=10, color=INK)
    ax.set_xscale("log"); ax.set_yscale("log")
    ax.set_xlim(lo, hi); ax.set_ylim(lo, hi)
    ticks = [0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10]
    ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.set_yticks(ticks); ax.set_yticklabels([f"{t:g}" for t in ticks])
    ax.minorticks_off()
    ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.set_xlabel("Viral-load factor recomputed from the printed Beta, 10^(−Beta/3)")
    ax.set_ylabel("Viral-load factor printed in Table 1")
    ax.legend(loc="upper left", frameon=False, fontsize=10)
    fig.tight_layout()
    return fig


CAPTIONS = {
    "A": "The two sets of statements the paper makes about the same comparison (B.1.177 vs Alpha, "
         "unvaccinated with no known prior exposure, Oxford samples), quoted verbatim. The orange-outlined "
         "panels hold the paper's text and Table 1 entry. On the left, the Abstract and one Results sentence "
         "say viral burden was 44% lower with Alpha. On the right, the Table 1 coefficient, the heading of the "
         "Results subsection that reports it, and the Discussion say Alpha was higher, by 115%. The blue strip "
         "at the bottom is the recomputation from the printed Beta score with the paper's own 3-Ct-per-10-fold "
         "rule; it agrees with the right-hand panel. A 44% reduction with Alpha would need a Beta of -0.76.",
    "B": "The Alpha / B.1.177 viral-load ratio implied by each statement in the paper (orange dots), and the "
         "ratio recomputed from the Table 1 Beta score of +1.00 with the paper's 3-Ct-per-10-fold rule (blue "
         "diamond), on one log axis. The vertical line marks no difference. The Abstract and the Results "
         "sentence imply 0.56 (Alpha lower); the Discussion and the recomputation give 2.15 (Alpha higher).",
    "C": "From the Table 1 coefficient to a percentage. The printed Beta of +1.00 Ct (orange), converted with "
         "the paper's rule, gives Alpha / B.1.177 = 2.15: Alpha 115% higher, or B.1.177 54% lower (blue). The "
         "Abstract and Results statement that Alpha was 44% lower would need a Beta of -0.76 Ct, the opposite "
         "sign.",
    "D": "The Ct difference, Ct(B.1.177) - Ct(Alpha), printed in Table 1 (orange) and the difference each text "
         "statement implies under the paper's 3-Ct-per-10-fold rule (blue). The Discussion implies +1.00, "
         "matching Table 1; the Abstract and Results sentence imply -0.76, the opposite sign.",
    "E": "Each orange dot is one of the 27 viral-load factors printed in Table 1, plotted against the factor "
         "recomputed from the Beta score printed beside it, 10^(-Beta/3) (both axes log scale; age is per 10 "
         "years, sample date per year). The blue line is where the two agree. Twenty-six entries lie on it. "
         "The exception (hollow orange circle) is B.1.177 vs Alpha among unvaccinated people with no known "
         "prior exposure: its Beta of +1.00 implies 0.46 (blue diamond), but 2.15, the reciprocal, is printed. "
         "The printed 2.15 is the Alpha / B.1.177 ratio, consistent with the Discussion; the Abstract's "
         "\"44% lower among Alpha\" matches neither orientation.",
}
FIGS = {"A": fig_A, "B": fig_B, "C": fig_C, "D": fig_D, "E": fig_E}
for name, fn in FIGS.items():
    save(fn(), os.path.join(CAND, f"cand_{name}"))
print("\nwrote candidates/cand_A..E (.svg, .png)")

if "--pick" in sys.argv:
    picks = sys.argv[sys.argv.index("--pick") + 1].upper().split(",")
    for i, name in enumerate(picks, 1):
        save(FIGS[name](), os.path.join(HERE, f"fig{i}"))
        print(f"\nfig{i} = cand_{name}\nFigure {i}. {CAPTIONS[name]}")

The abstract and the body disagree on nitric oxide after arg1 knockdown; which wording is right depends on the comparison

PLoS Pathog (2012) · doi:10.1371/journal.ppat.1002417 · PMC3261917

Summary

The abstract reports that miRNAi-mediated knockdown of host arginase 1 (arg1) in BHK cells "led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." The other three places in the body that describe the nitric oxide (NO) result of the same Fig. 6I experiment use the opposite wording: the Results subsection heading reads "promotes parasite killing without enhanced NO production," the Results prose says the enhanced killing "was not accompanied by greater parasite-induced NO production (Fig. 6I)," the Fig. 6I legend says "Arg1 knockdown did not induce NO production," and the Discussion says the effect "did not appear to be driven by enhanced production of NO." This is a wording inconsistency about the direction of a single result.

It can be resolved only by naming the comparison, because the two descriptions refer to different control groups and both can be literally true. The paper prints no NO numbers — they appear only inside the Fig. 6I bar chart — so we digitised the four bars from the publisher's figure image. Relative to infected non-targeted control cells, knockdown NO is about 1.37 times higher (≈5.5 vs ≈4.0 µM); the figure draws no significance marker for that difference, and this is the comparison the abstract's "increased generation of nitric oxide" fits. Relative to uninfected cells, knockdown NO is essentially unchanged (≈5.5 vs ≈5.4 µM); that is the comparison the heading, prose, legend and Discussion fit when they say NO was not induced or not enhanced. So the inconsistency is in how the finding is phrased and against which reference, not a contradiction of the underlying measurement. It matters because the abstract and the body read as opposite directional claims about the same experiment, and the paper's stated mechanism is NO-independent (polyamine-mediated), so a reader of the abstract alone could take the opposite of what the body concludes.

What the paper reports

The abstract sentence (the two claims are "increased NO" and "reduced parasite burden"):

"miRNAi-mediated selective knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." (Abstract)

The Results subsection heading for the same experiment:

"Knockdown of host arg1 promotes parasite killing without enhanced NO production" (Results, subsection heading)

The Results prose, citing Fig. 6I directly:

"This enhancement of parasite killing by knockdown of arg1 in BHK cells was not accompanied by greater parasite-induced NO production (Fig. 6I), even though BHK cells were fully capable of generating NO (see Fig. 5B). Thus it would appear that parasite-induced macrophage arg1 contributes to L. donovani replication through mechanisms other than reduction of NO production." (Results, same subsection)

The Fig. 6I legend:

"Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani." (Figure 6, legend, panel I)

The Discussion:

"This increased control of infection mediated by targeted arg1 knockdown did not appear to be driven by enhanced production of NO by the isolated infected macrophage, suggesting that the parasite-induced arginase has a pathological effect through the increase in polyamines that promote parasite growth." (Discussion)

What the paper's own figure shows

Method. No per-cell NO values are archived in text; the Fig. 6I bars are the only source. We downloaded the publisher's Figure 6 image (journals.plos.org, image g006, size=large) and measured the four bars in pixels against the panel's y-axis ticks (0 to 8 µM), using python. The extraction — axis calibration, bar tops, SD caps, and the position of the one significance bracket — is in recompute.py.txt, section 2, and reruns from the downloaded image. Reading precision is about ±0.05 µM (3 px), so the values below are estimates, not the authors' numbers.

Digitised Fig. 6I values (nitrites + nitrates, mean ± SD, µM):

Uninfected Infected (48 h)
Arg1 knockdown ≈5.4 ± 0.3 ≈5.5 ± 0.4
Non-targeted control ≈6.3 ± 0.5 ≈4.0 ± 0.5

The single significance bracket (**) runs between the two non-targeted bars (uninfected vs infected): infection lowered NO in control cells (≈6.3 → ≈4.0 µM). Knockdown cells held their baseline (≈5.4 → ≈5.5 µM). So:

Both the abstract and the body can therefore be read as literally true, each against a different control. The body's heading, prose and legend speak of "induction" and of NO relative to the uninfected or baseline state; the abstract's "increased generation of nitric oxide" fits a comparison of knockdown with the infected non-targeted control.

The second claim in the abstract sentence (reduced parasite burden) is supported by the Results, which give p<0.01 at 48 h and p<0.001 at 72 h (Fig. 6H); the abstract's bound p<0.005 is satisfied by the 72 h value. The knockdown efficiency (>90% reduction in arg1 mRNA, p=0.002; Fig. 6G) is not in dispute.

Quantity Location Reported Digitised / cross-check
NO after arg1 knockdown (wording) Abstract "increased generation of nitric oxide" Fits KD vs infected non-targeted control: ≈5.5 vs ≈4.0 µM (×1.37; no significance marker drawn)
NO after arg1 knockdown (wording) Results heading "without enhanced NO production" Fits KD infected vs uninfected: ≈5.5 vs ≈5.4 µM (not induced)
NO after arg1 knockdown (wording) Results prose (Fig. 6I) "not accompanied by greater parasite-induced NO production" Infection did not raise NO in KD cells above its uninfected level
NO after arg1 knockdown (wording) Fig. 6I legend "did not induce NO production" KD infected ≈5.5 µM vs KD uninfected ≈5.4 µM (no induction)
NO after arg1 knockdown (wording) Discussion "did not appear to be driven by enhanced production of NO" Hedged; consistent with the no-induction reading
Only significance marker in Fig. 6I (**) Fig. 6I between non-targeted bars Non-targeted uninfected ≈6.3 vs infected ≈4.0 µM (infection lowered control NO)
Reduced parasite burden p-value Abstract p<0.005 Consistent: Results give p<0.01 (48 h), p<0.001 (72 h); the 72 h value satisfies p<0.005
arg1 mRNA knockdown efficiency Results / Fig. 6G >90% reduction, p=0.002 Stated once; not in dispute

Figure

Two text panels — the abstract, which says NO increased, and the four body locations, which say it was not enhanced or induced — above a dot-and-interval plot of the four Fig. 6I values digitised from the published image, annotated to show the two comparisons each wording fits.

Figure 1. The abstract's wording (top left) and the body's wording (top right) describe the same Fig. 6I knockdown experiment but fit different reference groups. Below, the four Fig. 6I bars are shown as means ± SD on one axis, digitised from the publisher's figure image by pixel measurement (filled = Arg1 knockdown, open = non-targeted; the paper prints no NO numbers). Comparison (a): knockdown vs non-targeted among infected cells, ≈5.5 vs ≈4.0 µM (×1.37), which has no significance marker in the figure and matches the abstract's "increased" wording. Comparison (b): knockdown infected vs uninfected, ≈5.5 vs ≈5.4 µM, essentially unchanged, which matches the body's "not induced" wording. The paper's only significance mark (**) is between the two non-targeted bars (uninfected vs infected). All quotes are verbatim from the PMC-archived text; the blue values are digitised estimates, not the authors' numbers, and the paper's own figure is not reproduced.

Why it matters

The two statements sit in different parts of the paper — one in the abstract, which databases index, and the rest in the Results, legend and Discussion — and they read as opposite directional claims about the same measurement. The abstract's "increased generation of nitric oxide," taken on its own, suggests the canonical arginase-versus-NOS2 story in which inhibiting host arginase restores NO and that NO kills the parasite. The body instead concludes that enhanced parasite control was not driven by induction of NO and attributes it to reduced polyamine synthesis, explicitly NO-independent. Read against the Fig. 6I values, neither statement is simply wrong: the abstract's wording fits a comparison with infected control cells (for which the figure shows no significance marker), while the body's wording fits a comparison with uninfected or baseline cells. If a reader reconciles the abstract and the body by the wrong comparison, they could take away a mechanism the paper does not claim. The paper's principal conclusions are unaffected: arg1 knockdown reduces parasite burden, STAT6 knockdown reduces arg1 and enhances parasite control, and parasite-induced STAT6/arg1 drives progressive disease; these depend on the NO-independent reading. A secondary, cosmetic point: the abstract gives p<0.005 for the burden effect while the Results give p<0.01 at 48 h and p<0.001 at 72 h; the bound is satisfied by the 72 h value and changes no conclusion.

Explanations considered

Two smaller, cosmetic inconsistencies, noted for completeness; neither changes any conclusion:

Status

As of 2026-10-23. Version of record checked at the publisher (journals.plos.org, PLoS Pathogens): the abstract sentence, the Results heading, the Results prose, and the Fig. 6I legend match the PMC archive word for word, and the Discussion sentence matches. The Fig. 6I values were digitised from the publisher's Figure 6 image (image g006, size=large); the SHA-256 of the downloaded image is printed by recompute.py.txt. Supplements S1-S4 were identified from the publisher page and none covers NO after arg1 knockdown; there is no source-data file. Corrections: none found. Crossref lists no update/correction relation; src/known.py returned no PMC or Europe PMC notices; the Europe PMC record for PMID 22275864 has no comment/correction list and no erratum citing the DOI; the publisher page shows no correction notice and zero reader comments. PubPeer could not be checked (API behind Cloudflare, search page returned HTTP 403); web searches found no PubPeer thread or Retraction Watch coverage. Sources: src/show.py PMC3261917 --full (PMC-archived abstract, Results, Fig. 5/6 legends, Discussion, Methods); publisher HTML and Figure 6 image; Crossref and Europe PMC via src/known.py.

Suggested correction

Reconcile the abstract and the body by stating, in both, the comparison being made. For example, the abstract and the Fig. 6I-related text could say that after infection, arg1-knockdown cells retained their baseline NO while non-targeted control cells lost NO, so that knockdown NO was higher than in infected controls but was not induced above uninfected levels. We do not recommend simply reversing the abstract to "without increased generation of nitric oxide," because against the infected non-targeted control that wording could itself be inaccurate; the correct phrasing depends on which comparison the authors intend. The two cosmetic points above (the "**" p-value in the Fig. 6I legend, and the "see Fig. 5B" vs Fig. 6B citation) could be corrected in the same notice. A published author correction is the appropriate mechanism; no data or main conclusions need revision.

Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Checks for PMC3261917
"Progressive Visceral Leishmaniasis Is Driven by Dominant Parasite-induced
 STAT6 Activation and STAT6-dependent Host Arginase 1 Expression"
PLoS Pathog 2012, doi:10.1371/journal.ppat.1002417

The issue is a wording inconsistency about nitric oxide (NO) after miRNAi
knockdown of host arginase 1 (arg1) in BHK cells. The abstract says knockdown
"led to increased generation of nitric oxide"; the Results heading, Results
prose, Fig. 6I legend and Discussion say NO was not enhanced / not induced.
Whether each statement is true depends on which control the knockdown is
compared with, so this script also reads the Fig. 6I bars from the
publisher's published figure image.

Sections
  1. An ENCODING (not a computation) of what each quoted location says.
  2. Digitisation of Fig. 6I from the publisher's Figure 6 image
     (downloaded at run time to a temporary folder; the image is not stored
     or redrawn here).
  3. Computed comparisons on the digitised values and on stated p-values.
  4. Figures: fig1.svg/png (the layout used in writeup.md) and five
     alternative layouts in ./candidates/ (cand1..cand5).

Run:  python recompute.py.txt
Needs numpy and matplotlib, and network access to journals.plos.org.
"""
import os, sys, hashlib, tempfile, textwrap, urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from matplotlib.patches import FancyBboxPatch, ConnectionPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# ----------------------------------------------------------------------
# 1. ENCODING of quoted statements (hand-coded, not computed)
# ----------------------------------------------------------------------
# Each statement is quoted verbatim from the PMC-archived text
# (src/show.py PMC3261917 --full). The second field is the comparison the
# wording implies, as read by the write-up; it is an encoding, not a result.
statements = [
    ("Abstract",
     "led to increased generation of nitric oxide and reduced parasite burden (p<0.005)",
     "NO increased (reference not stated)"),
    ("Results heading",
     "Knockdown of host arg1 promotes parasite killing without enhanced NO production",
     "NO not enhanced"),
    ("Results text",
     "was not accompanied by greater parasite-induced NO production (Fig. 6I)",
     "no greater parasite-induced NO (i.e. no induction by infection)"),
    ("Fig. 6I legend",
     "Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani.",
     "NO not induced"),
    ("Discussion",
     "did not appear to be driven by enhanced production of NO",
     "NO not enhanced (hedged)"),
]
print("=" * 72)
print("1. ENCODING OF QUOTED STATEMENTS (hand-coded reading, not a computation)")
print("=" * 72)
for loc, q, reading in statements:
    print("  %-16s %s" % (loc, reading))
n_up = sum(1 for s in statements if s[2].startswith("NO increased"))
print("  Locations whose wording says NO increased: %d of %d" % (n_up, len(statements)))

# ----------------------------------------------------------------------
# 2. DIGITISE Fig. 6I from the publisher's Figure 6 image
# ----------------------------------------------------------------------
FIG_URL = ("https://journals.plos.org/plospathogens/article/figure/image"
           "?size=large&id=10.1371/journal.ppat.1002417.g006")
tmpdir = tempfile.mkdtemp(prefix="pmc3261917_")
img_path = os.path.join(tmpdir, "g006.png")
try:
    req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
    with urllib.request.urlopen(req, timeout=60) as r, open(img_path, "wb") as f:
        f.write(r.read())
except Exception as e:
    sys.exit("Could not download Figure 6 image (%s); digitisation needs it." % e)
sha = hashlib.sha256(open(img_path, "rb").read()).hexdigest()
img = mpimg.imread(img_path)[..., :3].mean(axis=2)
dark = img < 0.5
print("\n" + "=" * 72)
print("2. DIGITISATION OF Fig. 6I (publisher image g006, size=large)")
print("=" * 72)
print("  image %d x %d px, sha256 %s" % (img.shape[1], img.shape[0], sha))
assert img.shape[:2] == (2556, 2042), "image size changed; re-check pixel ranges"

def runs(v, off=0):
    out, s = [], None
    for i, b in enumerate(v):
        if b and s is None: s = i
        if not b and s is not None: out.append((s + off, i - 1 + off)); s = None
    if s is not None: out.append((s + off, len(v) - 1 + off))
    return out

# Panel I sits at x 1520-2042, y 1930-2440 of the image. The y axis line is at
# x = 1541-1542; tick marks extend left of it. Read the tick rows at x = 1535.
tick_runs = runs(dark[1930:2440, 1535], 1930)
tick_y = np.array([(a + b) / 2 for a, b in tick_runs])
tick_val = np.array([8, 6, 4, 2, 0], float)            # axis labels, top to bottom
assert len(tick_y) == 5, tick_runs
slope, icpt = np.polyfit(tick_y, tick_val, 1)          # uM per pixel row
resid = tick_val - (slope * tick_y + icpt)
px_per_uM = -1 / slope
print("  tick rows (px):", ", ".join("%.1f" % y for y in tick_y), "-> 0..8 uM")
print("  scale %.2f px per uM; max tick residual %.4f uM" % (px_per_uM, abs(resid).max()))
to_uM = lambda y: slope * y + icpt

# Bar column ranges (px), read from the image by eye; tops and caps are
# then found automatically. Black bars = Arg1 knockdown, white = non-targeted.
bars = {
    ("uninfected", "knockdown"):    (1578, 1652),
    ("uninfected", "non-targeted"): (1652, 1727),
    ("infected",   "knockdown"):    (1836, 1910),
    ("infected",   "non-targeted"): (1910, 1984),
}
Y_MIN = 2035   # below the significance bracket (row ~2031) and the legend key
D = {}
for key, (x0, x1) in bars.items():
    tops = []
    for x in (x0 + 6, x1 - 6):                    # near the bar edges, away from the SD stem
        r = [a for a, b in runs(dark[Y_MIN:2424, x], Y_MIN)]
        tops.append(r[0] + 0.5)                  # centre of the 2-px top outline
    top = float(np.mean(tops))
    xc = (x0 + x1) / 2
    caps = []
    for x in range(int(xc) - 8, int(xc) + 9):
        if abs(x - xc) <= 2: continue            # skip the vertical SD stem
        short = [a for a, b in runs(dark[2008:int(top) - 3, x], 2008) if b - a <= 4]
        if short: caps.append(max(short) + 0.5)  # lowest short run above the bar = SD cap
    cap = float(np.median(caps))
    D[key] = dict(mean=to_uM(top), sd=to_uM(cap) - to_uM(top), xc=xc)
    print("  %-10s %-13s mean %.2f uM   SD %.2f uM   (top row %.1f, cap row %.1f)"
          % (key[0], key[1], D[key]["mean"], D[key]["sd"], top, cap))

# The one significance bracket: a horizontal 2-px line at row ~2031 above the bars.
br = [r for r in runs(dark[2031, 1560:2020], 1560) if r[1] - r[0] > 40]
assert len(br) == 1, br
bx0, bx1 = br[0]
def bar_at(x):
    for k, (a, b) in bars.items():
        if a <= x <= b: return k
print("  significance bracket (**) spans x %d-%d: from %s %s to %s %s"
      % (bx0, bx1, *bar_at(bx0), *bar_at(bx1)))
print("  reading precision: about +/-0.05 uM (3 px); values are estimates.")

# ----------------------------------------------------------------------
# 3. COMPUTED COMPARISONS
# ----------------------------------------------------------------------
print("\n" + "=" * 72)
print("3. COMPARISONS ON THE DIGITISED Fig. 6I MEANS")
print("=" * 72)
ku = D[("uninfected", "knockdown")]["mean"]; nu = D[("uninfected", "non-targeted")]["mean"]
ki = D[("infected", "knockdown")]["mean"];   ni = D[("infected", "non-targeted")]["mean"]
cmp = {
    "KD infected vs non-targeted infected (abstract-type comparison)": (ki, ni),
    "KD infected vs KD uninfected (induction by infection)":           (ki, ku),
    "KD infected vs non-targeted uninfected":                          (ki, nu),
    "non-targeted infected vs non-targeted uninfected (** bracket)":   (ni, nu),
}
R = {}
for name, (a, b) in cmp.items():
    R[name] = (a - b, a / b)
    print("  %-66s diff %+.2f uM  ratio %.2f" % (name, a - b, a / b))
ratio_kd_nt_inf = ki / ni
print("  -> Relative to infected non-targeted cells, knockdown NO is higher (x%.2f);"
      % ratio_kd_nt_inf)
print("     no significance marker is drawn for this difference.")
print("  -> Relative to uninfected knockdown cells, infection changed NO by %+.2f uM"
      % (ki - ku))
print("     (not induced).")

print("\n  Stated p-value checks")
abs_bound = 0.005
results_burden = {"48 h": 0.01, "72 h": 0.001}      # Results: p<0.01, p<0.001
for t, p in results_burden.items():
    print("    parasite burden %s: stated p<%g; within the abstract's p<%g? %s"
          % (t, p, abs_bound, p <= abs_bound))
legend_key = {"*": 0.05, "**": 0.01, "***": 0.001}  # Fig. 6 legend key
panel_I_note = ("**", 0.001)                         # panel I text: "(**p<0.001)"
print("    Fig. 6I legend: '%s' given as p<%g; key defines '%s' as p<%g; same? %s"
      % (panel_I_note[0], panel_I_note[1], panel_I_note[0],
         legend_key[panel_I_note[0]], panel_I_note[1] == legend_key[panel_I_note[0]]))
print("    arg1 mRNA knockdown efficiency: >90% reduction, p = 0.002 (Fig. 6G); stated once")

# ----------------------------------------------------------------------
# 4. FIGURES
# ----------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; SEC = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3e0"; OR_BG = "#fdf0ea"; BL_BG = "#edf3fb"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
    "axes.edgecolor": SEC, "axes.labelcolor": INK, "xtick.color": SEC, "ytick.color": SEC,
    "figure.facecolor": BG, "savefig.facecolor": BG, "axes.facecolor": BG,
    "xtick.labelsize": 10, "ytick.labelsize": 10, "lines.linewidth": 1.5})
W = 7.2
os.makedirs(os.path.join(HERE, "candidates"), exist_ok=True)

def wrap(s, n): return "\n".join(textwrap.wrap(s, n))
def save(fig, path_stem):
    fig.savefig(path_stem + ".svg")
    fig.savefig(path_stem + ".png", dpi=200)
    plt.close(fig)
def panel(ov, x, y, w, h, fc, ec):
    ov.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.008",
                 fc=fc, ec=ec, lw=1.5, transform=ov.transAxes, zorder=1))
def overlay(fig):
    ov = fig.add_axes([0, 0, 1, 1]); ov.axis("off"); ov.set_xlim(0, 1); ov.set_ylim(0, 1)
    ov.set_zorder(-1); return ov

Q_ABS = '"...knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)."'
Q_BODY = [("Results heading", '"...promotes parasite killing without enhanced NO production"'),
          ("Results text", '"...was not accompanied by greater parasite-induced NO production (Fig. 6I)"'),
          ("Fig. 6I legend", '"Arg1 knockdown did not induce NO production..."'),
          ("Discussion", '"...did not appear to be driven by enhanced production of NO..."')]
CONDS = [("uninfected", "knockdown"), ("uninfected", "non-targeted"),
         ("infected", "knockdown"), ("infected", "non-targeted")]
LAB_ABS = "true if compared with infected\nnon-targeted cells: %.1f vs %.1f µM\n(×%.2f; no significance marker)" % (ki, ni, ratio_kd_nt_inf)
LAB_BODY = "true if compared with uninfected\ncells: %.1f vs %.1f µM (not induced)" % (ki, ku)
SRC = ("Quotes verbatim from the PMC-archived text. Blue values digitised by pixel measurement from the "
       "publisher's Fig. 6I image (mean ± SD, ±0.05 µM reading error); the paper prints no NO numbers.")

def dot_axis(ax, xpos, show_legend=True, ylabel=True):
    """Digitised Fig. 6I means ± SD as dots on one 0-8 uM axis."""
    ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8])
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"): ax.spines[s].set_visible(False)
    for k in CONDS:
        m, sd = D[k]["mean"], D[k]["sd"]; x = xpos[k]
        filled = k[1] == "knockdown"
        ax.errorbar(x, m, yerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if filled else BG,
                    mec=BLUE, mew=2, elinewidth=2, capsize=4, zorder=3)
    if ylabel: ax.set_ylabel("Nitrites + nitrates (µM)")
    if show_legend:
        h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BLUE, mew=2, label="Arg1 knockdown"),
             Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BG, mew=2, label="Non-targeted")]
        ax.legend(handles=h, title="Digitised from Fig. 6I", frameon=False, loc="lower left",
                  fontsize=10, title_fontsize=10)

def bracket(ax, x0, x1, y, text, color=INK, dy=0.15, va="bottom", fs=10):
    ax.plot([x0, x0, x1, x1], [y - dy, y, y, y - dy], color=color, lw=1.5, zorder=4)
    ax.text((x0 + x1) / 2, y + 0.08, text, ha="center", va=va, color=color, fontsize=fs)

# ---- Layout A: text panels on top, dot plot below with the comparisons ----
def layout_A(stem):
    fig = plt.figure(figsize=(W, 8.2)); ov = overlay(fig)
    fig.text(0.02, 0.982, "Arg1 knockdown and NO (Fig. 6I): the abstract and the body fit different reference groups",
             fontsize=10, color=SEC, va="top")
    panel(ov, 0.02, 0.60, 0.40, 0.35, OR_BG, ORANGE)
    fig.text(0.04, 0.932, "ABSTRACT  (paper's text)", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.04, 0.895, wrap(Q_ABS, 36), va="top")
    fig.text(0.04, 0.615, "says NO increased: fits (a)", color=ORANGE, fontweight="bold")
    panel(ov, 0.44, 0.60, 0.54, 0.35, OR_BG, ORANGE)
    fig.text(0.46, 0.932, "BODY, 4 locations  (paper's text)", color=ORANGE, fontweight="bold", va="top")
    y = 0.895
    for loc, q in Q_BODY:
        t = wrap(loc + ": " + q, 50)
        fig.text(0.46, y, t, va="top"); y -= 0.0265 * (t.count("\n") + 1) + 0.010
    fig.text(0.46, 0.615, "says NO not enhanced / not induced: fits (b)", color=ORANGE, fontweight="bold")
    fig.add_artist(ConnectionPatch(xyA=(0.22, 0.598), xyB=(0.71, 0.598), coordsA="figure fraction",
                   coordsB="figure fraction", arrowstyle="<->", color=INK, lw=2, connectionstyle="arc3,rad=0.12"))
    fig.text(0.465, 0.548, "opposite wording, same experiment", ha="center", va="top", fontweight="bold")
    ax = fig.add_axes([0.10, 0.12, 0.88, 0.37])
    xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.3, CONDS[3]: 2.7}
    dot_axis(ax, xpos)
    ax.set_xlim(0.4, 4.1); ax.set_xticks([1.0, 2.5]); ax.set_xticklabels(["Uninfected", "Infected (48 h)"])
    # (a) knockdown vs non-targeted, infected
    ax.plot([2.85, 2.95, 2.95, 2.85], [ki, ki, ni, ni], color=INK, lw=1.5)
    ax.text(3.02, (ki + ni) / 2, "(a) ×%.2f\nno significance\nmarker" % ratio_kd_nt_inf, va="center")
    # (b) knockdown infected vs uninfected
    ax.plot([0.8, 2.3], [ku, ki], color=SEC, lw=1.5, ls=":")
    ax.text(1.55, 4.95, "(b) %+.1f µM:\nnot induced" % (ki - ku), ha="center", va="top", color=INK)
    # the paper's only significance bracket
    bracket(ax, 1.2, 2.7, 7.25, "** (the paper's only mark): non-targeted, uninfected vs infected",
            color=SEC, dy=0.2)
    fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC, va="bottom")
    save(fig, stem)

# ---- Layout B: slope chart, quotes as side callouts ----
def layout_B(stem):
    fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
    ax = fig.add_axes([0.10, 0.22, 0.40, 0.68])
    for grp, mfc, ls in (("knockdown", BLUE, "-"), ("non-targeted", BG, "--")):
        m = [D[(c, grp)]["mean"] for c in ("uninfected", "infected")]
        s = [D[(c, grp)]["sd"] for c in ("uninfected", "infected")]
        ax.errorbar([0, 1], m, yerr=s, color=BLUE, ls=ls, lw=2, marker="o", ms=9, mfc=mfc, mew=2,
                    capsize=4, elinewidth=1.5, zorder=3)
        ax.text(1.08, m[1], ("Arg1 knockdown" if grp == "knockdown" else "Non-targeted") + "\n%.1f µM" % m[1],
                color=BLUE, va="center")
        ax.text(-0.08, m[0], "%.1f" % m[0], color=BLUE, va="center", ha="right")
    ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8]); ax.grid(axis="y", color=GRID, lw=0.8)
    ax.set_xlim(-0.35, 1.55); ax.set_xticks([0, 1]); ax.set_xticklabels(["Uninfected", "Infected"])
    for s in ("top", "right"): ax.spines[s].set_visible(False)
    ax.set_ylabel("Nitrites + nitrates (µM)")
    ax.set_title("Fig. 6I, digitised (mean ± SD)", fontsize=10, color=BLUE, loc="left")
    panel(ov, 0.60, 0.57, 0.38, 0.36, OR_BG, ORANGE)
    fig.text(0.62, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.62, 0.87, wrap('"...led to increased generation of nitric oxide..."', 34), va="top")
    fig.text(0.62, 0.78, wrap("Fits knockdown vs non-targeted, infected: %.1f vs %.1f µM (×%.2f), no significance marker."
                              % (ki, ni, ratio_kd_nt_inf), 34), va="top", color=SEC)
    panel(ov, 0.60, 0.14, 0.38, 0.40, OR_BG, ORANGE)
    fig.text(0.62, 0.52, "RESULTS, LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.62, 0.48, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
                              '"did not induce NO production"', 34), va="top")
    fig.text(0.62, 0.27, wrap("Fits knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced." % (ki, ku), 34),
             va="top", color=SEC)
    fig.text(0.02, 0.975, "Same experiment, two references: which comparison each text describes", fontsize=10, color=SEC, va="top")
    fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
    save(fig, stem)

# ---- Layout C: three columns, Abstract | digitised values | Body ----
def layout_C(stem):
    fig = plt.figure(figsize=(W, 5.0)); ov = overlay(fig)
    fig.text(0.02, 0.975, "Abstract vs body on NO after arg1 knockdown, read against the Fig. 6I values",
             fontsize=10, color=SEC, va="top")
    panel(ov, 0.02, 0.14, 0.26, 0.77, OR_BG, ORANGE)
    fig.text(0.035, 0.89, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.035, 0.845, wrap(Q_ABS, 23), va="top")
    fig.text(0.035, 0.165, "NO increased", color=ORANGE, fontweight="bold")
    panel(ov, 0.72, 0.14, 0.26, 0.77, OR_BG, ORANGE)
    fig.text(0.735, 0.89, "BODY (4 places)", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.735, 0.845, wrap('"without enhanced NO production"; "...not accompanied by greater parasite-induced NO production"; '
                                '"did not induce NO production"; "did not appear to be driven by enhanced production of NO"', 23), va="top")
    fig.text(0.735, 0.165, wrap("NO not enhanced / not induced", 18), color=ORANGE, fontweight="bold")
    panel(ov, 0.30, 0.14, 0.40, 0.77, BL_BG, BLUE)
    fig.text(0.50, 0.89, "Fig. 6I, digitised", color=BLUE, fontweight="bold", va="top", ha="center")
    fig.text(0.50, 0.845, "mean ± SD, µM", color=SEC, va="top", ha="center")
    rows = [("", "Uninfected", "Infected"),
            ("Knockdown", "%.1f ± %.1f" % (ku, D[CONDS[0]]["sd"]), "%.1f ± %.1f" % (ki, D[CONDS[2]]["sd"])),
            ("Non-targeted", "%.1f ± %.1f" % (nu, D[CONDS[1]]["sd"]), "%.1f ± %.1f" % (ni, D[CONDS[3]]["sd"]))]
    for i, r in enumerate(rows):
        yy = 0.77 - i * 0.075
        fig.text(0.315, yy, r[0], va="top")
        fig.text(0.515, yy, r[1], va="top", ha="center", color=BLUE if i else SEC)
        fig.text(0.625, yy, r[2], va="top", ha="center", color=BLUE if i else SEC)
    fig.text(0.315, 0.53, wrap("(a) knockdown vs non-targeted, infected: %.1f vs %.1f µM, ×%.2f, no significance marker"
                               % (ki, ni, ratio_kd_nt_inf), 36), va="top")
    fig.text(0.315, 0.39, wrap("(b) knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced" % (ki, ku), 36), va="top")
    fig.text(0.315, 0.27, wrap("The paper's only mark (**): non-targeted, uninfected vs infected", 40), va="top", color=SEC)
    fig.add_artist(ConnectionPatch(xyA=(0.28, 0.50), xyB=(0.31, 0.50), coordsA="figure fraction",
                   coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
    fig.add_artist(ConnectionPatch(xyA=(0.72, 0.36), xyB=(0.69, 0.36), coordsA="figure fraction",
                   coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
    fig.text(0.02, 0.012, wrap("Quotes verbatim (PMC text). Abstract fits (a); body fits (b). Values digitised from the "
                               "publisher's Fig. 6I image (±0.05 µM reading error); the paper prints no NO numbers.", 98),
             fontsize=10, color=SEC)
    save(fig, stem)

# ---- Layout D: horizontal dot-interval rows on one uM axis, quotes above ----
def layout_D(stem):
    fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
    panel(ov, 0.02, 0.75, 0.47, 0.21, OR_BG, ORANGE)
    fig.text(0.035, 0.94, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.035, 0.895, wrap('"...knockdown... led to increased generation of nitric oxide..."', 42), va="top")
    fig.text(0.035, 0.77, "NO increased: fits (a)", color=ORANGE, fontweight="bold")
    panel(ov, 0.51, 0.75, 0.47, 0.21, OR_BG, ORANGE)
    fig.text(0.525, 0.94, "RESULTS, Fig. 6I LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.525, 0.895, wrap('"without enhanced NO production"; "did not induce NO production"', 42), va="top")
    fig.text(0.525, 0.77, "NO not induced: fits (b)", color=ORANGE, fontweight="bold")
    ax = fig.add_axes([0.30, 0.25, 0.60, 0.44])
    order = [CONDS[0], CONDS[2], CONDS[3], CONDS[1]]          # top to bottom
    names = {CONDS[0]: "Knockdown, uninfected", CONDS[2]: "Knockdown, infected",
             CONDS[3]: "Non-targeted, infected", CONDS[1]: "Non-targeted, uninfected"}
    yv = {k: 3 - i for i, k in enumerate(order)}
    for k in order:
        m, sd = D[k]["mean"], D[k]["sd"]; f = k[1] == "knockdown"
        ax.errorbar(m, yv[k], xerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if f else BG, mew=2, elinewidth=2, capsize=4)
        ax.text(m, yv[k] + 0.22, "%.1f" % m, color=BLUE, ha="center", va="bottom")
    ax.set_yticks([yv[k] for k in order]); ax.set_yticklabels([names[k] for k in order])
    ax.set_xlim(0, 8); ax.set_xticks([0, 2, 4, 6, 8]); ax.set_ylim(-0.5, 3.7)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"): ax.spines[s].set_visible(False)
    ax.set_xlabel("Nitrites + nitrates (µM), digitised from Fig. 6I (mean ± SD)", color=BLUE)
    def vbr(y0, y1, txt, col):
        ax.plot([7.35, 7.5, 7.5, 7.35], [y0 - 0.05, y0 - 0.05, y1 + 0.05, y1 + 0.05], color=col, lw=1.5, clip_on=False)
        ax.text(7.6, (y0 + y1) / 2, txt, va="center", color=col, clip_on=False)
    vbr(3, 2, "(b) %+.1f" % (ki - ku), INK)
    vbr(2 - 0.1, 1 + 0.1, "(a) ×%.2f" % ratio_kd_nt_inf, INK)
    vbr(1 - 0.1, 0, "** paper", SEC)
    fig.text(0.02, 0.015, wrap("(a) knockdown vs non-targeted, infected: higher, no significance marker. (b) knockdown, "
                               "infected vs uninfected: not induced. ** is the paper's only mark. Quotes verbatim (PMC text); "
                               "values digitised from the publisher's Fig. 6I image (±0.05 µM reading error).", 98),
             fontsize=10, color=SEC)
    save(fig, stem)

# ---- Layout E: stacked text panels left, dot plot right, connectors to the comparisons ----
def layout_E(stem):
    fig = plt.figure(figsize=(W, 5.9)); ov = overlay(fig)
    panel(ov, 0.02, 0.55, 0.40, 0.38, OR_BG, ORANGE)
    fig.text(0.035, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.035, 0.865, wrap(Q_ABS, 36), va="top")
    fig.text(0.035, 0.575, "NO increased", color=ORANGE, fontweight="bold")
    panel(ov, 0.02, 0.12, 0.40, 0.40, OR_BG, ORANGE)
    fig.text(0.035, 0.50, "BODY (4 locations)", color=ORANGE, fontweight="bold", va="top")
    fig.text(0.035, 0.455, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
                                '"did not induce NO production" · "did not appear to be driven by enhanced production of NO"', 36), va="top")
    fig.text(0.035, 0.145, "NO not enhanced / not induced", color=ORANGE, fontweight="bold")
    ax = fig.add_axes([0.53, 0.12, 0.45, 0.78])
    xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.0, CONDS[3]: 2.4}
    dot_axis(ax, xpos, ylabel=False)
    ax.set_xlim(0.45, 2.75); ax.set_xticks([1.0, 2.2]); ax.set_xticklabels(["Uninfected", "Infected"])
    ax.set_title("Fig. 6I digitised, µM (mean ± SD)", fontsize=10, color=BLUE, loc="left")
    bracket(ax, 2.0, 2.4, 6.5, "×%.2f\nno mark" % ratio_kd_nt_inf)
    ax.plot([0.8, 0.8, 2.0, 2.0], [4.85, 4.65, 4.65, 4.85], color=INK, lw=1.5)
    ax.text(1.4, 4.5, "%+.1f µM\nnot induced" % (ki - ku), ha="center", va="top")
    fig.add_artist(ConnectionPatch(xyA=(0.42, 0.74), coordsA="figure fraction", xyB=(1.98, 7.2), coordsB=ax.transData,
                                   arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
    fig.add_artist(ConnectionPatch(xyA=(0.42, 0.32), coordsA="figure fraction", xyB=(1.0, 3.95), coordsB=ax.transData,
                                   arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=0.15"))
    fig.text(0.02, 0.975, "Two texts, two reference groups (Fig. 6I knockdown experiment)", fontsize=10, color=SEC, va="top")
    fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
    save(fig, stem)

LAYOUTS = [layout_A, layout_B, layout_C, layout_D, layout_E]
for i, f in enumerate(LAYOUTS, 1):
    f(os.path.join(HERE, "candidates", "cand%d" % i))
FIG1 = layout_A                      # layout used in writeup.md
FIG1(os.path.join(HERE, "fig1"))
print("\nWrote fig1.svg/png (layout %s) and candidates/cand1-cand5 in %s"
      % (FIG1.__name__[-1], HERE))

Abstract says vitamin D3 rose in Scribble knockout mice; Results and Fig. 3E show a fall

Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2220851120 · PMC10266016

Summary

The abstract states that knocking out Scribble in renal proximal tubules "caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels". The paper defines a single aggregate vitamin D3 measure: three metabolites, each normalized to littermate controls and then combined. Fig. 3E is the only place it is plotted. The Results text and the Fig. 3 legend both describe that measure as decreased, and the points plotted in Fig. 3E agree: digitized, the six tamoxifen-induced knockout values have a mean of 63.2% of control, and 5 of the 6 lie below 100%. Vitamin D is one of the two in vivo findings the abstract reports, so a reader of the abstract alone gets the direction of that finding reversed.

What the paper reports

Abstract (as archived in PMC, after the Significance statement):

The loss of Scribble impacted serum phosphate and vitamin D levels and caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels, whereas blood glucose levels remained unchanged.

Results, section "Loss of Scribble Impacts Renal Phosphate Homeostasis and Serum Vitamin D Levels", second paragraph:

Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels, as indicated in Methods, decreased only in Tam-injected animals and not in corresponding Cre littermates (Fig. 3E).

Fig. 3 legend:

(B) Targeted Scribble KO mice display increased serum phosphate (C), normal levels of calcium and glucose (D), decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E), and normal levels of PTH and FGF23 (F and G).

Materials and Methods, "Mice.", where the aggregate is defined:

Vitamin D3 was measured as described (52) on littermate pairs. 1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were normalized to control levels and combined.

Fig. 3E (an image; the axis title and group labels were read from the archived figure file pnas.2220851120fig03.jpg): the y-axis is "Plasma Vitamin D % control", the groups are Veh, Tam and Cre, and a bracket marks Veh vs Tam with "*" and Veh vs Cre with "ns".

Statements elsewhere that give no direction. Significance statement:

The loss of this interaction impacts serum phosphate and vitamin D levels in Scribble KO mice.

Discussion, second paragraph:

Targeted loss of Scribble in proximal tubules resulted in pronounced alterations to serum phosphate and vitamin D levels, whereas glucose, calcium, PTH and FGF23 levels remained unaffected.

Discussion, third paragraph, on mechanism:

PTHR activates the 25-hydroxyvitamin D3-1α-hydroxylase catalyzing the formation of biologically active 1,25(OH)2-vitamin D3.

What the paper's own numbers show

Method. The text gives no vitamin D values, means or test statistics, so the direction can be checked in two ways: by comparing what the text says in each place, and by reading the plotted points in Fig. 3E. For the second, I took the figure as archived in PMC and Europe PMC (708 × 927 px). I calibrated the y-axis from its five tick marks (0, 50, 100, 150 and 200%; largest residual 0.5 percentage points) and located each dot as the centroid of its connected block of dark pixels. As a check, the six Veh (control) dots, which are fixed at 100% by the normalization, come out at 99.8%. The mean computed from the digitized dots matches the mean bar drawn in the figure to within 0.7 percentage points for Tam and exactly for Cre. The paper does not say which test produced the "*". Because each knockout value is already a ratio to its own littermate control, a one-sample test against 100% is the natural analogue, and I report it only to show that the plotted points are consistent with the asterisk. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own figure
Direction of the aggregate vitamin D3 measure Abstract "increased" decrease (opposite sign)
Direction of the same measure Results, second paragraph "decreased only in Tam-injected animals" decrease (agrees)
Direction of panel E Fig. 3 legend "decreased … levels (E)" decrease (agrees)
Control (Veh) dots Fig. 3E fixed at 100% 99.8% (calibration check)
Tam (knockout) dots, % of control Fig. 3E plotted only 109.2, 78.0, 55.8, 49.2, 44.0, 42.7
Tam mean (median) Fig. 3E mean bar drawn, no value given 63.2% (52.5%); drawn bar reads 62.5%
Change vs control Fig. 3E — −36.8 percentage points
Tam dots below 100% Fig. 3E — 5 of 6
Tam 95% CI of the mean (t, df = 5) Fig. 3E — 35.9% to 90.4%; excludes 100%
Test of Tam vs control Fig. 3E "*" one-sample t = −3.47, two-sided p = 0.018; exact Wilcoxon p = 0.0625
Cre-only littermates, % of control Fig. 3E "ns" 150.8, 133.4, 126.0, 109.9, 78.5, 69.2; mean 111.3% (95% CI 77.8 to 144.9; p = 0.43)
Digitizing error needed to move the Tam mean to 100% — — 21.9 px for every dot; a dot is about 5 px across

The direction does not depend on how precisely the dots were located: every Tam dot would have to be misplaced by about four dot diameters for the mean to reach 100%. The only group whose mean lies above 100% is Cre, and that difference is marked "ns" and is not attributed to loss of Scribble.

Figure

The abstract says the aggregate vitamin D measure increased; the Results text, the Fig. 3 legend and the points plotted in Fig. 3E all show a decrease

Figure 1. Top: the two text statements about the same aggregate vitamin D measure. The abstract (orange outline) says "increased"; the Results text and Fig. 3 legend (grey outline) say "decreased". Bottom: the six knockout (Tam) values read from Fig. 3E, as % of littermate control (open blue circles), with their mean of 63.2% (filled blue circle) and its t-based 95% interval of 35.9% to 90.4% (blue line); the vertical line marks the control level of 100%. Blue values are recomputed by recompute.py.txt from the paper's own figure; they are not values printed in the paper. The axis starts at zero.

Design candidates for review. recompute.py.txt also writes five candidate layouts of this figure, fig1_A to fig1_E (fig1 is currently design D). Only the chosen design will be kept as fig1.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract, replace

caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels

with

caused significant plasma phosphate elevation and decreased aggregate vitamin D3 levels

so that the abstract agrees with the Results text, the Fig. 3 legend and the data in Fig. 3E. The authors may also wish to make the third metabolite the same in Results ("24(OH)D3") and Methods ("25 vitamin D3"), to describe panel E in the legend as the combined measure, and to give the test used for Fig. 3E with its n and P value.

Recomputation script (python recompute.py.txt)
"""Recompute the direction of the vitamin D result in PMC10266016
(Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2220851120).

Run:  python recompute.py.txt                 (uses the pixel positions recorded below)
      python recompute.py.txt <fig03.jpg>     (also re-measures them from the image)
Needs numpy, scipy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and the
five candidate designs fig1_A ... fig1_E (.svg/.png) next to this file.

Input: Fig. 3E of the paper ("Plasma Vitamin D % control"), as archived in PMC /
Europe PMC (pnas.2220851120fig03.jpg, 708 x 927 px). Panel E is the paper's only
vitamin D data. The plotted values are not printed anywhere in the text, so they were
read off the image: dark pixels (grey < 110) in the panel were grouped into connected
blobs and each dot's centre taken as the blob centroid; two Cre dots that touch were
split by their row profile, and one Cre dot drawn on top of its mean bar was read from
the row profile too (the re-measurement prints the blobs it can separate automatically).
The y-axis was calibrated from the five tick marks.
"""
import os
import sys

import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (Fig. 3E pixels)
TICK_ROW = np.array([448.0, 478.0, 508.0, 537.5, 567.0])   # tick marks on the y-axis (x = 71-75)
TICK_VAL = np.array([200.0, 150.0, 100.0, 50.0, 0.0])      # tick labels, % control
VEH_ROW = 507.8                                            # six Veh dots, one merged row
TAM_ROWS = np.array([502.2, 520.8, 534.0, 537.9, 541.0, 541.8])   # six Tam dots
TAM_BAR_ROW = 530.0                                        # Tam mean bar as drawn
CRE_ROWS = np.array([477.5, 487.8, 492.2, 501.8, 520.5, 526.0])   # six Cre dots
CRE_BAR_ROW = 501.0                                        # Cre mean bar as drawn

# ---------------------------------------------------------------- inputs (text, verbatim)
Q_ABSTRACT = ("caused significant plasma phosphate elevation and increased "
              "aggregate vitamin D3 levels")
Q_RESULTS = ("Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels ... "
             "decreased only in Tam-injected animals")
Q_LEGEND = "decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E)"
Q_METHODS = ("1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were "
             "normalized to control levels and combined.")


def remeasure(path):
    """Re-detect the Fig. 3E dots from the archived JPEG and print their rows."""
    from PIL import Image
    from scipy import ndimage
    im = np.asarray(Image.open(path).convert("L")).astype(float)
    y0, x0 = 440, 60
    sub = im[y0:580, x0:220] < 110
    lab, n = ndimage.label(sub)
    print(f"re-measured from {os.path.basename(path)} ({im.shape[1]}x{im.shape[0]} px):")
    for i, sl in enumerate(ndimage.find_objects(lab)):
        area = int((lab[sl] == i + 1).sum())
        cy, cx = ndimage.center_of_mass(lab == i + 1)
        w = sl[1].stop - sl[1].start
        h = sl[0].stop - sl[0].start
        if 15 <= area <= 50 and w <= 7 and 135 <= cx + x0 <= 205 and cy + y0 > 470:
            grp = "Tam" if cx + x0 < 170 else "Cre"
            note = "  (two touching dots)" if h > 8 else ""
            print(f"  {grp} dot  x={cx + x0:6.1f}  row={cy + y0:6.1f}  area={area}{note}")
    print()


# ---------------------------------------------------------------- calculation
slope, icpt = np.polyfit(TICK_ROW, TICK_VAL, 1)
resid = TICK_VAL - (slope * TICK_ROW + icpt)


def pct(row):
    return slope * np.asarray(row, dtype=float) + icpt


veh = float(pct(VEH_ROW))
tam = pct(TAM_ROWS)
cre = pct(CRE_ROWS)
tam_bar = float(pct(TAM_BAR_ROW))
cre_bar = float(pct(CRE_BAR_ROW))
n = len(tam)
tam_mean = tam.mean()
tam_sd = tam.std(ddof=1)
tam_se = tam_sd / np.sqrt(n)
tcrit = stats.t.ppf(0.975, n - 1)
tam_ci = (tam_mean - tcrit * tam_se, tam_mean + tcrit * tam_se)
tt = stats.ttest_1samp(tam, 100.0)
wx = stats.wilcoxon(tam - 100.0)
cre_tt = stats.ttest_1samp(cre, 100.0)
cre_ci = (cre.mean() - tcrit * cre.std(ddof=1) / np.sqrt(n),
          cre.mean() + tcrit * cre.std(ddof=1) / np.sqrt(n))
# how far the digitizing would have to be off for the sign to flip
px_per_pct = 1.0 / abs(slope)
shift_to_100 = (100.0 - tam_mean) * px_per_pct


def report():
    print("Fig. 3E, plasma vitamin D (% control), digitized")
    print(f"  axis calibration: {slope:.4f} % per px (max tick residual {abs(resid).max():.2f} %)")
    print(f"  Veh row -> {veh:.1f} %  (should be 100: the controls are the normalizing base)")
    print("  Tam points:", ", ".join(f"{v:.1f}" for v in tam))
    print(f"  Tam mean {tam_mean:.1f} %, median {np.median(tam):.1f} %, SD {tam_sd:.1f}")
    print(f"  Tam mean bar as drawn: {tam_bar:.1f} %")
    print(f"  Tam change vs control: {tam_mean - 100:+.1f} percentage points")
    print(f"  Tam points below 100 %: {(tam < 100).sum()} of {n}")
    print(f"  Tam 95% CI of mean (t, df={n - 1}): {tam_ci[0]:.1f} to {tam_ci[1]:.1f} %")
    print(f"  one-sample t vs 100: t = {tt.statistic:.2f}, two-sided p = {tt.pvalue:.4f}")
    print(f"  Wilcoxon signed-rank vs 100: p = {wx.pvalue:.4f} (exact, n = 6)")
    print("  Cre points:", ", ".join(f"{v:.1f}" for v in cre))
    print(f"  Cre mean {cre.mean():.1f} %, mean bar as drawn {cre_bar:.1f} %, "
          f"95% CI {cre_ci[0]:.1f} to {cre_ci[1]:.1f} %, t vs 100 p = {cre_tt.pvalue:.3f}")
    print(f"  digitizing error needed to move the Tam mean up to 100 %: "
          f"{shift_to_100:.1f} px for every dot (dot diameter about 5 px)")
    print()
    print("Direction of the same aggregate measure, as stated in the paper")
    rows = [("Abstract", "increased", "+"),
            ("Results, 'Loss of Scribble Impacts ...' p1", "decreased", "-"),
            ("Fig. 3 legend, panel E", "decreased", "-"),
            ("Fig. 3E plotted data (digitized)",
             f"mean {tam_mean:.0f} % of control", "-" if tam_mean < 100 else "+")]
    for loc, word, sign in rows:
        print(f"  {sign}  {loc:44s} {word}")
    print("  Abstract agrees with the plotted data:", rows[0][2] == rows[3][2])


# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"   # the paper's reported statement / value
BLUE = "#2a78d6"     # recomputed from the paper's own figure
GRID = "#e4e3df"
PANEL = "#f3f2ef"
W = 7.2

plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})

SUB = str.maketrans("23", "₂₃")


def vd(s):
    """Typeset vitamin D names with subscripts."""
    return (s.replace("(OH)2D3", "(OH)₂D₃").replace("(OH)D3", "(OH)D₃")
             .replace("[OH]2", "[OH]₂").replace("D3 ", "D₃ ").replace("D3\n", "D₃\n"))


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
    plt.close(fig)


def textbox(ax, x, y, w, h, title, body, hl=None, hl_color=ORANGE, edge=INK2, hl2=None):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc=PANEL, ec=edge, lw=1.0, transform=ax.transAxes))
    ax.text(x + 0.02, y + h - 0.05, title, transform=ax.transAxes, fontsize=10.5,
            fontweight="bold", va="top", color=INK)
    ax.text(x + 0.02, y + h - 0.16, body, transform=ax.transAxes, fontsize=10.5,
            va="top", color=INK, linespacing=1.45)
    if hl2:
        ax.text(x + 0.02, y + 0.035, hl2, transform=ax.transAxes, fontsize=10,
                fontweight="bold", va="bottom", color=BLUE)
    if hl:
        ax.text(x + 0.02, y + (0.11 if hl2 else 0.05), hl, transform=ax.transAxes, fontsize=11,
                fontweight="bold", va="bottom", color=hl_color)


def blank(figsize):
    fig = plt.figure(figsize=figsize)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_axis_off()
    return fig, ax


# ---------------------------------------------------------------- A: two text panels
def design_A():
    fig, ax = blank((W, 3.7))
    ax.text(0.03, 0.95, "The same aggregate vitamin D measure, described in opposite directions",
            fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
    textbox(ax, 0.03, 0.12, 0.40, 0.70, "Abstract",
            vd("“…caused significant plasma\nphosphate elevation and\n"
               "increased aggregate vitamin D3 \nlevels…”"),
            hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
    textbox(ax, 0.57, 0.12, 0.40, 0.70, "Results text and Fig. 3 legend",
            vd("Results: “Combined 1,25(OH)2D3 +\n24,25(OH)2D3 + 24(OH)D3 levels …\n"
               "decreased only in Tam-injected\nanimals”\n"
               "Legend: “decreased 1,25[OH]2-\nvitamin D … levels (E)”"),
            hl="▼ decreased", hl_color=INK2,
            hl2=f"Fig. 3E: Tam mean {tam_mean:.0f}% of control")
    ax.add_patch(FancyArrowPatch((0.435, 0.47), (0.565, 0.47), transform=ax.transAxes,
                                 arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
    ax.text(0.5, 0.55, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
            color=INK, transform=ax.transAxes, linespacing=1.2)
    ax.text(0.5, 0.40, "same\nmeasure\n(Methods)", ha="center", va="top", fontsize=10,
            color=INK2, transform=ax.transAxes, linespacing=1.2)
    ax.text(0.03, 0.03, "Orange: the abstract's statement.  Blue: recomputed from the "
            "points plotted in Fig. 3E.", fontsize=10, color=INK2, transform=ax.transAxes)
    return fig


# ---------------------------------------------------------------- B: direction ledger
def design_B():
    fig, ax = blank((W, 3.2))
    ax.text(0.03, 0.95, "Every place the paper gives the direction of the vitamin D change",
            fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
    rows = [("Abstract", vd("“increased aggregate vitamin D3 levels”"), "up", ORANGE,
             "stated in the paper"),
            ("Results, p1", vd("“…decreased only in Tam-injected animals”"), "down",
             INK2, "stated in the paper"),
            ("Fig. 3 legend", vd("“decreased 1,25[OH]2-vitamin D … levels (E)”"),
             "down", INK2, "stated in the paper"),
            ("Fig. 3E data", f"Tam mean {tam_mean:.1f}% of control; "
             f"{(tam < 100).sum()} of {n} dots below 100%", "down", BLUE,
             "recomputed (digitized)")]
    ax.text(0.80, 0.81, "increase", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
    ax.text(0.92, 0.81, "decrease", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
    for i, (loc, txt, d, col, src) in enumerate(rows):
        y = 0.68 - i * 0.165
        ax.plot([0.03, 0.97], [y - 0.075, y - 0.075], color=GRID, lw=1, transform=ax.transAxes)
        ax.text(0.03, y + 0.02, loc, fontweight="bold", fontsize=10.5, va="center",
                transform=ax.transAxes)
        ax.text(0.03, y - 0.04, src, fontsize=10, color=INK2, va="center",
                transform=ax.transAxes)
        ax.text(0.22, y, txt, fontsize=10.5, va="center", transform=ax.transAxes)
        xm = 0.80 if d == "up" else 0.92
        ax.text(xm, y, "▲" if d == "up" else "▼", fontsize=15, ha="center",
                va="center", color=col, transform=ax.transAxes)
    ax.text(0.03, 0.02, "Orange: the abstract.  Blue: recomputed from the plotted points.  "
            "Grey: the other text.", fontsize=10, color=INK2,
            transform=ax.transAxes)
    return fig


# ---------------------------------------------------------------- C: one-axis direction scale
def design_C():
    fig, ax = plt.subplots(figsize=(W, 2.9))
    fig.subplots_adjust(left=0.04, right=0.97, top=0.80, bottom=0.25)
    ax.set_xlim(0, 160)
    ax.set_ylim(-1, 1.55)
    ax.set_yticks([])
    for s in ("left", "right", "top"):
        ax.spines[s].set_visible(False)
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    ax.axvline(100, color=INK2, lw=1)
    ax.text(100, 1.50, "control = 100%", ha="center", va="bottom", fontsize=10, color=INK2)
    # abstract: direction only (no number given) -> arrow into the >100 region
    ax.annotate("", xy=(150, 0.65), xytext=(101, 0.65),
                arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
    ax.text(103, 0.78, "Abstract: “increased”\n(direction only, no value)",
            fontsize=10.5, color=INK, va="bottom", linespacing=1.25)
    # recomputed
    ax.scatter(tam, np.full(n, -0.35), s=42, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
    ax.plot(tam_ci, [-0.35, -0.35], color=BLUE, lw=2, zorder=2)
    ax.scatter([tam_mean], [-0.35], s=80, color=BLUE, zorder=4)
    ax.text(tam_mean, -0.05, f"Fig. 3E Tam mean {tam_mean:.1f}% (95% CI {tam_ci[0]:.0f}–"
            f"{tam_ci[1]:.0f}), recomputed", ha="center", fontsize=10.5, va="bottom")
    ax.text(2, -0.85, "Results and legend: “decreased”", fontsize=10, color=INK2)
    ax.set_xlabel("Combined plasma vitamin D in Scribble KO (Tam) mice, % of littermate control",
                  fontsize=10.5)
    fig.text(0.04, 0.93, "The abstract's direction lies on the wrong side of the control line",
             fontsize=11.5, fontweight="bold")
    return fig


# ---------------------------------------------------------------- D: text + strip
def design_D():
    fig = plt.figure(figsize=(W, 3.9))
    ax = fig.add_axes([0, 0.34, 1, 0.66])
    ax.set_axis_off()
    ax.text(0.03, 0.95, "Abstract versus the paper's own text and data on vitamin D",
            fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
    textbox(ax, 0.03, 0.04, 0.40, 0.74, "Abstract",
            vd("“…plasma phosphate elevation\nand increased aggregate\nvitamin D3 "
               "levels…”"), hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
    textbox(ax, 0.57, 0.04, 0.40, 0.74, "Results text and Fig. 3 legend",
            vd("“…decreased only in Tam-\ninjected animals” (Results)\n"
               "“decreased 1,25[OH]2-vitamin D\n… levels (E)” (legend)"),
            hl="▼ decreased", hl_color=INK2)
    ax.add_patch(FancyArrowPatch((0.435, 0.41), (0.565, 0.41), transform=ax.transAxes,
                                 arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
    ax.text(0.5, 0.47, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
            transform=ax.transAxes, linespacing=1.2)
    s = fig.add_axes([0.08, 0.16, 0.86, 0.12])
    s.set_xlim(0, 160)
    s.set_ylim(-1, 1)
    s.set_yticks([])
    for sp in ("left", "right", "top"):
        s.spines[sp].set_visible(False)
    s.grid(axis="x", color=GRID, lw=0.8)
    s.set_axisbelow(True)
    s.axvline(100, color=INK2, lw=1)
    s.scatter(tam, np.zeros(n), s=40, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
    s.plot(tam_ci, [0, 0], color=BLUE, lw=2)
    s.scatter([tam_mean], [0], s=80, color=BLUE, zorder=4)
    s.text(103, 0.15, "control = 100%", fontsize=10, color=INK2, va="bottom")
    s.set_xlabel(f"Fig. 3E, Tam mice: digitized points, mean {tam_mean:.1f}% and 95% CI "
                 f"({tam_ci[0]:.0f}–{tam_ci[1]:.0f}), % of control (recomputed)",
                 fontsize=10, color=INK)
    return fig


# ---------------------------------------------------------------- E: flow of the measure
def design_E():
    fig, ax = blank((W, 3.0))
    ax.text(0.03, 0.95, "One measure, traced from its definition to the abstract",
            fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
    boxes = [(0.03, "Methods", "1,25 + 24,25 +\n25 vitamin D₃,\nnormalized to\ncontrol, combined", INK2, None),
             (0.27, "Fig. 3E data", f"Tam mean\n{tam_mean:.1f}% of control\n"
              f"{(tam < 100).sum()} of {n} below 100", BLUE, "▼"),
             (0.51, "Results, legend", "“decreased”\n“decreased”", INK2,
              "▼"),
             (0.75, "Abstract", "“increased\naggregate\nvitamin D₃\nlevels”", ORANGE,
              "▲")]
    for x, t, b, c, mark in boxes:
        ax.add_patch(FancyBboxPatch((x, 0.22), 0.21, 0.58,
                                    boxstyle="round,pad=0,rounding_size=0.015", fc=PANEL,
                                    ec=c, lw=1.4 if c != INK2 else 1.0, transform=ax.transAxes))
        ax.text(x + 0.015, 0.75, t, fontweight="bold", fontsize=10.5, va="top",
                transform=ax.transAxes)
        ax.text(x + 0.015, 0.63, b, fontsize=10.5, va="top", transform=ax.transAxes,
                linespacing=1.35)
        if mark:
            ax.text(x + 0.015, 0.26, mark, fontsize=16, ha="left", va="bottom", color=c,
                    transform=ax.transAxes)
    for x0 in (0.24, 0.48, 0.72):
        ok = x0 < 0.7
        ax.add_patch(FancyArrowPatch((x0, 0.51), (x0 + 0.03, 0.51), transform=ax.transAxes,
                                     arrowstyle="-|>", mutation_scale=12, lw=2,
                                     color=INK2 if ok else ORANGE))
    ax.text(0.735, 0.14, "sign flips here", ha="center", fontsize=10.5, color=INK,
            fontweight="bold", transform=ax.transAxes)
    ax.text(0.03, 0.03, "Blue: recomputed from the plotted points.  Orange: the abstract's "
            "statement.", fontsize=10, color=INK2, transform=ax.transAxes)
    return fig


DESIGNS = {"A": design_A, "B": design_B, "C": design_C, "D": design_D, "E": design_E}
CHOSEN = "D"   # the design used as fig1 in writeup.md

if __name__ == "__main__":
    if len(sys.argv) > 1:
        remeasure(sys.argv[1])
    report()
    for k, f in DESIGNS.items():
        save(f(), f"fig1_{k}")
    save(DESIGNS[CHOSEN](), "fig1")
    print(f"\nwrote fig1_A..fig1_E (.svg/.png) and fig1 = design {CHOSEN} to {HERE}")

Abstract's tissue-culture counts (1/6, 6/6) conflict with the paper's Results and design (2/8, 8/8)

Proc Natl Acad Sci U S A (2024) · doi:10.1073/pnas.2318627121 · PMC11331133

Summary

The abstract reports the study's postmortem tissue-culture result as MAP-positive tissue in "1 out of 6" phage-treated calves and "6 out of 6" infected control calves. The Methods and the Fig. 1 legend assign 8 calves to each infected group, and the Results, using the same 49-day tissue culture and duplex qPCR at the single necropsy (16 weeks post-inoculation), report 8 MAP-positive calves in the MAP-only group and 2 in the phage-treated group, which is 8/8 and 2/8, with no calf reported as excluded. The direction and significance of the protective effect are unchanged (recomputed Fisher exact p = 0.015 for the abstract's counts, 0.0070 for the Results'), but the abstract understates the infected calves in each group as 6 rather than 8, and puts residual infection in the phage-treated group at 16.7% (1 calf) rather than 25.0% (2 calves).

What the paper reports

Abstract, the postmortem result:

Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 1 out of 6 animals in the phage-treated group testing positive for MAP colonized tissues compared to 6 out of 6 animals in the infected control group.

Results, "Oral Administration of Phages Prevented Establishment of MAP Infection in Intestinal Tissues", first paragraph:

To quantify the prevention of infection, MAP infection status was measured in intestinal tissues with high sensitivity (20) via duplex qPCR after samples were cultured for 49 d.

Within the MAP-only group, there were eight MAP-positive calves, each with 3 to 6 positive tissue types (Fig. 4A). However, in the phage-treated group, only two calves were MAP positive, with only one positive tissue type each.

Therefore, MP administration strongly reduced MAP-positive tissues (P value > 0.01) and provided almost complete protection against MAP infection.

Materials and Methods, "Study Design":

The first eight calves were designated the MAP-inoculated control group (MAP-only group). The next eight calves to arrive were designated the phage-treated MAP infection group (phage-treated group). The last three calves were designated the uninfected control (control).

Fig. 1 legend:

Newborn dairy calves (n = 19) were collected within 12 h of birth and randomly assigned into three groups: MAP-only (n = 8), phage-treated (n = 8), and control (n = 3).

Materials and Methods, "Inoculum":

Pellets were prepared per group (n = 8) per day.

Materials and Methods, "Necropsies and Tissue Cultures":

Calves were killed 16 wk postinoculation

Six tissue samples per calf were obtained: ileum, distal jejunum, ileocecal valve, and their associated lymph nodes.

The only "6 out of 8" in the paper concerns phage recovery, not MAP. Results, "Phages Were Persistent in the Small Intestine...":

MPs were detected in both the ileum and jejunum of 6 out of 8 of the phage-treated calves. The inability to recover MPs from the tissues of two calves was attributed to a documented sampling error with these specific samples

The only exclusions reported are for the blood assay. Results, first section:

Two samples were excluded from our results based on cell viability assessments (19).

The conclusion that rests on the tissue result, Discussion, first sentence:

This study demonstrated the ability of MPs to provide almost complete protection against MAP infection and the ability to eliminate fecal shedding by calves.

What the paper's own numbers show

Method. I took the group sizes from the Methods and the Fig. 1 legend and the numbers of MAP-positive calves from the Results text. I computed animal-level positivity for each group, with exact (Clopper–Pearson) 95% intervals, and did the same for the abstract's counts. For both versions I computed the risk difference, the relative reduction and a two-sided Fisher exact test on the 2×2 table. The paper itself reports only per-tissue chi-square tests, so the Fisher test is my addition and serves only to compare the two versions. I worked out which calves would have to be left out to turn the Results' counts into the abstract's. As a cross-check on the per-calf counts, I read the bar heights of the paper's Fig. 4A, which gives the number of positive animals for each tissue type. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Calves per group Methods; Fig. 1 legend; Inoculum MAP-only 8, phage-treated 8, control 3; total 19 8 + 8 + 3 = 19, consistent; 16 infected calves
Phage-treated calves with MAP-positive tissue Abstract 1 out of 6 16.7% (exact 95% CI 0.4–64.1%)
Phage-treated calves with MAP-positive tissue Results, tissue paragraph "only two calves were MAP positive" 2/8 = 25.0% (exact 95% CI 3.2–65.1%)
Infected-control calves with MAP-positive tissue Abstract 6 out of 6 100% (exact 95% CI 54.1–100%)
MAP-only calves with MAP-positive tissue Results, tissue paragraph "eight MAP-positive calves" 8/8 = 100% (exact 95% CI 63.1–100%)
Risk difference, phage minus control derived — abstract −83.3 points; Results −75.0 points
Relative reduction in positive calves derived — abstract 83.3%; Results 75.0%
Fisher exact test, two-sided derived (not in the paper) — abstract [[1, 5], [6, 0]] p = 0.0152; Results [[2, 6], [8, 0]] p = 0.0070
Calves that would have to be left out to get the abstract's counts derived — phage-treated: 1 positive and 1 negative; MAP-only: 2 positives
Most MAP-only animals positive in any one tissue type Fig. 4A (read from image) 8 (jejunum LN) consistent with 8 positive calves of 8
MAP-only positive tissues, summed over the six types Fig. 4A (read from image) 7, 7, 6, 7, 8, 3 38 over 8 calves = 4.75 per calf, consistent with "3 to 6 positive tissue types" each
Phage-treated positive tissues Fig. 4A (read from image) jejunum 1, ileocecal valve 1 2 tissues in 2 different tissue types, consistent with "two calves ... one positive tissue type each"

The Results text, the Methods group sizes and Fig. 4A agree with each other: 8 of 8 MAP-only calves and 2 of 8 phage-treated calves had at least one MAP-positive tissue. The abstract is the only place where the result appears as 1/6 and 6/6.

Three smaller inconsistencies turned up during the check. None affects the claim above or any conclusion.

Quantity Location Reported Recomputed / note
MAP-only fecal shedding "increased from 60 to 100%" Results, fecal shedding 60% 60% of 8 = 4.8 calves, not a whole number. Fig. 3 shows 3/8 = 37.5% (F57) or 5/8 = 62.5% (IS900) at 10 wk PI, and 8/8 at 16 wk
Significance of the tissue reduction Results, tissue paragraph "(P value > 0.01)" inconsistent with "strongly reduced"; Fig. 4A marks the significant tissue types with asterisks, so "<" was probably meant
qPCR positivity cutoff Methods "qPCR Procedure"; Fig. 4 legend; Fig. 3 legend "Ct values ≤ 37 considered positive"; "Ct < 37"; "Ct ... >37.0" the Fig. 3 legend reverses the direction of the cutoff

Figure

Calves with MAP-positive tissue at necropsy: the abstract's counts beside the Results and Methods counts

Figure 1. The abstract's sentence (left, orange) and the corresponding statements in the Results and Methods (right, blue), with the proportions each implies. The two panels refer to the same assay (49-day tissue culture followed by duplex F57/IS900 qPCR) and the same single necropsy at 16 weeks post-inoculation, and the paper reports no calf excluded from it. The numerators disagree (1 vs 2 positive phage-treated calves; 6 vs 8 positive infected controls), and so do the denominators (6 vs the 8 calves enrolled per group). The Fisher exact p-values at the bottom were recomputed from each pair of counts; the paper does not report them.

One square per calf, for each group and each version

Figure 2. One square per calf. Filled squares are calves with MAP-positive tissue, hollow squares are calves without, and dashed outlines are calves enrolled in the group (n = 8, Methods and Fig. 1 legend) that the abstract's count leaves out. Orange rows are the abstract's counts. Blue rows are the Results' counts set against the Methods group size. Going from the Results to the abstract takes away one positive and one negative calf in the phage-treated group and two positive calves in the infected control group.

Why it matters

The postmortem tissue result is the study's direct measure of infection, and the abstract is where most readers, and abstracting services, will take it from. As published, the abstract:

  1. Misstates the sample size. It describes two groups of 6 animals (12 in total) where the study enrolled two infected groups of 8 (16 in total). A reader using the abstract would take the study to be smaller than it was.
  2. Overstates the protective effect somewhat. It puts residual infection in the phage-treated group at 1 calf (16.7%) instead of 2 (25.0%), and the relative reduction at 83.3% instead of 75.0%. With groups this small, each calf moves the proportion by 12.5 points.
  3. Does not change the qualitative conclusion. Both versions show a large reduction in the same direction, and both are significant by Fisher's exact test (p = 0.015 and p = 0.0070). The exact 95% intervals for the phage-treated proportion are wide either way (0.4–64.1% and 3.2–65.1%). The title, the Significance statement, the Discussion and the Conclusion state the effect qualitatively and are not affected. The fecal-shedding result is separate and is also not affected.

So this is an error in a headline number, not in the study's main finding. It matters for anyone who quotes the abstract's figures, pools them in a meta-analysis or uses them to plan a follow-up study's sample size.

Explanations considered

Status

Suggested correction

  1. Change the abstract sentence to match the Results and the design, for example: "Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 2 out of 8 animals in the phage-treated group testing positive for MAP colonized tissues compared to 8 out of 8 animals in the infected control group." If the abstract's figures came from a different analysis or subset, state that analysis and its denominators in the Methods and Results instead.
  2. Minor: correct "(P value > 0.01)" in the tissue paragraph, presumably to "<"; give the MAP-only fecal-shedding proportions as counts out of 8 (for example 5/8 at 10 weeks by IS900, or 3/8 by F57), since 60% is not a possible proportion of 8 calves; and change "Ct value of >37.0" in the Fig. 3 legend to match the Methods ("≤ 37").
Recomputation script (python recompute.py.txt)
"""Recompute the animal-level tissue-culture result of PMC11331133
(Proc Natl Acad Sci U S A 2024, doi:10.1073/pnas.2318627121).

Run:  python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC:
  Abstract:            "only 1 out of 6 animals in the phage-treated group testing positive
                        for MAP colonized tissues compared to 6 out of 6 animals in the
                        infected control group"
  Results (tissues):   "Within the MAP-only group, there were eight MAP-positive calves ...
                        in the phage-treated group, only two calves were MAP positive"
  Methods, Fig. 1:     MAP-only n = 8, phage-treated n = 8, control n = 3 (n = 19)
  Results (feces):     MAP-only shedding "increased from 60 to 100%"
"""
import os
import warnings

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402
from scipy.stats import beta, fisher_exact  # noqa: E402

warnings.filterwarnings("ignore", message=".*findfont.*")
HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
GROUP_N = {"MAP-only": 8, "phage-treated": 8, "control": 3}       # Methods; Fig. 1 legend
TOTAL_STATED = 19                                                 # Methods; Fig. 1 legend
ABSTRACT = {"phage-treated": (1, 6), "MAP-only": (6, 6)}          # (positive, total)
RESULTS_POS = {"phage-treated": 2, "MAP-only": 8}                 # positive calves
RESULTS = {g: (RESULTS_POS[g], GROUP_N[g]) for g in RESULTS_POS}  # total = group size
SHEDDING_START_PCT = 60                                           # Results, fecal shedding

# Read from the bar heights of the paper's Fig. 4A image (pnas.2318627121fig04.jpg, PMC):
# number of MAP-positive animals per tissue type after 49-d culture + duplex qPCR.
TISSUES = ("ileum", "jejunum", "ileocecal valve", "ileum LN", "jejunum LN", "ileocecal valve LN")
FIG4A = {"MAP-only": (7, 7, 6, 7, 8, 3), "phage-treated": (0, 1, 1, 0, 0, 0)}
# Read from the paper's Fig. 3A/3B image (pnas.2318627121fig03.jpg): MAP-only calves with
# positive fecal culture at 10 wk PI (first week of active shedding) and 16 wk PI.
FIG3_MAP_ONLY = {"F57": {10: 3, 16: 8}, "IS900": {10: 5, 16: 8}}


def cp_ci(k, n, a=0.05):
    """Clopper-Pearson exact 95% interval for k/n."""
    lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
    hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
    return lo, hi


def summarise(label, d):
    kp, np_ = d["phage-treated"]
    kc, nc = d["MAP-only"]
    pp, pc = kp / np_, kc / nc
    table = [[kp, np_ - kp], [kc, nc - kc]]
    p = fisher_exact(table)[1]
    print(f"\n{label}")
    for g, (k, n) in (("phage-treated", (kp, np_)), ("MAP-only / infected control", (kc, nc))):
        lo, hi = cp_ci(k, n)
        print(f"  {g:28s} {k}/{n} = {100 * k / n:5.1f}%   exact 95% CI {100 * lo:4.1f}-{100 * hi:5.1f}%")
    print(f"  risk difference (phage - control) = {100 * (pp - pc):.1f} percentage points")
    print(f"  relative reduction 1 - p_phage/p_control = {100 * (1 - pp / pc):.1f}%")
    print(f"  2x2 table {table}: Fisher exact two-sided p = {p:.4f}")
    return p


print("=== Design (Methods, Fig. 1 legend)")
s = sum(GROUP_N.values())
print(f"Group sizes {GROUP_N}; sum = {s} (stated {TOTAL_STATED}); consistent = {s == TOTAL_STATED}")
print(f"Infected calves (MAP-only + phage-treated) = {GROUP_N['MAP-only'] + GROUP_N['phage-treated']}")

print("\n=== Animal-level tissue-culture positivity")
p_abs = summarise("Abstract", ABSTRACT)
p_res = summarise("Results, with the Methods group sizes", RESULTS)

print("\n=== What the abstract's figures would require")
for g in ("phage-treated", "MAP-only"):
    ka, na = ABSTRACT[g]
    kr, nr = RESULTS[g]
    drop_pos, drop_neg = kr - ka, (nr - kr) - (na - ka)
    print(f"{g}: from {kr}/{nr} to {ka}/{na} requires leaving out {drop_pos} positive "
          f"and {drop_neg} negative calves ({nr - na} calves in total)")
print("Calves in the abstract's two groups: 6 + 6 = 12, versus 8 + 8 = 16 infected calves enrolled")

print("\n=== Tissue-type reading (Fig. 4A, read from the image)")
for g in ("MAP-only", "phage-treated"):
    c = FIG4A[g]
    types = sum(1 for x in c if x > 0)
    print(f"{g}: positive tissue types {types}/6; most animals positive in one tissue type = "
          f"{max(c)}; positive tissues summed over types = {sum(c)}")
print(f"MAP-only: {sum(FIG4A['MAP-only'])} positive tissues over 8 calves = "
      f"{sum(FIG4A['MAP-only']) / 8:.2f} per calf (Results: 3 to 6 each)")
print("Phage-treated: 2 positive tissues in 2 different tissue types (jejunum, ileocecal valve),"
      " so a tissue-type count would be 2/6, not 1/6")

print("\n=== Side check: fecal shedding 'from 60 to 100%' in a group of 8")
print("Possible percentages of 8 calves:", ", ".join(f"{k}/8 = {100 * k / 8:.1f}%" for k in range(9)))
print(f"{SHEDDING_START_PCT}% of 8 = {SHEDDING_START_PCT / 100 * 8:.1f} calves (not a whole number); "
      f"nearest are 5/8 = 62.5% and 4/8 = 50.0%")
for gene, d in FIG3_MAP_ONLY.items():
    print(f"Fig. 3 {gene}: MAP-only calves shedding at 10 wk PI = {d[10]}/8 = {100 * d[10] / 8:.1f}%, "
          f"at 16 wk PI = {d[16]}/8 = {100 * d[16] / 8:.1f}%")

# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
    "text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)
    print("wrote", name + ".svg and", name + ".png")


# Figure 1: abstract vs Results/Methods, two text panels joined by a connector
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.9)
ax.axis("off")


def panel(x, w, colour, title, sub, lines):
    y0, h = 1.2, 3.25
    ax.add_patch(FancyBboxPatch((x, y0), w, h, boxstyle="round,pad=0,rounding_size=0.08",
                                fc="white", ec=colour, lw=1.5))
    ax.add_patch(Rectangle((x, y0 + h - 0.08), w, 0.08, fc=colour, ec="none"))
    ax.text(x + 0.15, y0 + h - 0.32, title, fontsize=12, fontweight="bold", va="center")
    ax.text(x + 0.15, y0 + h - 0.58, sub, fontsize=10, color=INK2, va="center")
    y = y0 + h - 0.9
    for kind, txt in lines:
        if kind == "q":
            ax.text(x + 0.15, y, txt, fontsize=10, color=INK, va="top", style="italic",
                    linespacing=1.35)
            y -= 0.2 * (txt.count("\n") + 1) + 0.2
        else:
            ax.text(x + 0.15, y, txt, fontsize=11.5, color=colour, va="top", fontweight="bold")
            y -= 0.36


panel(0.2, 3.0, ORANGE, "Abstract", "as reported",
      [("q", "“only 1 out of 6 animals in the\nphage-treated group testing positive\n"
             "for MAP colonized tissues compared\nto 6 out of 6 animals in the\n"
             "infected control group”"),
       ("n", "phage-treated: 1/6 = 16.7%"),
       ("n", "infected control: 6/6 = 100%")])
panel(4.0, 3.0, BLUE, "Results + Methods", "paper's own counts",
      [("q", "“there were eight MAP-positive\ncalves” (MAP-only group)\n"
             "“only two calves were MAP positive”\n(phage-treated group)\n"
             "Group sizes (Methods, Fig. 1):\nMAP-only n = 8, phage-treated n = 8"),
       ("n", "phage-treated: 2/8 = 25.0%"),
       ("n", "MAP-only: 8/8 = 100%")])

ax.annotate("", xy=(4.0, 2.85), xytext=(3.2, 2.85),
            arrowprops=dict(arrowstyle="<->", color=INK, lw=2, shrinkA=0, shrinkB=0))
ax.text(3.6, 3.05, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 0.6, "Same assay (49-d tissue culture + duplex qPCR), same single necropsy at 16 wk PI,\n"
        "no exclusions reported. Numerators differ (1 vs 2; 6 vs 8) and denominators differ (6 vs 8).\n"
        f"Both versions are significant: Fisher exact p = {p_abs:.3f} (abstract) vs {p_res:.4f} (Results).",
        ha="center", va="center", fontsize=10, color=INK2, linespacing=1.4)
ax.text(0.2, 4.68, "Calves with MAP-positive tissue at necropsy: abstract vs body of the paper",
        fontsize=12, fontweight="bold", va="center")
save(fig, "fig1")

# Figure 2: one square per calf, so numerator and denominator are both visible
rows = [
    ("Phage-treated, abstract", ABSTRACT["phage-treated"], ORANGE),
    ("Phage-treated, Results", RESULTS["phage-treated"], BLUE),
    ("Infected control, abstract", ABSTRACT["MAP-only"], ORANGE),
    ("MAP-only, Results", RESULTS["MAP-only"], BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
fig.subplots_adjust(left=0.30, right=0.80, top=0.80, bottom=0.17)
size, gap = 0.8, 0.2
for i, (lab, (k, n), col) in enumerate(rows):
    y = [4.0, 3.0, 1.4, 0.4][i]
    for j in range(8):
        if j < n:
            filled = j < k
            ax.add_patch(Rectangle((j, y - size / 2), size, size, fc=col if filled else "white",
                                   ec=col, lw=2))
        else:
            ax.add_patch(Rectangle((j, y - size / 2), size, size, fc="none", ec="#b9b8b3",
                                   lw=1.2, ls=(0, (2, 2))))
    ax.text(-0.3, y, lab, ha="right", va="center", fontsize=10.5)
    ax.text(8.2, y, f"{k}/{n} = {100 * k / n:.1f}%", ha="left", va="center", fontsize=10.5,
            fontweight="bold", color=INK)
ax.set_xlim(-0.1, 8.0)
ax.set_ylim(-0.2, 4.6)
ax.axis("off")
fig.text(0.02, 0.94, "One square per calf: MAP-positive tissue at necropsy", fontsize=12,
         fontweight="bold")
fig.text(0.02, 0.87, "Filled = positive, hollow = negative, dashed = calf enrolled (n = 8) "
         "but absent from the abstract's count", fontsize=10, color=INK2)
fig.legend(handles=[Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE),
                    Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE)],
           labels=["Reported in the abstract", "Paper's own Results counts, Methods group size"],
           loc="lower center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
save(fig, "fig2")

Title says the 2025–2030 guidelines raise water use; the paper's own results show lower water use

Proc Natl Acad Sci U S A (2026) · doi:10.1073/pnas.2604814123 · PMC13273247

Summary

This PNAS brief report compares diets that follow the 2025–2030 Dietary Guidelines for Americans (no ultraprocessed foods, protein raised to 1.2–1.6 g/kg) with the current Mean American Diet (MAD) on greenhouse gas (GHG) emissions, freshwater use, land use and nitrogen fertilizer use. Its title says the guidelines are "associated with higher land, water and nitrogen use, and greenhouse gas emissions", but its abstract, Results and Discussion all report that water use is "lower by 7 to 19% than the MAD". The paper's own Dataset S1 gives −6.6% to −19.5% for those scenarios (the text's range, to the nearest percent) and lower water use than the MAD in every scenario modelled, while the other three outcomes rise by up to 31.8%. The title thus states the wrong direction for one of the four outcomes it names, and a reader who sees only the title, as indexes, citations and news coverage present it, gets the water result backwards.

What the paper reports

Title:

The 2025–2030 Dietary Guidelines for Americans are associated with higher land, water and nitrogen use, and greenhouse gas emissions

Abstract, which states the comparator and the water result:

Here, we quantified the greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use of adherence to these new guidelines in comparison to the current Mean American Diet (MAD).

We find that although eliminating ultraprocessed foods from the MAD could yield substantial environmental benefits—given their large share of total intake and animal-based makeup—the recommended increase in animal-based proteins, will more than offset this environmental dividend in GHG, land use, and nitrogen fertilizer use by as much as 32%, but not in water use (i.e., water use is lower by 7 to 19% than the MAD).

Overall, adherence to the 2025–2030 Dietary Guidelines for Americans would confer net environmental harm across most key metrics relative to the prevailing U.S. diet.

Introduction, last paragraph:

We then quantified greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use for each diet and compared them against the MAD.

Results, second paragraph:

Our results indicate that GHG, land use, and nitrogen fertilizer dividends of eliminating UPFs from the MAD are outweighed by the impacts associated with increased protein intake (1.2 to 1.6 g/kg) by as much as 32%. In contrast, eliminating UPF and increasing protein levels to 2025–2030 DGAs reduced the water footprint compared to MAD by 7 to 19%.

Discussion, first paragraph:

For land use, GHG emissions, and nitrogen fertilizer use the net effects are adverse. For water use, the net effect is beneficial, but would be even more so were the avoidance of UPFs combined with a lower protein threshold, and the sourcing of protein preferentially from plants.

The paper has no tables. Fig. 1 panel B ("freshwater use (panel B)") plots the water results. All quotes above were checked verbatim against the PMC-archived article text.

What the paper's own numbers show

Method. Dataset S1 (pnas.2604814123.sd01.xlsx) has a "summary" sheet. Its rows 5–11 give annual per-capita GHG (kg CO2e), water (litres), land (m²·yr) and nitrogen fertilizer (g Nr), plus daily kcal and protein, for the MAD, the ultraprocessed-food (UPF) component of the MAD, and five UPF-free scenarios: 0.8, 1.2 and 1.6 g/kg, and 1.6 g/kg weighted toward plant or animal sources. I took the cached cell values and computed each scenario's percentage change against the MAD. I then compared the results with the ranges stated in the text. The sheet is identical to the one in the data file that the paper links under Data Availability. The sheet's own "Range" cells (K15:L15) pool GHG, land and nitrogen for the 1.2, 1.6 and 1.6 g/kg animal scenarios, and leave water out. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
Direction of water use, guideline diets vs MAD Title "higher … water … use" Lower in all 5 scenarios (−6.6% to −27.7%)
Water, 1.2 g/kg vs MAD Abstract; Results ¶2; Fig. 1B "lower by 7 to 19%" (upper end) −19.45%
Water, 1.6 g/kg vs MAD Abstract; Results ¶2; Fig. 1B "lower by 7 to 19%" (lower end) −6.58%
Water, 1.6 g/kg animal vs MAD Fig. 1B within 7–19% lower −13.14%
Water, 1.6 g/kg plant vs MAD Fig. 1B lower (not quantified) −24.92%
Water, 0.8 g/kg vs MAD Fig. 1B lower (not quantified) −27.73%
Net water effect Discussion ¶1 "beneficial" Consistent: every scenario is below the MAD
GHG, land, N fertilizer vs MAD, 1.2–1.6 g/kg scenarios Abstract; Results ¶2 "by as much as 32%" −2.5% (GHG, 1.2 g/kg) to +31.8% (land, 1.6 g/kg)
1.6 g/kg vs MAD: GHG / land / N Fig. 1A, C, D higher +21.5% / +31.8% / +7.7%
UPF share of MAD impacts (GHG, water, land, N) Results ¶2 "40 to 58%" 46.1%, 52.4%, 40.3%, 57.6%
Protein, 0.8 / 1.2 / 1.6 g/kg × 84 kg Results ¶1; Methods 67 / – / 134 g per day 67.2 / 100.8 / 134.4 g (modelled 67.3 / 100.8 / 134.3)

Every number in the abstract, Results and Discussion reproduces from Dataset S1. The title is the only statement that disagrees with the data, and it disagrees only on the direction of water use.

Figure

Title states higher water use; abstract, Results, Discussion and Dataset S1 all show lower water use than the MAD

Figure 1. The title clause (left, orange) lists water among the outcomes that are "higher" under the 2025–2030 guidelines. On the right are the three places where the text reports the water result, each against the Mean American Diet (MAD) (gray): the abstract ("lower by 7 to 19% than the MAD"), the second paragraph of Results ("reduced the water footprint compared to MAD by 7 to 19%") and the first paragraph of Discussion ("the net effect is beneficial"). Below them (blue) are the water changes recomputed from the summary sheet of Dataset S1 for the three scenarios in the guidelines' protein range that the text's 7–19% covers. The 1.6 g/kg plant scenario (−24.9%) and the 0.8 g/kg scenario (−27.7%) are also below the MAD and are not shown in this panel.

Percent change versus the MAD for the four title outcomes, recomputed from Dataset S1, with the paper's stated values

Figure 2. Change against the MAD for each of the four outcomes named in the title, in the four UPF-free scenarios at 1.2–1.6 g/kg protein (blue markers, recomputed from Dataset S1; the hollow marker is the plant-weighted 1.6 g/kg diet). The thin vertical line marks no change. Orange marks show what the paper states: the "lower by 7 to 19%" interval for water, the "as much as 32%" maximum for GHG, land and nitrogen (reached for land), and the direction that the title gives for water (arrow pointing to higher values). Land, GHG and nitrogen fertilizer rise in the 1.6 g/kg scenarios. Every water value is below zero, so none matches the direction the title gives.

Why it matters

The title is the paper's headline claim. It names four outcomes and says all four are higher under the guidelines. Against the comparator the paper states throughout (the Mean American Diet), one of those four, freshwater use, goes the other way in every scenario the paper models: it falls by 6.6–19.5% at the guidelines' protein levels and by up to 27.7% overall. The body of the paper reports this correctly and calls the net water effect "beneficial". The abstract's closing sentence is qualified accordingly ("net environmental harm across most key metrics").

The title is what bibliographic databases index, what citing papers and reference lists reproduce, and what headlines and social media posts pick up. A reader who relies on the title, which is the usual case for a widely covered policy paper, comes away with the water result reversed. One news item that links to this PMC record says the guidelines are linked to "16 percent more water consumption", alongside 33% higher emissions and 28% greater land use, and gives no comparator (Species Unite). In Dataset S1 those three figures coincide with the 1.6 g/kg animal versus 1.6 g/kg plant contrast (+32.6% GHG, +27.7% land, +15.7% water). That is a comparison between two guideline-range diets, not between the guidelines and the current diet.

The following results are not affected: the GHG, land-use and nitrogen-fertilizer increases (up to +31.8% against the MAD), the finding that plant-sourced protein does better than animal-sourced protein on every metric, the UPF shares, and the paper's main conclusion of net environmental harm on most metrics. The paper reports scenario point estimates without inferential statistics, so no test or interval changes.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Change the title to give the direction that the paper's results report for water, for example:

The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use and greenhouse gas emissions, but lower water use

or, if the title should keep to the adverse outcomes only:

The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use, and greenhouse gas emissions

Either wording matches the abstract ("but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"), the Results and the Discussion. No numbers, figures or conclusions in the body need to change.

Recomputation script (python recompute.py.txt)
"""Recompute the direction of each environmental outcome in PMC13273247
(Proc Natl Acad Sci U S A 2026; doi:10.1073/pnas.2604814123) from the paper's own
data, and redraw fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy and matplotlib)

Inputs are the cached cell values of the "summary" sheet (rows 5-11, columns F-K)
of Dataset S1 (pnas.2604814123.sd01.xlsx), the supporting-information workbook of
the article. The same sheet, with identical values, is in the data file the paper
links under Data Availability (NOVA+PROTEIN_DGA25-30_ver5.xlsx). Units are annual
per-capita totals, as in the paper's Fig. 1 (GHG kg CO2e, water litres, land m2-yr,
nitrogen fertilizer g Nr), plus daily kcal and protein (g).
Quoted text is copied verbatim from the PMC-archived article.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
METRICS = ["GHG", "Water", "Land", "N fertilizer"]
# Dataset S1, sheet "summary": GHG (F), water (G), land (H), N fertilizer (I), kcal (J), protein (K)
S1 = {
    "MAD":             [1620.3738784771374, 1884982.1709933979, 16238.053625377528, 48733.303782843548, 2500.3896000000004, 105.88118154726627],
    "UPF (in MAD)":    [747.61481329437038, 987787.62738541316, 6537.7528699819459, 28078.039550923815, 1636.7655999999999, 50.996616966708991],
    "0.8 g/kg":        [1253.3556489623356, 1362294.9058782607, 12576.133868282266, 47734.201604786555, 2499.6447058832632, 67.285004777165909],
    "1.2 g/kg":        [1579.418981096685,  1518270.9459631008, 16646.023128729423, 49705.242645896185, 2499.6957642603857, 100.84518830094257],
    "1.6 g/kg":        [1969.1244734977586, 1760953.6181536382, 21407.659406569564, 52485.30446813177,  2499.516060583017,  134.26708806627164],
    "1.6 g/kg PLANT":  [1497.7793450200566, 1415155.5062714668, 16597.693788935609, 43883.534702103032, 2499.6400527792202, 134.37530603776761],
    "1.6 g/kg ANIMAL": [1985.3613800760761, 1637370.2722960121, 21199.247774193638, 52751.029784245409, 2500.187619501251,  134.21656241614866],
}
SCEN = ["0.8 g/kg", "1.2 g/kg", "1.6 g/kg", "1.6 g/kg PLANT", "1.6 g/kg ANIMAL"]
# Scenarios the authors' own "Range" cells (summary!K15:L15) pool: 1.2, 1.6 and 1.6 ANIMAL
DGA = ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]
# What the paper states
REPORTED_WATER = (-19.0, -7.0)   # Abstract: "water use is lower by 7 to 19% than the MAD"
REPORTED_MAX_OTHER = 32.0        # Abstract/Results: "by as much as 32%" (GHG, land, N fertilizer)
REPORTED_UPF_SHARE = (40.0, 58.0)  # Results: "(40 to 58%)"
BODY_WEIGHT = 84.0               # Methods: average adult weight, kg

TITLE = ("The 2025–2030 Dietary Guidelines for Americans are associated with higher land, "
         "water and nitrogen use, and greenhouse gas emissions")
ABSTRACT_Q = "but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"
RESULTS_Q = "reduced the water footprint compared to MAD by 7 to 19%"
DISCUSSION_Q = "For water use, the net effect is beneficial"


def pc(a, b):
    return 100.0 * (a - b) / b


mad = np.array(S1["MAD"])
ch = {k: pc(np.array(S1[k][:4]), mad[:4]) for k in SCEN}

print("=== 1. Percent change versus the Mean American Diet (MAD), Dataset S1 summary sheet ===")
print(f"{'scenario':18s}" + "".join(f"{m:>14s}" for m in METRICS) + f"{'kcal/d':>9s}{'protein g/d':>13s}")
for k in ["MAD"] + SCEN:
    v = S1[k]
    row = "".join(f"{x:+13.2f}%" for x in ch[k]) if k != "MAD" else "".join(f"{'(baseline)':>14s}" for _ in METRICS)
    print(f"{k:18s}{row}{v[4]:9.1f}{v[5]:13.1f}")

print("\n=== 2. Water: the paper's stated range and the title's direction ===")
w_dga = [ch[k][1] for k in DGA]
print("Water change vs MAD, DGA-range scenarios (1.2, 1.6, 1.6 ANIMAL):", [f"{x:+.2f}%" for x in w_dga])
print(f"  reductions run from {-max(w_dga):.1f}% to {-min(w_dga):.1f}%  "
      f"(paper: 'lower by {-REPORTED_WATER[1]:.0f} to {-REPORTED_WATER[0]:.0f}%') -> "
      f"matches after rounding: {round(-max(w_dga)) == -REPORTED_WATER[1] and round(-min(w_dga)) == -REPORTED_WATER[0]}")
w_all = {k: ch[k][1] for k in SCEN}
print("Water change vs MAD, every modelled scenario:", {k: f"{v:+.1f}%" for k, v in w_all.items()})
print("  any scenario with HIGHER water use than the MAD?", any(v > 0 for v in w_all.values()))
print("  smallest reduction:", f"{max(w_all.values()):+.1f}%", " largest reduction:", f"{min(w_all.values()):+.1f}%")

print("\n=== 3. GHG, land and N fertilizer (the other three title outcomes) ===")
other = [(k, METRICS[j], ch[k][j]) for k in DGA for j in (0, 2, 3)]
lo = min(other, key=lambda t: t[2]); hi = max(other, key=lambda t: t[2])
print(f"  range over 1.2, 1.6, 1.6 ANIMAL: {lo[2]:+.1f}% ({lo[1]}, {lo[0]}) to {hi[2]:+.1f}% ({hi[1]}, {hi[0]})")
print(f"  paper: 'as much as {REPORTED_MAX_OTHER:.0f}%' -> max rounds to {round(hi[2])}: {round(hi[2]) == REPORTED_MAX_OTHER}")
for k in DGA:
    print(f"  {k:16s} GHG {ch[k][0]:+.1f}%  land {ch[k][2]:+.1f}%  N {ch[k][3]:+.1f}%  | water {ch[k][1]:+.1f}%")

print("\n=== 4. Other numbers in the text ===")
share = [100 * S1["UPF (in MAD)"][j] / mad[j] for j in range(4)]
print("UPF share of MAD impacts (GHG, water, land, N):", [f"{x:.1f}%" for x in share],
      f" range {min(share):.1f}-{max(share):.1f}% (paper: 40 to 58%)")
print(f"UPF share of MAD calories: {100*S1['UPF (in MAD)'][4]/mad[4]:.1f}% (paper: 'over 50%')")
print("Protein targets 0.8/1.2/1.6 g/kg x 84 kg =", [round(p * BODY_WEIGHT, 1) for p in (0.8, 1.2, 1.6)],
      " modelled:", [round(S1[k][5], 1) for k in ("0.8 g/kg", "1.2 g/kg", "1.6 g/kg")])
print(f"MAD protein: {mad[5]:.1f} g/d = {mad[5]/BODY_WEIGHT:.2f} g/kg (paper: 'MAD protein intake level is 1.2 g/kg')")

print("\n=== 5. Alternative comparators examined as explanations for the title ===")
for k in ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]:
    print(f"  water, {k:16s} vs 0.8 g/kg UPF-free diet: {pc(S1[k][1], S1['0.8 g/kg'][1]):+.1f}%")
print(f"  water, 1.6 g/kg ANIMAL vs 1.6 g/kg PLANT: {pc(S1['1.6 g/kg ANIMAL'][1], S1['1.6 g/kg PLANT'][1]):+.1f}%")
print("  1.6 g/kg ANIMAL vs 1.6 g/kg PLANT, (GHG, water, land, N):",
      [f"{pc(S1['1.6 g/kg ANIMAL'][j], S1['1.6 g/kg PLANT'][j]):+.1f}%" for j in range(4)])
print("  water per gram of daily protein vs MAD:",
      {k: f"{pc(S1[k][1]/S1[k][5], mad[1]/mad[5]):+.1f}%" for k in SCEN})
print("  water per 1,000 kcal vs MAD (all diets ~2,500 kcal):",
      {k: f"{pc(S1[k][1]/S1[k][4], mad[1]/mad[4]):+.1f}%" for k in SCEN})

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG,
                     "savefig.facecolor": BG})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: title vs body text (text-vs-text contradiction)
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")


def panel(x, y, w, h, head, body, edge, lw=2, ls="-", body_color=INK, size=10):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.06",
                                fc="white", ec=edge, lw=lw, ls=ls))
    ax.text(x + 0.12, y + h - 0.12, head, ha="left", va="top", fontsize=10, color=INK2, weight="bold")
    ax.text(x + 0.12, y + h - 0.42, body, ha="left", va="top", fontsize=size, color=body_color,
            linespacing=1.35)


panel(0.15, 2.55, 2.5, 1.6, "Title",
      "\u201c\u2026are associated with\nhigher land, water and\nnitrogen use, and\ngreenhouse gas emissions\u201d",
      ORANGE, lw=2.5)
ax.text(0.27, 2.65, "Water: higher", color=ORANGE, fontsize=11, weight="bold", ha="left", va="bottom")

wd = {k: ch[k][1] for k in DGA}
fmt = lambda x: f"{x:+.1f}%".replace("-", "\u2212")
rx, rw = 4.0, 3.05
panel(rx, 3.62, rw, 0.98, "Abstract", "\u201c\u2026but not in water use (i.e., water\nuse is lower by 7 to 19% than the MAD)\u201d",
      INK2, lw=1.2, size=9.5)
panel(rx, 2.66, rw, 0.86, "Results, 2nd paragraph", "\u201c\u2026reduced the water footprint\ncompared to MAD by 7 to 19%\u201d",
      INK2, lw=1.2, size=9.5)
panel(rx, 1.90, rw, 0.66, "Discussion, 1st paragraph", "\u201cFor water use, the net effect is beneficial\u201d",
      INK2, lw=1.2, size=9.5)
panel(rx, 0.55, rw, 1.25, "Recomputed from Dataset S1", "", BLUE, lw=2.5)
for i, k in enumerate(DGA):
    yy = 1.38 - 0.19 * i
    ax.text(rx + 0.12, yy, k.replace("ANIMAL", "animal"), color=BLUE, fontsize=10, ha="left", va="top")
    ax.text(rx + 2.0, yy, fmt(wd[k]), color=BLUE, fontsize=10, ha="right", va="top")
ax.text(rx + 0.12, 0.64, "Water: lower in all scenarios", color=BLUE, fontsize=10, weight="bold",
        ha="left", va="bottom")
# bracket grouping the right-hand panels
bx = rx - 0.12
ax.plot([bx, bx], [0.55, 4.60], color=INK2, lw=1.2)
for yy in (0.55, 4.60):
    ax.plot([bx, bx + 0.07], [yy, yy], color=INK2, lw=1.2)
ax.add_patch(FancyArrowPatch((2.7, 3.35), (bx - 0.05, 2.6), arrowstyle="<|-|>", mutation_scale=14,
                             color=INK, lw=2, connectionstyle="arc3,rad=0.0"))
ax.text(3.27, 3.55, "Opposite\ndirection", ha="center", va="bottom", fontsize=10.5, weight="bold", color=INK)
ax.text(3.27, 2.55, "Paper's stated\ncomparator:\nMean American\nDiet (MAD)", ha="center", va="top", fontsize=9.5,
        color=INK2, linespacing=1.3)
ax.text(0.15, 4.72, "Freshwater use under the 2025\u20132030 guideline diets, relative to the MAD",
        fontsize=11, weight="bold", ha="left", va="center")
ax.text(0.15, 2.2, "Orange: the paper's headline\nclaim (title).\nGray: the paper's own text.\n"
        "Blue: recomputed from the\npaper's Dataset S1\n(summary sheet).", fontsize=9.5, color=INK2,
        ha="left", va="top", linespacing=1.35)
save(fig, "fig1")

# Figure 2: percent change vs MAD for all four title outcomes
fig, ax = plt.subplots(figsize=(7.2, 4.1))
fig.subplots_adjust(left=0.17, right=0.97, top=0.80, bottom=0.14)
order = ["GHG", "Water", "Land", "N fertilizer"]
ypos = {m: i for i, m in enumerate(reversed(order))}
marks = {"1.2 g/kg": "o", "1.6 g/kg": "s", "1.6 g/kg ANIMAL": "^", "1.6 g/kg PLANT": "v"}
offs = {"1.2 g/kg": 0.12, "1.6 g/kg": 0.04, "1.6 g/kg ANIMAL": -0.04, "1.6 g/kg PLANT": -0.12}
ax.axvline(0, color=INK2, lw=1)
for m in order:
    j = METRICS.index(m); y = ypos[m]
    vals = [ch[k][j] for k in marks]
    ax.plot([min(vals), max(vals)], [y, y], color=BLUE, lw=2, alpha=0.35, zorder=1)
    for k, mk in marks.items():
        filled = k != "1.6 g/kg PLANT"
        ax.plot(ch[k][j], y + offs[k], mk, ms=8.5, mec=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
# reported values
yw = ypos["Water"] + 0.33
ax.plot(REPORTED_WATER, [yw, yw], color=ORANGE, lw=2)
for x in REPORTED_WATER:
    ax.plot([x, x], [yw - 0.07, yw + 0.07], color=ORANGE, lw=2)
ax.text(np.mean(REPORTED_WATER), yw + 0.1, "Abstract/Results: “lower by 7 to 19%”", color=INK,
        fontsize=9.5, ha="center", va="bottom")
ax.annotate("", xy=(13, yw - 0.02), xytext=(0.6, yw - 0.02),
            arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=13))
ax.text(13.8, yw - 0.02, "Title: “higher … water”", color=INK, fontsize=9.5, ha="left", va="center")
yl = ypos["Land"] + 0.33
ax.plot(REPORTED_MAX_OTHER, yl, "D", ms=8.5, color=ORANGE, zorder=3)
ax.text(REPORTED_MAX_OTHER - 1.2, yl + 0.02, "Text: “as much as 32%”", color=INK, fontsize=9.5,
        ha="right", va="center")
ax.set_yticks([ypos[m] for m in order]); ax.set_yticklabels(order, fontsize=10.5, color=INK)
ax.set_ylim(-0.55, 3.62)
ax.set_xlim(-32, 40)
ax.set_xlabel("Change versus the Mean American Diet (%)", fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], ls="", marker=mk, ms=8, mec=BLUE, mew=2, mfc=BLUE if k != "1.6 g/kg PLANT" else BG,
            label=k.replace("ANIMAL", "animal").replace("PLANT", "plant")) for k, mk in marks.items()]
h += [Line2D([], [], color=ORANGE, lw=2, marker="D", ms=7, label="reported in the paper")]
leg = ax.legend(handles=h, ncol=5, loc="lower left", bbox_to_anchor=(-0.2, 1.01), frameon=False,
                fontsize=9, handletextpad=0.3, columnspacing=0.9)
fig.text(0.02, 0.965, "Blue: recomputed from Dataset S1. Orange: values and direction stated in the paper.",
         fontsize=10, color=INK2, ha="left", va="top")
save(fig, "fig2")
print("\nFigures written:", [os.path.join(HERE, f"fig{i}.{e}") for i in (1, 2) for e in ("svg", "png")])

The abstract names the inhibited RNA strand as negative, but the paper's own data show positive

Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1805195115 · PMC6176606

Summary

The paper studies a dengue virus genome carrying an NS3-G459L mutation that blocks an internal self-cleavage and acts as a trans-dominant inhibitor of wild-type virus in the same cell. The abstract states that this variant "dominantly inhibited negative-strand RNA synthesis." The Results section, using strand-specific RT-qPCR (Fig. 5 F and G), reports the opposite direction: positive-strand synthesis was strongly inhibited, while negative-strand synthesis was allowed (negative-strand RNA was significantly greater than the polymerase-dead control). The Results conclusion and the Discussion both state that positive-strand synthesis is the inhibited step. The single word "negative-strand" in the abstract therefore reverses the direction of the paper's key mechanistic result, and should read "positive-strand."

What the paper reports

an NS3-G459L variant that is incapable of cleavage at the internal NS3 junction dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell

— Abstract

To identify whether the dominant NS3-G459L mutant genome inhibited negative-strand synthesis (and therefore positive-strand synthesis as well) or just positive-strand synthesis, we performed strand-specific RT-qPCR to measure both strands of WT and GDD replicons (Fig. 5 F and G). ... The amount of positive-strand replicon RNA detected when cotransfected with the NS3-G459L dominant mutant was indistinguishable from that detected for the GDD replicon alone, indicating that positive-strand synthesis was strongly inhibited (Fig. 5F). By contrast, the amount of negative-strand replicon RNA was significantly greater than that for the GDD replicon alone (Fig. 5G), indicating that the NS3-G459L dominant mutant allows negative-strand synthesis and therefore specifically inhibits positive-strand synthesis.

— Results, "NS3-G459L Is a Dominant Inhibitor of WT Viral Growth"

Most importantly, viral genomes that abrogate this self-processing event trans-dominantly inhibit positive-strand RNA synthesis of WT genomes present in the same cell

— Discussion, "NS3 Internal Cleavage"

(F) Positive-strand replicon RNA was detected by strand-specific RT-qPCR. (G) Negative-strand replicon RNA was detected by strand-specific RT-qPCR (n = 2).

— Legend to Fig. 5

What the paper's own numbers show

There is no arithmetic to redo. The discrepancy is a reversed qualitative direction between two parts of the same paper describing one experiment: a strand-specific RT-qPCR of the luciferase replicon cotransfected with the NS3-G459L variant, measured at a single time point and normalized to the wild-type replicon alone, with the polymerase-dead GDD replicon as the no-synthesis reference. The two panels of that one experiment (Fig. 5F, positive strand; Fig. 5G, negative strand) are each reported relative to the GDD control. Taking those two relations as stated fixes which strand is inhibited.

quantity location reported recomputed (from the paper's own numbers)
Strand whose synthesis G459L inhibits Abstract negative-strand positive-strand
Positive-strand replicon RNA, G459L vs GDD-dead Results / Fig. 5F = GDD → positive synthesis strongly inhibited positive-strand inhibited (consistent with body)
Negative-strand replicon RNA, G459L vs GDD-dead (n = 2) Results / Fig. 5G > GDD → negative synthesis allowed negative-strand allowed, not inhibited
Strand inhibited (stated) Results conclusion positive-strand positive-strand
Strand inhibited (stated) Discussion positive-strand positive-strand

Four statements in the body (Fig. 5F, Fig. 5G, the Results conclusion, and the Discussion) place the inhibition on the positive strand; one statement in the abstract places it on the negative strand. recompute.py.txt encodes the two reported relations and prints VERDICT: claim CONFIRMED.

Figure

Two-panel diagram: the abstract sentence naming negative-strand synthesis as inhibited, versus the Results/Discussion showing positive-strand synthesis inhibited and negative-strand allowed, joined by a connector labeled "direction reversed."

The abstract (left, orange, as reported) states that negative-strand synthesis is inhibited. The Results and Discussion (right, blue, from the paper's own numbers) state that positive-strand synthesis is inhibited while negative-strand synthesis is allowed, based on the strand-specific RT-qPCR in Fig. 5F/G of the same cotransfection experiment. The connector names the conflict: the two descriptions assign the inhibition to opposite strands of one assay. Colors are reinforced by the panel labels and legend so the distinction does not rely on color alone.

Why it matters

Which RNA strand's synthesis is blocked is the mechanistic heart of this result. A reader using the abstract alone would conclude that the dominant-negative genome blocks negative-strand (template) synthesis, an early step of replication. The paper's data support the reverse: negative-strand synthesis proceeds and positive-strand (progeny genome) synthesis is specifically inhibited. Flavivirus and antiviral-mechanism researchers who cite the abstract without reading the Results could adopt an inverted picture of where in the replication cycle the trans-dominant block acts. The paper's broader conclusion — that abrogating the internal NS3 self-cleavage yields trans-dominant inhibition of viral replication, supporting the proposed antiviral strategy — is unaffected in direction, magnitude, or significance, because it does not depend on which strand is affected. The error is confined to one word of one abstract sentence and is contradicted correctly elsewhere in the same paper.

Explanations considered

Status

Suggested correction

In the abstract, change "dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell" to "dominantly inhibited positive-strand RNA synthesis of wild-type virus present in the same cell," making the abstract agree with the Results (Fig. 5 F and G), the Results conclusion, and the Discussion. No figure, table, or numeric value needs to change.

Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Recompute / re-derivation for the documented error in PMC6176606
(Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1805195115).

The error is a reversed qualitative direction, not an arithmetic slip:
the Abstract names one RNA strand (negative) as the one whose synthesis the
NS3-G459L trans-dominant variant inhibits, while the Results (strand-specific
RT-qPCR, Fig. 5 F and G) and the Discussion both report the opposite --
positive-strand synthesis is inhibited and negative-strand synthesis is allowed.

Running this file:
  python recompute.py.txt
prints every value used in the write-up and regenerates fig1.svg / fig1.png
in the folder that holds this script.

No data are taken from outside the paper. The RT-qPCR panels (Fig. 5F, 5G) are
reported in the paper only qualitatively relative to the polymerase-dead GDD
control ("indistinguishable from" / "significantly greater than"); those two
relations are the inputs, encoded verbatim below.
"""

import os

# ----------------------------------------------------------------------
# Inputs: the paper's own statements (strand-specific RT-qPCR, one assay,
# 26 h, normalized to the WT replicon alone). GDD = polymerase-dead control.
# ----------------------------------------------------------------------
statements = {
    "abstract": {
        "text": ("an NS3-G459L variant that is incapable of cleavage at the "
                 "internal NS3 junction dominantly inhibited NEGATIVE-strand "
                 "RNA synthesis of wild-type virus present in the same cell"),
        "inhibited_strand": "negative",
    },
    "results_Fig5F_positive": {
        "observation": "positive-strand RNA (G459L) == GDD-dead control",
        "meaning": "positive-strand synthesis strongly INHIBITED",
        "inhibited": "positive",
    },
    "results_Fig5G_negative": {
        "observation": "negative-strand RNA (G459L) > GDD-dead control",
        "meaning": "negative-strand synthesis ALLOWED (not inhibited)",
        "allowed": "negative",
        "n": 2,
    },
    "results_conclusion": {
        "text": ("the NS3-G459L dominant mutant allows negative-strand "
                 "synthesis and therefore specifically inhibits "
                 "positive-strand synthesis"),
        "inhibited_strand": "positive",
        "allowed_strand": "negative",
    },
    "discussion": {
        "text": ("viral genomes that abrogate this self-processing event "
                 "trans-dominantly inhibit positive-strand RNA synthesis of "
                 "WT genomes present in the same cell"),
        "inhibited_strand": "positive",
    },
}

# ----------------------------------------------------------------------
# Re-derivation
# ----------------------------------------------------------------------
body_inhibited = "positive"   # Fig. 5F + Results conclusion + Discussion
body_allowed = "negative"     # Fig. 5G + Results conclusion
abstract_inhibited = statements["abstract"]["inhibited_strand"]  # "negative"

contradiction = (abstract_inhibited != body_inhibited)
reversed_direction = (abstract_inhibited == body_allowed)


def main():
    print("Source: PMC6176606, doi:10.1073/pnas.1805195115")
    print("Error type: abstract-vs-body contradiction (reversed strand direction)")
    print("-" * 68)
    print("Fig. 5F positive strand : G459L == GDD(dead)  -> positive synthesis INHIBITED")
    print("Fig. 5G negative strand : G459L  > GDD(dead)  -> negative synthesis ALLOWED (n=2)")
    print()
    print(f"BODY: strand inhibited  = {body_inhibited!r}")
    print(f"BODY: strand allowed    = {body_allowed!r}")
    print(f"ABSTRACT: strand stated inhibited = {abstract_inhibited!r}")
    print()
    print(f"Abstract contradicts body?                              {contradiction}")
    print(f"Direction reversed (abstract names ALLOWED strand as    {reversed_direction}")
    print(f"  the inhibited one)?")
    print("-" * 68)

    # Table mirrored in the write-up
    rows = [
        ("Strand G459L inhibits", "Abstract",
         "negative-strand", "positive-strand"),
        ("Positive-strand RNA, G459L vs GDD-dead", "Results / Fig. 5F",
         "= GDD => positive INHIBITED", "positive INHIBITED (consistent)"),
        ("Negative-strand RNA, G459L vs GDD-dead (n=2)", "Results / Fig. 5G",
         "> GDD => negative ALLOWED", "negative ALLOWED, not inhibited"),
        ("Strand inhibited (stated)", "Results conclusion",
         "positive-strand", "positive-strand"),
        ("Strand inhibited (stated)", "Discussion",
         "positive-strand", "positive-strand"),
    ]
    w = [44, 20, 30, 32]
    hdr = ("quantity", "location", "reported", "recomputed")
    print(" | ".join(h.ljust(wi) for h, wi in zip(hdr, w)))
    print("-+-".join("-" * wi for wi in w))
    for r in rows:
        print(" | ".join(str(c).ljust(wi) for c, wi in zip(r, w)))
    print("-" * 68)

    # Innocent explanations checked
    checks = {
        "Different experiment/population?":
            "No - same strand-specific RT-qPCR, same G459L cotransfection.",
        "Different time/normalization/units?":
            "No - one assay (26 h, normalized to WT-alone); 5F & 5G are the two strands.",
        "Inhibiting + strand loosely lowers - strand?":
            "No - 5G shows negative SIGNIFICANTLY GREATER than GDD; text says ALLOWED.",
        "Logical framing rescues abstract?":
            "No - Results pose 'inhibit negative (hence both) OR just positive' and answer 'just positive'.",
        "One-sided/adjusted test / stated convention?":
            "N/A - reversed qualitative direction, not a statistical threshold.",
        "Correction/erratum?":
            "No Crossref/PMC/EuropePMC notice; no PubPeer thread.",
    }
    for q, a in checks.items():
        print(f"- {q} {a}")
    print("-" * 68)

    verdict = "CONFIRMED" if (contradiction and reversed_direction) else "NOT CONFIRMED"
    print(f"VERDICT: claim {verdict}")

    make_figure()


# ----------------------------------------------------------------------
# Figure: two text panels (Abstract vs Results/Discussion) joined by an
# annotated connector naming the conflict.
# ----------------------------------------------------------------------
def make_figure():
    import matplotlib
    matplotlib.use("Agg")
    import matplotlib.pyplot as plt
    from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

    BG = "#fcfcfb"
    INK = "#0b0b0b"
    SUB = "#52514e"
    ORANGE = "#eb6834"   # the paper's reported value (abstract)
    BLUE = "#2a78d6"     # value recomputed from the paper's own numbers
    GRID = "#d8d7d4"

    here = os.path.dirname(os.path.abspath(__file__))

    fig, ax = plt.subplots(figsize=(7.2, 4.5))
    fig.patch.set_facecolor(BG)
    ax.set_facecolor(BG)
    ax.set_xlim(0, 100)
    ax.set_ylim(0, 100)
    ax.axis("off")

    # Title
    ax.text(50, 96, "Abstract names the allowed strand as the inhibited one",
            ha="center", va="top", fontsize=13, fontweight="bold", color=INK)
    ax.text(50, 90.5,
            "PMC6176606 - strand-specific RT-qPCR (Fig. 5 F and G)",
            ha="center", va="top", fontsize=10, color=SUB)

    # ---- Left panel: Abstract (reported, orange) ----
    lx, lw = 2.5, 38
    ly, lh = 28, 50
    ax.add_patch(FancyBboxPatch((lx, ly), lw, lh,
                 boxstyle="round,pad=0.6,rounding_size=2.2",
                 linewidth=2, edgecolor=ORANGE, facecolor="#fdece4"))
    ax.text(lx + lw / 2, ly + lh - 4, "ABSTRACT", ha="center", va="top",
            fontsize=11, fontweight="bold", color=ORANGE)
    ax.text(lx + lw / 2, ly + lh - 11,
            "\"...dominantly inhibited\n"
            "NEGATIVE-strand RNA\n"
            "synthesis of wild-type virus\n"
            "present in the same cell\"",
            ha="center", va="top", fontsize=9.5, color=INK, style="italic")
    ax.text(lx + lw / 2, ly + 8,
            "Reads: NEGATIVE-strand\nsynthesis is inhibited",
            ha="center", va="center", fontsize=10, color=ORANGE,
            fontweight="bold")

    # ---- Right panel: Results / Discussion (recomputed, blue) ----
    rx, rw = 59.5, 38
    ax.add_patch(FancyBboxPatch((rx, ly), rw, lh,
                 boxstyle="round,pad=0.6,rounding_size=2.2",
                 linewidth=2, edgecolor=BLUE, facecolor="#e7f0fb"))
    ax.text(rx + rw / 2, ly + lh - 4, "RESULTS / DISCUSSION",
            ha="center", va="top", fontsize=11, fontweight="bold", color=BLUE)
    ax.text(rx + rw / 2, ly + lh - 11,
            "Fig. 5F  positive = GDD-dead\n"
            "   -> positive INHIBITED\n"
            "Fig. 5G  negative > GDD-dead (n=2)\n"
            "   -> negative ALLOWED",
            ha="center", va="top", fontsize=9, color=INK)
    ax.text(rx + rw / 2, ly + 8,
            "States: POSITIVE-strand\nsynthesis is inhibited",
            ha="center", va="center", fontsize=10, color=BLUE,
            fontweight="bold")

    # ---- Connector naming the conflict (in the central gap only) ----
    gap_l, gap_r = lx + lw, rx            # 40.5 -> 59.5
    cy = ly + lh / 2
    con = FancyArrowPatch((gap_l + 0.5, cy), (gap_r - 0.5, cy),
                          arrowstyle="<|-|>", mutation_scale=14,
                          linewidth=2, color=INK,
                          connectionstyle="arc3,rad=0.0")
    ax.add_patch(con)
    mid = (gap_l + gap_r) / 2
    ax.text(mid, cy + 6, "DIRECTION", ha="center", va="bottom",
            fontsize=9.5, fontweight="bold", color=INK)
    ax.text(mid, cy + 2.2, "REVERSED", ha="center", va="bottom",
            fontsize=9.5, fontweight="bold", color=INK)
    ax.text(mid, cy - 4, "same assay,", ha="center", va="top",
            fontsize=8.5, color=SUB)
    ax.text(mid, cy - 7.5, "opposite strand", ha="center", va="top",
            fontsize=8.5, color=SUB)

    # ---- Footer legend (color never the sole signal) ----
    ax.plot([8, 12], [12, 12], color=ORANGE, linewidth=2)
    ax.text(13, 12, "as reported in the paper (abstract)",
            ha="left", va="center", fontsize=9.5, color=SUB)
    ax.plot([8, 12], [6, 6], color=BLUE, linewidth=2)
    ax.text(13, 6, "from the paper's own numbers (Results / Discussion)",
            ha="left", va="center", fontsize=9.5, color=SUB)

    # faint recessive grid baseline (single horizontal)
    for gy in (22,):
        ax.axhline(gy, color=GRID, linewidth=0.8, zorder=0)

    fig.tight_layout()
    svg = os.path.join(here, "fig1.svg")
    png = os.path.join(here, "fig1.png")
    fig.savefig(svg, facecolor=BG)
    fig.savefig(png, dpi=200, facecolor=BG)
    plt.close(fig)
    print(f"wrote {svg}")
    print(f"wrote {png}")


if __name__ == "__main__":
    main()

Significance statement says tracer chromosome goes "exclusively" with X; Results report 28 of 42 cells

Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1809583115 · PMC6310853

Summary

The paper's Significance statement says that male Ts65Dn translocation mice carrying a maternally derived supernumerary chromosome place this "tracer" chromosome "exclusively" in the haploid set that contains the X chromosome. The only experiment behind that sentence is in Results (Fig. 4 H–L). It reports the tracer in the X-containing hemisphere in "the majority of cases": n = 28/42 cells, P = 0.022. That is 66.7% (exact 95% CI 50.5% to 80.4%), with 14 of 42 cells (33.3%) not counted on the X side. The reported P reproduces as a one-sided binomial test against 0.5 (0.0218; two-sided 0.0436). The paper's own Fig. 4L also shows some tracer markers below the chromosome 16 midpoint, which the legend defines as the Y-containing hemisphere. The Significance statement thus describes as all-or-none a modest majority that only just reaches significance, in the experiment that the Results present as the direct test of the parental-origin model. The paper's main finding, that homologs are kept apart in mitosis, rests on other experiments and is not affected.

What the paper reports

Male translocation mice with a maternal-derived supernumerary chromosome display the tracer chromosome exclusively to the haploid set containing the X chromosome.

— Abstract, Significance statement

Using 3D chromosome imaging of a translocation mouse with a supernumerary chromosome, we found that this maternally derived chromosome is positioned by parental origin. These data, taken together, support the identity of haploid sets by parental origin.

— Abstract (main)

In the majority of cases we examined, the hemisphere containing the translocated chromosome 1716 also contained the maternally derived X chromosome (n = 28/42 cells, P = 0.022; Fig. 4 H–L and Movie S7).

— Results › In Vitro and in Vivo Segregation of Mouse Chromosomes, paragraph 2 (the plain-text archive renders 1716 as "1716")

To identify which X-, or Y-containing haploid set was most likely associated with the translocation chromosome, we first determined the approximate boundary between the haploid sets by finding the midpoint of the pair of homologous chromosome 16. We then mapped the position of the translocated chromosome (Fig. 4L).

— Same paragraph, immediately before

A maternally derived supernumerary chromosome segregates with the X chromosome.

— Fig. 4 legend, title sentence

The X- or Y-containing hemisphere domains are defined as being above/below the plus sign (+), respectively.

— Fig. 4 legend, panel L (n = 42)

Movie S7. The translocated chromosome in the Ts65Dn MEFs was consistently detected in the same hemisphere as the maternal-derived X chromosome.

— SI Appendix, movie legends (the Results cite Movie S7 alongside n = 28/42)

Center of mass calculations were used for hemisphere localization; however, chromosomes that were positioned with the majority of their staining along the boundary of the hemispheres, due to the limits of resolution, were scored to support the null hypothesis, which goes against our model, in the calculation of the one-tailed P value which gives rise to an underestimation of statistical significance.

— Materials and Methods › Statistical Analysis

Although our data support the paradigm (13, 14, 47) that haploid sets are derived by parental origin, there are limitations to our analysis.

— Discussion, paragraph 3

The same paragraph adds that in the reciprocal (paternal) cross the tracer "localized to the boundary between two nuclear hemispheres, making it difficult to determine its association to either X- or Y-containing hemispheres (SI Appendix, Fig. S5 C–I)."

What the paper's own numbers show

Method. The Methods state that hemisphere assignments were tested with a binomial distribution against p = 0.5, using a one-tailed P value. I recomputed every count that the paper reports with a P value, as one-sided P(X ≥ x | n, 0.5), as two-sided exact P, and as the single-term binomial probability b(x; n, 0.5) that the Methods formula writes out. For the tracer result I computed the exact (Clopper–Pearson) 95% CI for the proportion. As a separate, approximate check, I read the positions of the tracer circles and the chromosome 16 midpoint markers ("+") off the published Fig. 4 image (757 × 449 px) by pixel colour. Each circle was classed as above, within ±3 axis units of, or below its local midpoint. All numbers below are printed by recompute.py.txt.

Quantity Location Reported Recomputed
Tracer 1716 in the maternal-X hemisphere Significance statement "exclusively", i.e. 42/42 (100%) —
Same result Results, mouse section para 2; Fig. 4 H–L n = 28/42 cells, P = 0.022, "the majority of cases" 66.7%; exact 95% CI 50.5% to 80.4% (excludes 100%); one-sided P = 0.0218, two-sided P = 0.0436
Cells not counted in the X hemisphere Same — 14/42 (33.3%)
Margin over the significance threshold Same — smallest count out of 42 with one-sided P < 0.05 is 27; the reported 28 is one cell above it
Compatibility with a near-exclusive effect Same — if the true share were 90%, P(≤ 28 of 42) = 3.5 × 10⁻⁵; if 95%, 8.5 × 10⁻⁹
Tracer circles vs midpoint, read from the image (approximate) Fig. 4L (n = 42) no count given in the legend 41 of 42 circles resolved: 35 above (X side), 2 within ±3 units, 4 below (Y side), all 4 among the nuclei at the right of the plot
XY one per hemisphere, WT male MEFs Results, mouse section para 1; Fig. 4 E, F 28/42, P = 0.022 one-sided P = 0.0218 (match)
Chr 16 homologs in separate hemispheres, Ts65Dn MEFs Results, mouse section para 2; SI Fig. S5 A, B 28/42, P = 0.022 one-sided P = 0.0218 (match)
Chr 17 homologs, WT MEF anaphase Results, mouse section para 1 28/36, P = 0.001 one-sided 0.0006 (match)
Chr 4, HUVEC metaphase / anaphase Results, Homologous and XY section para 1 15/20, P = 0.021; 28/38, P = 0.003 0.0207; 0.0025 (match)
XX, female HUVEC metaphase / anaphase Same section, para 2 18/24, P = 0.011; 32/42, P = 0.001 0.0113; 0.0005 (match)
XY, male HUVEC metaphase Same section, para 2 19/28, P = 0.044 0.0436 (match)
XY, male HUVEC anaphase (side finding) Same section, para 2; Fig. 2 D–F 20/28, P = 0.001 one-sided 0.0178, two-sided 0.0357, b(x) 0.0116; none gives 0.001 (that would need about 23/28)

Two things follow. First, every reported P except one reproduces as a one-sided binomial tail probability, so the 28/42 count is the paper's own figure for this result, not a typo. Second, a count of 28/42 is a majority, not an exclusive pattern, and the CI rules out anything near 100%.

The image read-out is resolution-limited and is not meant as a recount. It suggests more circles above the midpoint than the 28 counted in the text. One possible reason is the Methods' rule that boundary chromosomes were scored against the model. I do not treat this difference as a second error. The read-out matters here only because it shows tracer markers on the Y side of the boundary in the paper's own figure.

Figure

Significance statement wording ("exclusively", implying 42/42) set against the Results count of 28/42 cells for the same experiment

Figure 1. The same experiment described in two places. Left: the Significance statement, whose word "exclusively" implies the tracer is in the X-containing set in all cells (orange, the paper's wording). Right: the Results sentence that reports the experiment, n = 28/42 cells, which is 66.7% (blue, recomputed from the paper's own count). The connector marks that both sentences describe the same 42 cells. Below: the 14 cells not counted on the X side, and the exact 95% CI, which excludes 100%.

Dot-and-interval plot on a 0–100% axis: the Significance statement at 100% and the Results estimate of 66.7% with exact 95% CI 50.5–80.4%, with a reference line at 50%

Figure 2. Share of the 42 male Ts65Dn MEF cells with the maternal-origin tracer chromosome in the X-containing hemisphere, on one 0–100% axis. Orange: the share implied by "exclusively" in the Significance statement (100%). Blue: the share reported in Results, 28/42 = 66.7%, with its exact 95% CI (50.5% to 80.4%), computed in recompute.py.txt. The dashed line marks the chance value of 50% used as the null in the paper's binomial test. The lower CI limit lies just above that line, and the upper limit lies well below 100%.

Why it matters

Explanations considered

None of these accounts for the gap between "exclusively" and 28 of 42 cells.

Status

As of 2026-10-23:

Suggested correction

In the Significance statement, replace "display the tracer chromosome exclusively to the haploid set containing the X chromosome" with wording that matches the Results, for example: "display the tracer chromosome more often in the haploid set containing the X chromosome (28 of 42 cells, 67%; one-sided binomial P = 0.022)". Correspondingly, the Fig. 4 legend title could read "A maternally derived supernumerary chromosome segregates preferentially with the X chromosome", and "consistently" in the Movie S7 legend could be replaced by "in most cells (28/42)". The authors may also wish to:

Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the parental-origin (Ts65Dn tracer) result in PMC6310853
(Proc Natl Acad Sci U S A 2018;115(52):E12235-E12244, doi:10.1073/pnas.1809583115),
and regenerate fig1/fig2 in this folder.

Run:  python recompute.py.txt [path/to/pnas.1809583115fig04.jpg]
Needs numpy, scipy, matplotlib; Pillow only for the optional Fig. 4L read-out (part 5).
If no image path is given, part 5 downloads the published Fig. 4 image from the PMC CDN
into a temporary directory that is deleted afterwards; if that fails, part 5 is skipped and
the figures are still produced. The image is only read, never copied into this folder.
"""
import os
import shutil
import sys
import tempfile
import urllib.request
from pathlib import Path

import numpy as np
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = Path(__file__).resolve().parent

# ------------------------------------------------------------------ inputs (paper)
# (label, x, n, reported P, location)
ROWS = [
    ("Chr 4, HUVEC metaphase", 15, 20, "0.021", "Results, Homologous and XY ..., para 1"),
    ("Chr 4, HUVEC anaphase", 28, 38, "0.003", "Results, Homologous and XY ..., para 1"),
    ("XX, female HUVEC metaphase", 18, 24, "0.011", "Results, Homologous and XY ..., para 2"),
    ("XX, female HUVEC anaphase", 32, 42, "0.001", "Results, Homologous and XY ..., para 2"),
    ("XY, male HUVEC metaphase", 19, 28, "0.044", "Results, Homologous and XY ..., para 2"),
    ("XY, male HUVEC anaphase", 20, 28, "0.001", "Results, Homologous and XY ..., para 2"),
    ("Chr 17, WT MEF anaphase", 28, 36, "0.001", "Results, Mouse section, para 1"),
    ("XY, WT male MEF", 28, 42, "0.022", "Results, Mouse section, para 1"),
    ("Chr 16 homologs apart, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
    ("Tracer 17(16) with maternal X, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
]
X_KEY, N_KEY = 28, 42

print("=" * 78)
print("1. Binomial tests against p = 0.5 for every count reported with a P value")
print("   Methods: binomial null, 'one-tailed P value'.")
print(f"   {'comparison':42s} {'x/n':>7s} {'%':>6s} {'P1(>=x)':>9s} {'P2':>8s} {'b(x)':>8s} {'reported':>8s}")
results = {}
for lab, x, n, rep, loc in ROWS:
    p1 = binom.sf(x - 1, n, 0.5)
    p2 = binomtest(x, n, 0.5).pvalue
    pmf = binom.pmf(x, n, 0.5)
    results[lab] = (p1, p2, pmf)
    print(f"   {lab:42s} {x:>3d}/{n:<3d} {100*x/n:6.1f} {p1:9.4f} {p2:8.4f} {pmf:8.4f} {rep:>8s}")

print("\n2. The parental-origin result (tracer chromosome 17(16) in the maternal-X hemisphere)")
frac = X_KEY / N_KEY
ci = binomtest(X_KEY, N_KEY, 0.5).proportion_ci(confidence_level=0.95, method="exact")
p1, p2, _ = results["Tracer 17(16) with maternal X, Ts65Dn MEF"]
print(f"   reported: n = {X_KEY}/{N_KEY} cells, P = 0.022")
print(f"   fraction in X hemisphere = {X_KEY}/{N_KEY} = {frac:.4f} ({100*frac:.1f}%)")
print(f"   cells not counted as X hemisphere = {N_KEY - X_KEY}/{N_KEY} ({100*(N_KEY - X_KEY)/N_KEY:.1f}%)")
print(f"   exact (Clopper-Pearson) 95% CI for the fraction = {ci.low:.3f} to {ci.high:.3f} "
      f"({100*ci.low:.1f}% to {100*ci.high:.1f}%)")
print(f"   one-sided binomial P(X >= {X_KEY} | n = {N_KEY}, p = 0.5) = {p1:.4f}; two-sided = {p2:.4f}")
print(f"   'exclusively' corresponds to {N_KEY}/{N_KEY} = 100%; 100% lies outside the CI "
      f"(upper limit {100*ci.high:.1f}%)")
for ptrue in (0.90, 0.95):
    print(f"   if the true fraction were {ptrue:.2f}: P(X <= {X_KEY} | n = {N_KEY}) = {binom.cdf(X_KEY, N_KEY, ptrue):.1e}")
xmin = int(min(k for k in range(N_KEY + 1) if binom.sf(k - 1, N_KEY, 0.5) < 0.05))
print(f"   smallest count out of {N_KEY} with one-sided P < 0.05: {xmin} "
      f"(P = {binom.sf(xmin - 1, N_KEY, 0.5):.4f}); the reported 28 is {X_KEY - xmin} above it")
print(f"   one-sided P for 26/42 = {binom.sf(25, N_KEY, 0.5):.4f} (not < 0.05)")

print("\n3. Identical count reported for three different measurements")
for lab in ("XY, WT male MEF", "Chr 16 homologs apart, Ts65Dn MEF", "Tracer 17(16) with maternal X, Ts65Dn MEF"):
    print(f"   {lab:42s} 28/42, P = 0.022 reported; one-sided P recomputed {results[lab][0]:.4f}")

print("\n4. Side finding: XY male HUVEC anaphase")
p1, p2, pmf = results["XY, male HUVEC anaphase"]
print(f"   reported 20/28, P = 0.001; recomputed one-sided {p1:.4f}, two-sided {p2:.4f}, pmf {pmf:.4f}")
for x in range(20, 29):
    if binom.sf(x - 1, 28, 0.5) < 0.0015:
        print(f"   smallest x out of 28 whose one-sided P rounds to <= 0.001: {x} (P = {binom.sf(x-1, 28, 0.5):.4f})")
        break

# ------------------------------------------------------------------ part 5: Fig. 4L read-out
FIG_URL = ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/c3f5/6310853/c1c7e28b198f/"
           "pnas.1809583115fig04.jpg")


TMPDIR = None


def get_fig4():
    global TMPDIR
    if len(sys.argv) > 1 and os.path.exists(sys.argv[1]):
        return sys.argv[1]
    try:
        d = TMPDIR = tempfile.mkdtemp(prefix="pmc6310853_")
        path = os.path.join(d, "fig4.jpg")
        req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
        with urllib.request.urlopen(req, timeout=60) as r, open(path, "wb") as f:
            f.write(r.read())
        return path
    except Exception as e:  # network not available
        print(f"   (could not download Fig. 4: {e})")
        return None


print("\n5. Fig. 4L read from the published image (757 x 449 px; tracer circles vs chr16 midpoint '+')")
fig4 = get_fig4()
if fig4 is None:
    print("   skipped")
else:
    from PIL import Image
    from scipy import ndimage as ndi
    im = np.asarray(Image.open(fig4).convert("RGB")).astype(int)
    if im.shape[:2] != (449, 757):
        print(f"   unexpected image size {im.shape[:2]}; skipped")
    else:
        y0, x0 = 336, 64                      # inside the panel L frame (frame rows 334-424, cols 62-182)
        sub = im[y0:423, x0:181]
        r, g, b = sub[..., 0], sub[..., 1], sub[..., 2]
        TOP, BOT = 334, 424                   # frame lines = +50 / -50 on the y axis
        pct = lambda y: 50 - 100 * (y - TOP) / (BOT - TOP)
        yel = (r > 170) & (g > 170) & (b < 160)
        circ = ndi.binary_dilation(yel, iterations=3)
        gry = (abs(r - g) < 25) & (abs(g - b) < 25) & (r > 100) & (r < 225) & ~circ
        gry[:15, :] = False                   # drop the 'n=42' label
        ys, xs = np.nonzero(gry)
        p = np.polyfit(xs + x0, ys + y0, 1)
        for _ in range(3):
            keep = abs(ys + y0 - np.polyval(p, xs + x0)) < 3
            p = np.polyfit(xs[keep] + x0, ys[keep] + y0, 1)

        def mid(X):
            m = keep & (abs(xs + x0 - X) <= 5)
            if m.sum() < 3:
                return pct(np.polyval(p, X))
            return pct(np.median(ys[m] + y0))

        lab, nb = ndi.label(yel)
        cs = ndi.center_of_mass(yel, lab, range(1, nb + 1))
        sz = ndi.sum(yel, lab, range(1, nb + 1))
        cnt = {"above": 0, "near": 0, "below": 0}
        tot = 0
        for (cy, cx), s in sorted(zip(cs, sz), key=lambda t: t[0][1]):
            if s < 4:
                continue
            k = 2 if s > 20 else 1            # two overlapping circles merged into one blob
            tot += k
            d = pct(cy + y0) - mid(cx + x0)
            t = "above" if d >= 3 else ("below" if d <= -3 else "near")
            cnt[t] += k
            if t != "above":
                print(f"   x = {cx + x0:5.1f} px: tracer {pct(cy + y0):5.1f}, midpoint {mid(cx + x0):5.1f}, "
                      f"difference {d:5.1f} -> {t}")
        print(f"   circles resolved: {tot} of 42; above midpoint (X side) {cnt['above']}, "
              f"within +/-3 units {cnt['near']}, below (Y side) {cnt['below']}")
        print("   (approximate: low resolution, overlapping circles; boundary threshold 3 axis units)")
if TMPDIR:
    shutil.rmtree(TMPDIR, ignore_errors=True)

# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})


def save(fig, name):
    fig.savefig(HERE / f"{name}.svg")
    fig.savefig(HERE / f"{name}.png", dpi=200)
    plt.close(fig)
    print(f"   wrote {name}.svg / {name}.png")


print("\n6. Figures")
# ---- fig1: Significance statement vs Results (abstract-vs-body contradiction)
fig = plt.figure(figsize=(7.2, 4.1))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.1); ax.axis("off")


def panel(x, y, w, h, title, body, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.15, y + h - 0.18, title, fontsize=10.5, fontweight="bold", color=INK, va="top")
    ax.text(x + 0.15, y + h - 0.55, body, fontsize=10, color=INK, va="top", linespacing=1.45)


panel(0.12, 1.35, 2.85, 2.55, "Significance statement",
      "“Male translocation mice with a\nmaternal-derived supernumerary\n"
      "chromosome display the tracer\nchromosome exclusively to the\n"
      "haploid set containing the X\nchromosome.”", INK2)
panel(4.23, 1.35, 2.85, 2.55, "Results, mouse section, para 2",
      "“In the majority of cases we\nexamined, the hemisphere\ncontaining "
      "the translocated\nchromosome 17¹⁶ also contained\nthe maternally derived X\n"
      "chromosome (n = 28/42 cells,\nP = 0.022 ...)”", INK2)
ax.text(1.545, 1.55, "implies 42/42 = 100%", ha="center", fontsize=10.5, color=ORANGE, fontweight="bold")
ax.text(5.655, 1.55, f"{X_KEY}/{N_KEY} = {100*frac:.1f}%", ha="center", fontsize=10.5, color=BLUE, fontweight="bold")
arr = FancyArrowPatch((3.0, 2.62), (4.2, 2.62), arrowstyle="<->", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(arr)
ax.text(3.6, 2.80, "same 42\ncells", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(3.6, 2.45, "“exclusively”\nvs\n“majority”", ha="center", va="top", fontsize=10, color=INK)
ax.text(3.6, 1.02, "Both sentences describe one experiment: Fig. 4 H–L, male Ts65Dn MEFs, n = 42.",
        ha="center", fontsize=10, color=INK2)
ax.text(3.6, 0.62,
        f"{N_KEY - X_KEY} of {N_KEY} cells ({100*(N_KEY - X_KEY)/N_KEY:.1f}%) not counted on the X side; "
        f"exact 95% CI {100*ci.low:.1f}–{100*ci.high:.1f}% excludes 100%",
        ha="center", fontsize=10, color=INK)
ax.text(3.6, 0.25, "orange = paper's wording (Significance); blue = recomputed from the paper's own count",
        ha="center", fontsize=9.5, color=INK2, style="italic")
save(fig, "fig1")

# ---- fig2: proportion on one axis, claim vs data with exact CI, null reference
fig, ax = plt.subplots(figsize=(7.2, 2.9))
fig.subplots_adjust(left=0.30, right=0.97, top=0.86, bottom=0.22)
ax.axvline(50, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(49, 0.45, "chance (p = 0.5)", ha="right", va="bottom", fontsize=9.5, color=INK2)
ax.plot([100], [2], "o", ms=10, color=ORANGE, zorder=3)
ax.annotate("100%", (100, 2), xytext=(-10, 0), textcoords="offset points", ha="right", va="center",
            fontsize=10, color=INK)
ax.plot([100 * ci.low, 100 * ci.high], [1, 1], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for v in (ci.low, ci.high):
    ax.plot([100 * v, 100 * v], [0.88, 1.12], color=BLUE, lw=2, zorder=2)
ax.plot([100 * frac], [1], "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"{100*frac:.1f}%  (95% CI {100*ci.low:.1f}–{100*ci.high:.1f}%)", (52, 1),
            xytext=(0, -14), textcoords="offset points", ha="left", va="top", fontsize=10, color=INK)
ax.set_yticks([2, 1])
ax.set_yticklabels(["Significance statement:\n“exclusively” (paper)",
                    "Results: 28/42 cells\n(recomputed, exact CI)"], fontsize=10, color=INK)
ax.set_ylim(0.35, 2.6)
ax.set_xlim(0, 105)
ax.set_xticks([0, 25, 50, 75, 100])
ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"])
ax.set_xlabel("Cells with the tracer chromosome in the X-containing hemisphere", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title("Share of 42 male Ts65Dn MEF cells, maternal-origin tracer (Fig. 4 H–L)",
             fontsize=10.5, color=INK, loc="left", x=-0.39)
save(fig, "fig2")

Figure 3 legend calls a transduction P value of 0.096 statistically significant

Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.1910844119 · PMC8812682

Summary

The Figure 3 legend reports a Friedman/exact test of CAR transduction across three activation conditions (CD3, CD3/CD28, CD3/CD28/CD81) and states that the data "reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096)". At the conventional 0.05 level, which the paper's parallel Figure 2 statement also uses (P = 0.038, called significant), a P value of 0.096 does not reject the null hypothesis. No other significance level, one-sided test or adjustment is stated anywhere in the article. This P value is the only inferential statistic the article gives for any transduction comparison, and transduction is the outcome named in the title. The observed direction of the effect is unaffected, but its stated statistical significance is not supported by the number printed.

What the paper reports

Figure 3 legend ("CD81 costimulation improves CAR transduction efficiency"):

These data are representative of three experiments, with transduction efficiency across experiments illustrated. For CAR transduction, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median transduction between groups. Our data reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096) for cells treated with additional CD81 costimulation.

Figure 2 legend ("CD81 costimulation enhances activation of isolated naive T cells"), the parallel statement for activation:

These data are representative of four experiments with average CD69 expression prior to CAR transduction across experiments illustrated. For activation, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median activation between groups. Our data reject the null hypothesis, indicating a statistically significant difference in activation (P = 0.038) for cells treated with additional CD81 costimulation.

Results, "CD81 Costimulation Enhances CAR Transduction in Naive T Cells":

In specific, activating naive T cells with additional CD81 costimulation achieved greater transduction efficiency as compared with CD3 or CD3/CD28 activation conditions (Fig. 3A and SI Appendix, Fig. S2A).

Abstract (Significance statement):

We show that CD81 costimulation of naive T cells prior to CAR transduction can lead to enhanced CAR expression in this T cell subset.

Discussion, second paragraph:

Importantly, the T cell product derived from these CD81-costimulated naive T cells achieved greater transduction efficiency, with greater levels of CD19 CAR expression.

The article has no statistics subsection in its Methods and states no significance level. The only P values in the article are the two above and the three in the Figure 1C legend (CD4 "<0.0001", CD8 "<0.0002", naive "<0.0001"; seven experiments; no test named).

What the paper's own numbers show

Method. I compared each reported P value with the conventional significance level of 0.05, checked whether rounding, halving (a one-sided reading) or a 0.10 level would change the comparison, and confirmed that the parallel Figure 2 statement follows the same template with a P value below 0.05. To see whether a P value below 0.05 was reachable with the stated design, I enumerated the exact null distribution of the Friedman statistic for three conditions and three, four and seven experiments (all (3!)^n equally likely within-experiment rankings, assuming no ties). The per-experiment transduction values are not in the article, so the test itself cannot be recomputed. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Transduction P value Fig. 3 legend P = 0.096, "reject the null hypothesis", "statistically significant" 0.096 > 0.05, so H0 is not rejected at 0.05; P is 1.92 times the threshold
Unrounded range of the transduction P value Fig. 3 legend 0.096 any value in [0.0955, 0.0965) rounds to 0.096; all are above 0.05
One-sided reading derived not stated 0.096 / 2 = 0.048 (< 0.05), but the legend prints 0.096 as the P it calls significant
0.10 level derived not stated 0.096 ≤ 0.10, but no 0.10 level is stated anywhere in the article
Activation P value Fig. 2 legend P = 0.038, "statistically significant" 0.038 < 0.05, so H0 is rejected at 0.05; consistent
Smallest attainable exact Friedman P, 3 experiments × 3 conditions Fig. 3 design not stated 0.0278 (6/216), reached only when all three experiments rank the conditions identically; the next value is 0.1944
Is 0.096 an attainable exact P for 3 × 3? Fig. 3 design 0.096 no; the nearest chi-square-approximation P is 0.0970 (Q = 4.667), whose exact P is 0.1944
Smallest attainable exact Friedman P, 4 experiments × 3 conditions Fig. 2 design not stated 0.0046; 0.038 is not attainable exactly (nearest: exact 0.0417, chi-square 0.0388 at Q = 6.5)
Smallest attainable exact Friedman P, 7 experiments × 3 conditions Fig. 1C design "<0.0001", "<0.0002", "<0.0001" 0.000021, so values below 0.0001 are attainable; consistent if the same test was used

The central finding needs no reconstruction: the legend's own P value is above 0.05, and the legend's own conclusion is that the null hypothesis is rejected. The attainability rows are a secondary observation. Neither 0.096 nor 0.038 matches the exact Friedman distribution for the simplest reading of the stated designs, which suggests that the printed values come from a different computation (for example the chi-square approximation, ties, or a different number of blocks) or contain a transcription slip. The article does not give enough detail to tell which.

Figure

Reported P values from the Fig. 2 and Fig. 3 legends compared with the 0.05 level and with the exact Friedman P values the stated designs can produce

P values on a log scale, with the 0.05 level as a vertical reference line. Filled orange points are the P values printed in the Figure 2 (activation, four experiments) and Figure 3 (transduction, three experiments) legends; both legends describe their result as statistically significant. Open blue points are the exact Friedman P values that three conditions and the stated number of experiments can produce (no ties), computed by recompute.py.txt. The Figure 2 value falls to the left of the line, where H0 is rejected. The Figure 3 value falls to the right of it, where H0 is not rejected. For the Figure 3 design an exact P below 0.05 (0.028) was attainable, and 0.096 itself is not one of the attainable values.

Why it matters

The article's title states that CD81 costimulation "skews CAR transduction toward naive T cells", and the Significance statement and Discussion describe enhanced transduction and CAR expression. The Figure 3 legend contains the only inferential test the article reports for any transduction comparison: the transduction results in Figures 4 and 5 carry no P values. The legend presents that test as a rejection of the null hypothesis. Read at the conventional 0.05 level, the test does not reject it. With three experiments the result is better described as a non-significant difference in the stated direction. The observed direction and size of the transduction difference are unchanged, and the activation results (Fig. 1C; Fig. 2, P = 0.038) are not affected. The paper calls itself a proof of principle, and its qualitative message survives. What does not survive is the claim that the transduction difference in Figure 3 is statistically significant. A reader or a meta-analyst taking the legend at its word would count as significant a result that is not significant by the paper's own figure.

Explanations considered

Status

Suggested correction

Either correct the conclusion or correct the number, whichever the underlying data support:

In either case, stating the significance level used for all tests (presumably 0.05, as in Figure 2) would remove the ambiguity.

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8812682 and regenerate fig1.svg / fig1.png.

Run:  python recompute.py.txt   (needs numpy, scipy, matplotlib)

Inputs are only numbers printed in the article:
  Fig. 3 legend: transduction, Friedman test and exact test, "three experiments", P = 0.096
  Fig. 2 legend: activation,   Friedman test and exact test, "four experiments",  P = 0.038
  Fig. 1C legend: activation, "seven experiments", P < 0.0001 / < 0.0002 / < 0.0001
  Three activation conditions throughout: CD3, CD3/CD28, CD3/CD28/CD81 (k = 3).
"""
import itertools
import os
from collections import Counter
from fractions import Fraction

from scipy.stats import chi2

HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05
P_FIG3 = 0.096   # transduction, called "statistically significant"
P_FIG2 = 0.038   # activation, called "statistically significant"
K = 3            # conditions: CD3, CD3/CD28, CD3/CD28/CD81


def friedman_exact(n, k):
    """Exact null distribution of the Friedman statistic Q (no ties).

    Returns list of (Q, exact upper-tail P, chi-square approximation P), Q ascending.
    Every within-block ranking is equally likely under H0: (k!)^n configurations.
    """
    perms = list(itertools.permutations(range(1, k + 1)))
    sums = Counter({tuple([0] * k): 1})
    for _ in range(n):
        nxt = Counter()
        for s, m in sums.items():
            for p in perms:
                nxt[tuple(a + b for a, b in zip(s, p))] += m
        sums = nxt
    total = len(perms) ** n
    qc = Counter()
    for s, m in sums.items():
        q = Fraction(12, n * k * (k + 1)) * sum(r * r for r in s) - 3 * n * (k + 1)
        qc[q] += m
    qs = sorted(qc)
    out = []
    for q in qs:
        tail = sum(qc[x] for x in qs if x >= q)
        out.append((float(q), tail / total, float(chi2.sf(float(q), k - 1))))
    return out, total


print("=== 1. The significance statement in the Fig. 3 legend ===")
print(f"Reported P (Fig. 3, transduction)      = {P_FIG3}")
print(f"Conventional alpha                     = {ALPHA}")
print(f"P <= alpha ?                           = {P_FIG3 <= ALPHA}  -> H0 not rejected at 0.05")
print(f"P / alpha                              = {P_FIG3 / ALPHA:.2f}")
print(f"P - alpha                              = {P_FIG3 - ALPHA:.3f}")
lo, hi = P_FIG3 - 0.0005, P_FIG3 + 0.0005
print(f"Range of unrounded P that rounds to 0.096 = [{lo:.4f}, {hi:.4f}); lower end > 0.05: {lo > ALPHA}")
print(f"Half of reported P (one-sided reading) = {P_FIG3 / 2:.3f}  (< 0.05: {P_FIG3 / 2 < ALPHA})")
print(f"Reported P <= 0.10 ?                   = {P_FIG3 <= 0.10}")
print()
print("=== 2. The parallel statement in the Fig. 2 legend ===")
print(f"Reported P (Fig. 2, activation)        = {P_FIG2}")
print(f"P <= alpha ?                           = {P_FIG2 <= ALPHA}  -> H0 rejected at 0.05 (consistent)")
print()

print("=== 3. Exact Friedman P values attainable with k = 3 conditions (no ties) ===")
designs = {}
for n, note in [(3, "Fig. 3: three experiments"), (4, "Fig. 2: four experiments"),
                (7, "Fig. 1C: seven experiments")]:
    dist, total = friedman_exact(n, K)
    designs[n] = dist
    smallest = min(p for _, p, _ in dist)
    print(f"n = {n} blocks x k = {K}  ({note}); {total} equally likely rank configurations")
    print(f"  smallest attainable exact P = {smallest:.6f}   (< 0.05: {smallest < ALPHA})")
    if n <= 4:
        print("  attainable values (Q: exact P | chi-square approx P):")
        for q, p, a in sorted(dist, key=lambda t: -t[0]):
            print(f"    Q = {q:6.3f}:  exact {p:.4f} | chi2 {a:.4f}")
    else:
        print(f"  smallest attainable exact P < 0.0001: {smallest < 0.0001}")
    print()

d3 = designs[3]
print("Fig. 3 design (n = 3): is 0.096 an attainable exact P?",
      any(abs(p - P_FIG3) < 0.0005 for _, p, _ in d3))
q_near = min(d3, key=lambda t: abs(t[2] - P_FIG3))
print(f"  nearest chi-square-approximation P: Q = {q_near[0]:.3f}, chi2 P = {q_near[2]:.4f}, "
      f"exact P for the same Q = {q_near[1]:.4f}")
d4 = designs[4]
print("Fig. 2 design (n = 4): is 0.038 an attainable exact P?",
      any(abs(p - P_FIG2) < 0.0005 for _, p, _ in d4))
q_near4 = min(d4, key=lambda t: abs(t[2] - P_FIG2))
print(f"  nearest chi-square-approximation P: Q = {q_near4[0]:.3f}, chi2 P = {q_near4[2]:.4f}, "
      f"exact P for the same Q = {q_near4[1]:.4f}")
print()

# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
                     "svg.fonttype": "none"})

fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xscale("log")
XMIN, XMAX = 0.003, 1.0
ax.set_xlim(XMIN, XMAX)

rows = [  # (y, label)
    (3.0, "Fig. 2 (activation)\nreported P"),
    (2.3, "Fig. 2 design\nattainable exact P"),
    (1.0, "Fig. 3 (transduction)\nreported P"),
    (0.3, "Fig. 3 design\nattainable exact P"),
]
ax.set_ylim(-0.25, 3.55)
ax.set_yticks([y for y, _ in rows])
ax.set_yticklabels([l for _, l in rows], fontsize=10)

# alpha reference line and regions
ax.axvline(ALPHA, color=INK2, lw=1, zorder=1)
ax.text(ALPHA * 0.92, -0.2, "\u2190 P \u2264 \u03b1 = 0.05: reject H\u2080", ha="right", va="bottom",
        color=INK2, fontsize=10)
ax.text(ALPHA * 1.08, -0.2, "P > 0.05: do not reject H\u2080 \u2192", ha="left", va="bottom",
        color=INK2, fontsize=10)

# attainable exact P values (blue, recomputed); label the ones that bracket 0.05
labels = {2.3: {0.0046: -1, 0.0417: -1, 0.0694: 1}, 0.3: {0.0278: 1, 0.1944: 1}}
for y, dist in [(2.3, designs[4]), (0.3, designs[3])]:
    ps = sorted({round(p, 4) for _, p, _ in dist if p >= XMIN})
    ax.plot(ps, [y] * len(ps), ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
    for p, side in labels[y].items():
        ax.annotate(f"{p:.3f}", (p, y), xytext=(0, 11 * side), textcoords="offset points",
                    ha="center", va="bottom" if side > 0 else "top", color=INK2, fontsize=10,
                    bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)

# reported P values (orange, from the paper)
for y, p, verdict in [(3.0, P_FIG2, "described as significant"),
                      (1.0, P_FIG3, "described as significant")]:
    ax.plot([p], [y], ls="none", marker="o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=4)
    ax.annotate(f"P = {p:.3f}, {verdict}", (p, y), xytext=(0, 11),
                textcoords="offset points", ha="center", va="bottom", color=INK, fontsize=10,
                bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)

ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("P value (log scale)", color=INK)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.tick_params(colors=INK2, axis="x")

handles = [
    Line2D([], [], ls="none", marker="o", ms=10, color=ORANGE, label="Reported in the paper"),
    Line2D([], [], ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2,
           label="Recomputed: exact Friedman P values the stated design can produce"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1,
          frameon=False, fontsize=10, labelcolor=INK)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
    out = os.path.join(HERE, f"fig1.{ext}")
    fig.savefig(out, facecolor=BG, bbox_inches="tight", **kw)
    print("wrote", out)

Chi-square P = 0.0379 for casein B-cell responders does not follow from 19/45 vs 10/35

Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.2117034119 · PMC8916005

Summary

The paper reports that more patients with multiple sclerosis (MS) than patients with other neurologic diseases (OND) had casein-specific B cells in an ELISPOT assay: 19/45 (42%) vs 10/35 (28%), "P = 0.0379; χ2 test". A chi-square test on those counts gives χ² = 1.59, P = 0.208. Fisher's exact test gives P = 0.246, and none of the eight other standard two-group tests tried comes below P = 0.10. The reported P is reproduced exactly (P = 0.03794) only when the printed percentages are entered as if they were counts out of 100 patients per group. This comparison is the only between-group support the paper gives for the B-cell half of the abstract's statement that patients with MS had "significantly higher B cell and antibody reactivity to bovine casein". The mouse experiments and the separate serum IgG ELISA comparison are not affected.

What the paper reports

The result itself:

"While the number of casein-specific B cell spots/patient was similar in the two patient groups, the number of patients positive for casein-specific B cells was higher in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; χ2 test)." (Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3)

The group sizes are stated in the same paragraph, in the Methods and in Table 2:

"we tested the CNS- and casein-specific B cell response in patients with MS (n = 45) and those with ONDs (n = 35)" (Results, same paragraph)

"PBMCs from 45 patients with MS and 35 patients with ONDs were tested by ELISPOT assay." (Materials and Methods, "Patient Sample ELISPOT")

"ELISPOT | 35 | 45" and "ELISPOT casein responders, %† | 28 | 42" (Table 2, columns "Patients with ONDs" | "Patients with MS")

The claims that rest on this comparison:

"The results obtained from the mouse model were complemented by clinical data showing that serum samples from patients with MS contained significantly higher B cell and antibody reactivity to bovine casein than those from patients with other neurologic diseases." (Abstract)

"Patients with MS show elevated numbers of B cell responses and IgG titers to bovine casein in the blood." (Fig. 4 title)

The Statistical Analysis section lists the tests used and the significance level; it does not mention a chi-square test:

"differences between two parametric groups were assessed using an unpaired t test or paired t test while ANOVA was used for greater than two groups. For nonparametric datasets, a Mann–Whitney U test was used. For both parametric and nonparametric datasets, a significance level of 5% was chosen." (Materials and Methods, "Statistical Analysis")

What the paper's own numbers show

Method. The stated counts give the 2×2 table MS 19 positive / 26 negative and OND 10 positive / 25 negative (80 patients). recompute.py.txt runs the usual two-group tests for a difference in proportions on this table with scipy: Pearson chi-square with and without Yates' continuity correction, its one-sided version, the likelihood-ratio G-test, the N−1 chi-square, and Fisher's, Barnard's and Boschloo's exact tests. It then searches every 2×2 table with row totals 45 and 35 for one whose chi-square P (uncorrected or Yates) equals 0.0379. Finally it tries tests that are not two-group comparisons and a chi-square on the printed percentages entered as counts.

Quantity Location Reported Recomputed
MS responders Results para 3; Table 2 19/45; 42% 42.22%
OND responders Results para 3; Table 2 10/35; 28% 28.57% (29% with conventional rounding)
Difference in responder proportion derived 42% vs 28% 13.7 percentage points; Newcombe 95% CI −7.5 to 32.6
Pearson χ², no correction Results para 3 P = 0.0379 (χ2 test) χ² = 1.588, df 1, P = 0.2077
Pearson χ², Yates correction Results para 3 P = 0.0379 χ² = 1.052, P = 0.3051
Pearson χ², one-sided Results para 3 P = 0.0379 P = 0.1038
G-test / N−1 χ² Results para 3 P = 0.0379 P = 0.2050 / 0.2105
Fisher exact, two-sided / one-sided Results para 3 P = 0.0379 P = 0.2463 / 0.1525
Barnard / Boschloo exact, two-sided Results para 3 P = 0.0379 P = 0.2326 / 0.2175
χ² needed for P = 0.0379 (df 1) derived — 4.31; observed 1.59
Any table with row totals 45 and 35 giving P = 0.0379 derived — none (closest: Yates P = 0.0373 for 17/45 vs 5/35, and 0.0384 for 23/45 vs 9/35)
Percentages entered as counts, 42/100 vs 28/100, Pearson χ² derived P = 0.0379 χ² = 4.308, P = 0.037940 (exact match)
Same, 42/100 vs 29/100 (conventional rounding) derived — P = 0.0547
Same, with Yates correction derived — P = 0.0539
MS rate against a fixed rate of 10/35 or 0.28 (goodness of fit; binomial) derived — P = 0.043 or 0.034; binomial P = 0.048 or 0.045
ELISA OD > 1.0 rates, 16/39 vs 5/23 Table 2 (no P given) 41.02% vs 21.7% χ² P = 0.1211; Fisher P = 0.1673
Group sizes needed at the same proportions for P < 0.05 derived — about 3 times larger (135/105: P = 0.029; 90/70: P = 0.075)

The expected numbers of responders under no group difference are 16.3 (MS) and 12.7 (OND), against 19 and 10 observed. The four cell contributions to Pearson's statistic sum to 1.59, well short of the 4.31 needed for P = 0.0379. The 2×2 table is fully determined by the text and Table 2, and the table's row totals equal the totals in the Methods, so there is no unstated count that could be varied.

Figure

Reported versus recomputed P values for the MS vs OND B-cell responder comparison

Figure 1. P values for the comparison of casein-specific B-cell responders, 19/45 (MS) vs 10/35 (OND), on one log-scaled axis. The orange point is the value reported in the Results (P = 0.0379, "χ2 test"). The filled blue points are nine standard two-group tests recomputed from the stated counts. All lie between 0.10 and 0.31, on the far side of the dashed line at the paper's stated 5% significance level. The open blue point in the shaded row is a Pearson chi-square run on the printed percentages as if each group had 100 patients (42/100 vs 28/100). It gives P = 0.0379, the reported value, to four decimals. All values come from recompute.py.txt.

Why it matters

The comparison changes from significant (as reported) to not significant (as recomputed) at the 5% level the paper sets. The recomputed 95% CI for the difference in responder proportions runs from −7.5 to +32.6 percentage points and includes zero. The same paragraph says that "the number of casein-specific B cell spots/patient was similar in the two patient groups", so the paper's ELISPOT data provide no between-group difference in B-cell reactivity that is statistically significant.

This affects the following statements:

The direction of the observed difference is unchanged (42% vs 29%), and the data are compatible with a modest difference. They are equally compatible with none. At the observed proportions, about three times as many patients per group would have been needed for P < 0.05.

The main mouse findings do not depend on this test: casein immunization with demyelination and IgG deposition, MAG cross-reactivity shown with antibodies and adsorption, and complement dependence. Neither do the ELISA comparison (P = 0.032) or the MS-only correlation between CNS- and casein-specific spot counts (r = 0.605).

Explanations considered

Status

As of 2026-10-23:

Sources: PMC full text via src/show.py PMC8916005; src/known.py 10.1073/pnas.2117034119; api.crossref.org; Europe PMC REST (search, fullTextXML, supplementaryFiles); PubMed E-utilities.

Suggested correction

In Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3, the sentence could read: "the proportion of patients positive for casein-specific B cells was 42% (19/45) in the MS group and 29% (10/35) in the OND group; this difference was not statistically significant (P = 0.21, χ2 test; Fisher's exact test P = 0.25)." Reporting the difference with its 95% CI (13.7 percentage points, −7.5 to 32.6) would show the uncertainty.

Consequential changes:

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8916005 and regenerate fig1.

Run:  python recompute.py.txt   (writes fig1.svg and fig1.png next to this file)

Inputs come from the article as archived in PubMed Central (PMC8916005):
  Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells",
  paragraph 3: "the number of patients positive for casein-specific B cells was higher
  in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; chi2 test)".
  Table 2: ELISPOT n = 35 (OND), 45 (MS); "ELISPOT casein responders, %" 28 (OND), 42 (MS);
  ELISA n = 23 (OND), 39 (MS); "ELISA OD value > 1.0, %" 21.7 (OND), 41.02 (MS).
Requires numpy, scipy (>= 1.7 for barnard_exact/boschloo_exact) and matplotlib.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))

REPORTED_P = 0.0379
MS_POS, MS_N = 19, 45     # MS patients positive for casein-specific B cells
OND_POS, OND_N = 10, 35   # OND patients positive
REPORTED_PCT = (42, 28)   # percentages as printed in the text and in Table 2

tab = np.array([[MS_POS, MS_N - MS_POS], [OND_POS, OND_N - OND_POS]])

print("== Inputs ==")
print("2x2 table [[pos, neg] MS; [pos, neg] OND]:", tab.tolist())
print(f"MS  responders: {MS_POS}/{MS_N} = {100*MS_POS/MS_N:.2f}%  (reported 42%)")
print(f"OND responders: {OND_POS}/{OND_N} = {100*OND_POS/OND_N:.2f}%  (reported 28%; "
      f"conventional rounding gives {round(100*OND_POS/OND_N)}%)")
diff = MS_POS/MS_N - OND_POS/OND_N
print(f"Difference in proportions: {100*diff:.2f} percentage points")

exp = stats.contingency.expected_freq(tab)
print("Expected counts under no difference:", np.round(exp, 2).tolist())
contrib = (tab - exp) ** 2 / exp
print("Pearson contributions:", np.round(contrib.ravel(), 3).tolist())

print("\n== Two-group tests on the stated counts 19/45 vs 10/35 ==")
res = {}
c, p, _, _ = stats.chi2_contingency(tab, correction=False)
res["Pearson chi2, no correction"] = (c, p)
c, p, _, _ = stats.chi2_contingency(tab, correction=True)
res["Pearson chi2, Yates correction"] = (c, p)
res["Pearson chi2, one-sided"] = (res["Pearson chi2, no correction"][0],
                                  res["Pearson chi2, no correction"][1] / 2)
g, p, _, _ = stats.chi2_contingency(tab, correction=False, lambda_="log-likelihood")
res["G-test (likelihood ratio)"] = (g, p)
N = tab.sum()
cN1 = res["Pearson chi2, no correction"][0] * (N - 1) / N
res["N-1 chi2"] = (cN1, stats.chi2.sf(cN1, 1))
_, p = stats.fisher_exact(tab)
res["Fisher exact, two-sided"] = (np.nan, p)
_, p = stats.fisher_exact(tab, alternative="greater")
res["Fisher exact, one-sided"] = (np.nan, p)
r = stats.barnard_exact(tab)
res["Barnard exact, two-sided"] = (r.statistic, r.pvalue)
r = stats.boschloo_exact(tab)
res["Boschloo exact, two-sided"] = (r.statistic, r.pvalue)
for k, (s, p) in res.items():
    s_txt = "" if np.isnan(s) else f"stat = {s:.4f}"
    print(f"  {k:32s} {s_txt:16s} P = {p:.4f}")

chi_needed = stats.chi2.isf(REPORTED_P, 1)
print(f"\nChi-square (df 1) needed for P = {REPORTED_P}: {chi_needed:.4f}; "
      f"observed {res['Pearson chi2, no correction'][0]:.4f}")

# Newcombe (Wilson score) 95% CI for the difference in proportions
def wilson(x, n, z=1.959964):
    ph = x / n
    centre = (ph + z*z/(2*n)) / (1 + z*z/n)
    half = z*np.sqrt(ph*(1-ph)/n + z*z/(4*n*n)) / (1 + z*z/n)
    return centre - half, centre + half
l1, u1 = wilson(MS_POS, MS_N)
l2, u2 = wilson(OND_POS, OND_N)
p1, p2 = MS_POS/MS_N, OND_POS/OND_N
lo = diff - np.sqrt((p1-l1)**2 + (u2-p2)**2)
hi = diff + np.sqrt((u1-p1)**2 + (p2-l2)**2)
print(f"Newcombe 95% CI for the difference: {100*lo:.1f} to {100*hi:.1f} percentage points")

print("\n== Exhaustive search: all 2x2 tables with row totals 45 and 35 ==")
best = []
for a in range(MS_N + 1):
    for b in range(OND_N + 1):
        t = np.array([[a, MS_N - a], [b, OND_N - b]])
        if (t.sum(0) == 0).any():
            continue
        for corr in (False, True):
            _, p, _, _ = stats.chi2_contingency(t, correction=corr)
            best.append((abs(p - REPORTED_P), a, b, corr, p))
best.sort()
print("Closest five (uncorrected or Yates) to P = 0.0379:")
for d, a, b, corr, p in best[:5]:
    print(f"  {a}/45 vs {b}/35, Yates={corr}: P = {p:.4f}")
n_match = sum(1 for d, *_ in best if d < 0.00005)
print(f"Tables giving P = 0.0379 to 4 decimals: {n_match}; "
      f"among tables within 4 responders of 19 and 10: "
      f"{sum(1 for d,a,b,c,p in best if d < 0.00005 and abs(a-19)<=4 and abs(b-10)<=4)}")

print("\n== Percentages entered as counts out of 100 ==")
pct_res = {}
for a, b in [(42, 28), (42.22, 28.57), (42, 29)]:
    t = np.array([[a, 100 - a], [b, 100 - b]])
    c, p, _, _ = stats.chi2_contingency(t, correction=False)
    pct_res[(a, b)] = (c, p)
    print(f"  {a}/100 vs {b}/100, uncorrected: chi2 = {c:.4f}, P = {p:.6f}")
c, p, _, _ = stats.chi2_contingency(np.array([[42, 58], [28, 72]]), correction=True)
print(f"  42/100 vs 28/100, Yates:       chi2 = {c:.4f}, P = {p:.6f}")

print("\n== Other tests that do not compare two groups ==")
for lab, p0 in [("MS vs fixed p0 = 10/35", 10/35), ("MS vs fixed p0 = 0.28", 0.28)]:
    o = np.array([MS_POS, MS_N - MS_POS]); e = np.array([p0*MS_N, (1-p0)*MS_N])
    c, p = stats.chisquare(o, e)
    pb = stats.binomtest(MS_POS, MS_N, p0, alternative="two-sided").pvalue
    print(f"  {lab}: goodness-of-fit chi2 = {c:.4f}, P = {p:.4f}; binomial two-sided P = {pb:.4f}")
c, p = stats.chisquare([MS_POS, OND_POS])
print(f"  19 vs 10 responders, equal expected: chi2 = {c:.4f}, P = {p:.4f}")

print("\n== ELISA OD > 1.0 rates in Table 2 (no P value given there) ==")
ms_e, ond_e = 0.4102*39, 0.217*23
print(f"  41.02% of 39 = {ms_e:.2f}; 21.7% of 23 = {ond_e:.2f}  -> 16/39 vs 5/23")
te = np.array([[16, 23], [5, 18]])
c, p, _, _ = stats.chi2_contingency(te, correction=False)
print(f"  Pearson chi2 = {c:.4f}, P = {p:.4f}")
_, p = stats.fisher_exact(te)
print(f"  Fisher exact two-sided P = {p:.4f}")

print("\n== Sample size needed at the same proportions (19/45 vs 10/35) ==")
for k in range(1, 5):
    t = tab * k
    c, p, _, _ = stats.chi2_contingency(t, correction=False)
    print(f"  x{k}: n = {MS_N*k}/{OND_N*k}, P = {p:.4f}")

# ------------------------------------------------------------------ figure 1
BG, TXT, SUB = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
                     "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SUB, "ytick.color": TXT,
                     "svg.fonttype": "none"})

rows = [
    ("Reported in the paper (χ² test)", REPORTED_P, "rep"),
    ("Pearson χ², no correction", res["Pearson chi2, no correction"][1], "re"),
    ("Pearson χ², Yates correction", res["Pearson chi2, Yates correction"][1], "re"),
    ("Pearson χ², one-sided", res["Pearson chi2, one-sided"][1], "re"),
    ("G-test (likelihood ratio)", res["G-test (likelihood ratio)"][1], "re"),
    ("N−1 χ²", res["N-1 chi2"][1], "re"),
    ("Fisher exact, two-sided", res["Fisher exact, two-sided"][1], "re"),
    ("Fisher exact, one-sided", res["Fisher exact, one-sided"][1], "re"),
    ("Barnard exact, two-sided", res["Barnard exact, two-sided"][1], "re"),
    ("Boschloo exact, two-sided", res["Boschloo exact, two-sided"][1], "re"),
    ("Percentages entered as counts\n(42/100 vs 28/100), Pearson χ²", pct_res[(42, 28)][1], "pct"),
]

fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
ys[-1] -= 0.5        # separate the "percentages as counts" row
ys[1:-1] -= 0.25     # separate the reported row from the recomputed block
for (lab, p, kind), y in zip(rows, ys):
    if kind == "rep":
        ax.plot(p, y, "o", ms=10, color=ORANGE, zorder=3)
        ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
                    va="center", ha="right", color=ORANGE, fontsize=10, fontweight="bold")
    elif kind == "re":
        ax.plot(p, y, "o", ms=9, color=BLUE, zorder=3)
        ax.annotate(f"{p:.3f}", (p, y), xytext=(9, 0), textcoords="offset points",
                    va="center", color=BLUE, fontsize=10)
    else:
        ax.plot(p, y, "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
        ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
                    va="center", ha="right", color=BLUE, fontsize=10)
        ax.text(0.058, y, "matches the reported value,\nbut assumes 100 patients per group",
                va="center", ha="left", color=BLUE, fontsize=10)
ax.axvline(0.05, color=SUB, lw=1, ls="--", zorder=1)
ax.text(0.05, ys[0] + 0.75, "P = 0.05", color=SUB, fontsize=10, ha="center", va="bottom",
        bbox=dict(facecolor=BG, edgecolor="none", pad=1.5), zorder=2)
ax.axhspan(ys[-1] - 0.5, ys[-1] + 0.6, color="#efeeea", zorder=0)

ax.set_xscale("log")
ax.set_xlim(0.01, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_ylim(ys[-1] - 0.7, ys[0] + 1.3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for t, (_, _, kind) in zip(ax.get_yticklabels(), rows):
    t.set_color(ORANGE if kind == "rep" else TXT)
ax.set_xlabel("P value (log scale)", fontsize=10)
ax.grid(axis="x", color="#e4e3df", lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SUB)

from matplotlib.lines import Line2D
handles = [Line2D([], [], marker="o", ls="", ms=10, color=ORANGE, label="Reported in the paper"),
           Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
                  label="Recomputed from the stated counts 19/45 vs 10/35"),
           Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
                  label="Recomputed from the printed percentages as counts")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=1,
           frameon=False, fontsize=10)
fig.text(0.01, 0.80, "MS vs OND casein-specific B-cell responders (Results, ELISPOT)",
         fontsize=10, color=SUB)
fig.subplots_adjust(left=0.37, right=0.97, top=0.78, bottom=0.10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png to", HERE)

Abstract reports PGS-ADHD odds ratios above 1 as decreased odds; Results report them below 1

Psychol Med (2024) · doi:10.1017/S0033291724002186 · PMC11578915

Summary

The study fits multinomial regressions of four treatment trajectories on polygenic scores (PGS) in 10,577 people with early-onset depression, with the brief contact trajectory as the reference. The Abstract says higher PGS-ADHD was linked to "decreased odds" of the prolonged initial contact and persistent contact trajectories, but it prints odds ratios above 1 for both: 1.06 (1.01–1.11) and 1.12 (1.03–1.21). For the same comparisons the Results text and Supplementary Table 2 give 0.91 (0.87–0.96), p = 0.0002, and 0.90 (0.83–0.97), p = 0.007. All four intervals exclude 1, so the Abstract's numbers disagree with the body in direction and value, and the two Abstract triples are identical to the PGS-AN rows of Supplementary Table 2. The Abstract is the paper's most widely read summary, and its printed figures reverse the direction of the result the Discussion calls the strongest.

What the paper reports

Relative to the brief contact trajectory, higher PGS for ADHD was associated with a decreased odds of membership in the prolonged initial contact (odds ratio = 1.06, 95% confidence interval = 1.01–1.11) and persistent contact (1.12, 1.03–1.21) trajectories, while PGS-AN was associated with increased odds of membership in the persistent contact trajectory (1.12, 1.03–1.21).

— Abstract, Results

PGS-ADHD was associated with decreased odds of membership in the prolonged initial contact (OR = 0.91 [0.87–0.96], p = 0.0002) and persistent contact classes (OR = 0.90 [0.83–0.97], p = 0.007). PGS-AN was associated with increased odds of membership in the persistent contact class (OR = 1.12 [1.03–1.21], p = 0.005).

— Results, "PGS and trajectory patterns in secondary care", second paragraph

For each PGS and family history variable, we calculated the odds ratio (OR), 95% confidence intervals (CIs), and p-value for trajectory class membership relative to the brief contact class using multinomial regression models adjusted for sex, age-at-onset, and calendar year-at-diagnosis in three-year bands.

— Methods, "Statistical analysis"

Statistical significance was assessed at the Bonferroni-adjusted alpha of 0.0083 for the PGS (0.05/6 PGSs), and 0.017 for parental history (0.05/3 parental history variables).

— Methods, "Statistical analysis"

The brief contact class (class 1) was used as the reference category in the multinomial regressions.

— Figure 2 legend

The strongest results were for PGS-ADHD, which was associated with increased likelihood of a brief contact pattern relative to the prolonged initial contact and persistent contact patterns, particularly for younger individuals.

— Discussion

Supplementary Table 2 ("Associations between continuous polygenic scores (PGSs) and trajectory classes"; brief contact is the reference) lists, as OR, SE, p-value, lower CI, upper CI:

Prolonged initial contact | PGS-ADHD | 0.91 | 0.02 | 2.12x10-04 | 0.87 | 0.96 Persistent contact | PGS-ADHD | 0.90 | 0.04 | 6.99x10-03 | 0.83 | 0.97 Prolonged initial contact | PGS-AN | 1.06 | 0.02 | 3.06x10-02 | 1.01 | 1.11 Persistent contact | PGS-AN | 1.12 | 0.04 | 5.00x10-03 | 1.03 | 1.21

— Supplementary material (S0033291724002186sup001.docx), STable 2, rows for PGS-ADHD and PGS-AN

What the paper's own numbers show

Method. In a multinomial logistic model with brief contact as the reference, the odds ratio for class k is exp(β), where β is the change in log[P(k)/P(brief)] per SD of PGS. An OR above 1 therefore means increased odds of class k relative to brief contact, and an OR below 1 means decreased odds. A Wald 95% CI is exp(β ± 1.96 SE), so SE = (ln hi − ln lo)/3.92, z = ln(OR)/SE and the two-sided p follows. I applied this to every Abstract and Results triple. I also took reciprocals of the Results values, to test whether a swapped reference category could produce the Abstract's numbers, and searched all 18 rows of Supplementary Table 2, and the other trajectory-class tables of the supplement (STables 3, 4, 5 and 18), for the Abstract's exact OR and CI bounds and for any PGS-ADHD estimate above 1 for these two classes. Every number below is printed by recompute.py.txt (Python, numpy/scipy).

Quantity Location Reported Recomputed from the paper's numbers
PGS-ADHD, prolonged initial contact Abstract 1.06 (1.01–1.11), "decreased odds" OR > 1, so increased odds (+6% per SD); z = +2.42, p = 0.0155
PGS-ADHD, prolonged initial contact Results text; STable 2 0.91 [0.87–0.96], p = 0.0002 OR < 1, decreased odds (−9% per SD); z = −3.76, p = 0.0002
PGS-ADHD, persistent contact Abstract 1.12 (1.03–1.21), "decreased odds" OR > 1, so increased odds (+12% per SD); z = +2.76, p = 0.0058
PGS-ADHD, persistent contact Results text; STable 2 0.90 [0.83–0.97], p = 0.007 OR < 1, decreased odds (−10% per SD); z = −2.65, p = 0.0081
PGS-AN, persistent contact Abstract; Results text; STable 2 1.12 [1.03–1.21] (p = 0.005 in Results text and STable 2) z = +2.76, p = 0.0058; all three places agree
Reciprocal of Results PGS-ADHD, prolonged derived Abstract: 1.06 (1.01–1.11) 1.10 [1.04–1.15]; does not match
Reciprocal of Results PGS-ADHD, persistent derived Abstract: 1.12 (1.03–1.21) 1.11 [1.03–1.20]; does not match
Supplement rows equal to the Abstract's "ADHD, prolonged" triple STable 2; STable 5 1.06 (1.01–1.11) PGS-AN, prolonged initial contact, in STable 2 (printed p = 0.0306) and in STable 5 (printed p = 0.0281); no PGS-ADHD row
STable 2 rows equal to the Abstract's "ADHD, persistent" triple STable 2 1.12 (1.03–1.21) one row: PGS-AN, persistent contact
Abstract prolonged-contact value against the Bonferroni α Abstract; Methods α = 0.05/6 = 0.0083 p = 0.0155 from the CI (0.0306 as printed in STable 2 for the matching PGS-AN row); neither passes
PGS-ADHD in the mutually adjusted model STable 5 0.90 [0.86–0.95] and 0.91 [0.84–0.99] both below 1
PGS-ADHD for these two classes in the age strata and top-decile/5% contrasts STable 18; STables 3–4 18 estimates 16 below 1; 2 point estimates above 1 (age 21–25, persistent contact, 1.13 [0.98–1.31], p = 0.0968; top 5% vs bottom 5%, persistent contact, 1.04 [0.62–1.74]), both with CIs that include 1; none has a CI wholly above 1 and none matches an Abstract triple

The Results-text and STable 2 values are internally consistent: the p values recovered from their CIs (0.0002 and 0.0081) match the printed 0.0002 and 0.007 to within rounding of the two-decimal bounds. The Abstract's two "ADHD" triples are not consistent with the word "decreased" in the same sentence, with the Results text, or with any reciprocal of it. Each matches, digit for digit, a PGS-AN row of Supplementary Table 2.

Figure

Abstract and Results text side by side: the same two PGS-ADHD comparisons are printed with odds ratios above 1 in the Abstract and below 1 in the Results

Figure 1. The two PGS-ADHD comparisons against the brief contact trajectory, as printed in the Abstract (left) and in the Results text (right). All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn. The Abstract prints OR 1.06 (1.01–1.11) for prolonged initial contact and 1.12 (1.03–1.21) for persistent contact, both above 1. The Results text, which agrees with Supplementary Table 2, prints 0.91 [0.87–0.96] (p = 0.0002) and 0.90 [0.83–0.97] (p = 0.007), both below 1. Each pair of intervals excludes 1 on opposite sides, so the two sources disagree in direction, not only in value. Both sources describe the association in words as "decreased odds", which fits only the Results values. The Abstract's two triples are identical to the PGS-AN rows of Supplementary Table 2 for the same classes.

Why it matters

The PGS-ADHD association is the result the Discussion singles out as "the strongest". The Abstract is what most readers, indexers and citing authors see, and it is the only part of the paper whose numbers point the other way. A reader who takes the Abstract's numbers at face value would conclude that higher PGS-ADHD raises the odds of the longer-treatment trajectories, by about 6% and 12% per SD. The body says the opposite: it lowers them, by about 9% and 10% per SD. The Abstract's prolonged-contact value would also fail the paper's own multiple-testing threshold (p = 0.0155 from its CI, against α = 0.0083), whereas the body's value passes (p = 0.0002). Anyone pooling these estimates in a meta-analysis or citing them from the Abstract would carry the wrong sign.

The error is confined to the Abstract's numbers. The Abstract's wording, the Results text, Figure 2 (brief contact as reference), Supplementary Tables 2 and 5, the Discussion and the Conclusions all agree with ORs below 1, and the age-stratified results describe "the negative association between PGS-ADHD and the persistent contact class" as driven by ages 10–20 (STable 18: 0.81 [0.71–0.94] at 10–15 and 0.84 [0.74–0.95] at 16–20). The paper's main conclusions are therefore not affected.

Explanations considered

No explanation found reconciles the Abstract's numbers with the rest of the paper.

Status

Suggested correction

In the Abstract, Results paragraph, replace "(odds ratio = 1.06, 95% confidence interval = 1.01–1.11)" with "(odds ratio = 0.91, 95% confidence interval = 0.87–0.96)" and "persistent contact (1.12, 1.03–1.21) trajectories" with "persistent contact (0.90, 0.83–0.97) trajectories", matching the Results text and Supplementary Table 2. The PGS-AN estimate in the same sentence (1.12, 1.03–1.21) is correct and needs no change. Separately, the Bonferroni alpha in the footnotes of Supplementary Tables 2 and 5 could be corrected from 0.00083 to 0.0083 to match the Methods.

Recomputation script (python recompute.py.txt)
"""Recompute the PGS-ADHD abstract-versus-Results discrepancy in PMC11578915
(Psychol Med 2024, doi:10.1017/S0033291724002186) and regenerate the figures.

Run:   python recompute.py.txt
Needs: numpy, scipy, matplotlib.

Prints every recomputed number used in writeup.md. Writes, into the folder that
holds this file:
  candidates/cand_A..cand_E (.svg/.png)  five alternative designs for Figure 1
  fig1.svg / fig1.png                    a copy of the design named in FIG1 below

All inputs are copied from the paper: the Abstract, the Results section
("PGS and trajectory patterns in secondary care"), the Methods (Bonferroni
alpha) and Supplementary Table 2 (STable 2; continuous PGS vs trajectory class,
brief contact = reference). STable 5 (mutually adjusted model), STable 18
(age-stratified models) and STables 3-4 (top 10% / top 5% contrasts) are used only as
further checks.

Method: Wald inference on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964),
z = ln(OR) / SE and two-sided p = 2 * (1 - Phi(|z|)).
"""
import os
import shutil

import numpy as np
from scipy.stats import norm

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
FIG1 = "A"          # which candidate design is copied to fig1.*

Z = norm.ppf(0.975)
ALPHA_BONF = 0.05 / 6   # Methods: "0.0083 for the PGS (0.05/6 PGSs)"


def wald(o, lo, hi):
    se = (np.log(hi) - np.log(lo)) / (2 * Z)
    z = np.log(o) / se
    return se, z, 2 * norm.sf(abs(z))


line = "-" * 78

# ------------------------------------------------------------------ inputs
ABSTRACT = {  # label: (OR, lo, hi)  -- Abstract, Results paragraph
    "ADHD, prolonged initial contact": (1.06, 1.01, 1.11),
    "ADHD, persistent contact": (1.12, 1.03, 1.21),
    "AN, persistent contact": (1.12, 1.03, 1.21),
}
RESULTS = {  # label: (OR, lo, hi, printed p) -- Results text
    "ADHD, prolonged initial contact": (0.91, 0.87, 0.96, 0.0002),
    "ADHD, persistent contact": (0.90, 0.83, 0.97, 0.007),
    "AN, persistent contact": (1.12, 1.03, 1.21, 0.005),
}
# STable 2: (class, PGS): (OR, SE, p, lo, hi)
ST2 = {
    ("Prolonged initial contact", "MDD"): (0.96, 0.02, 6.60e-02, 0.91, 1.00),
    ("Later re-entry", "MDD"): (1.09, 0.04, 1.47e-02, 1.02, 1.17),
    ("Persistent contact", "MDD"): (0.95, 0.04, 1.89e-01, 0.88, 1.03),
    ("Prolonged initial contact", "ADHD"): (0.91, 0.02, 2.12e-04, 0.87, 0.96),
    ("Later re-entry", "ADHD"): (1.00, 0.04, 9.32e-01, 0.93, 1.07),
    ("Persistent contact", "ADHD"): (0.90, 0.04, 6.99e-03, 0.83, 0.97),
    ("Prolonged initial contact", "ASD"): (1.01, 0.02, 5.79e-01, 0.97, 1.06),
    ("Later re-entry", "ASD"): (1.08, 0.04, 3.15e-02, 1.01, 1.16),
    ("Persistent contact", "ASD"): (0.95, 0.04, 2.20e-01, 0.88, 1.03),
    ("Prolonged initial contact", "BD"): (1.03, 0.02, 2.00e-01, 0.98, 1.08),
    ("Later re-entry", "BD"): (0.99, 0.04, 8.68e-01, 0.92, 1.07),
    ("Persistent contact", "BD"): (0.99, 0.04, 7.15e-01, 0.91, 1.07),
    ("Prolonged initial contact", "SCZ"): (0.99, 0.02, 6.37e-01, 0.94, 1.04),
    ("Later re-entry", "SCZ"): (1.05, 0.04, 2.31e-01, 0.97, 1.12),
    ("Persistent contact", "SCZ"): (0.99, 0.04, 7.83e-01, 0.91, 1.07),
    ("Prolonged initial contact", "AN"): (1.06, 0.02, 3.06e-02, 1.01, 1.11),
    ("Later re-entry", "AN"): (1.02, 0.04, 5.99e-01, 0.95, 1.10),
    ("Persistent contact", "AN"): (1.12, 0.04, 5.00e-03, 1.03, 1.21),
}
ST5_ADHD = {  # mutually adjusted model, STable 5
    "prolonged initial contact": (0.90, 0.86, 0.95, 1.93e-04),
    "persistent contact": (0.91, 0.84, 0.99, 3.28e-02),
}
ST5_AN_PRO = (1.06, 1.01, 1.11, 2.81e-02)   # STable 5, PGS-AN, prolonged initial contact
# Every other PGS-ADHD estimate in the supplement for the two classes at issue
# (source table, model/subgroup, class): (OR, lo, hi, printed p)
OTHER_ADHD = {
    ("STable 18", "age 10-15", "prolonged initial contact"): (0.85, 0.77, 0.95, 2.83e-03),
    ("STable 18", "age 10-15", "persistent contact"): (0.81, 0.71, 0.94, 3.78e-03),
    ("STable 18", "age 16-20", "prolonged initial contact"): (0.95, 0.88, 1.01, 1.17e-01),
    ("STable 18", "age 16-20", "persistent contact"): (0.84, 0.74, 0.95, 5.03e-03),
    ("STable 18", "age 21-25", "prolonged initial contact"): (0.90, 0.82, 0.99, 3.01e-02),
    ("STable 18", "age 21-25", "persistent contact"): (1.13, 0.98, 1.31, 9.68e-02),
    ("STable 3", "top 10% vs bottom 10%", "prolonged initial contact"): (0.70, 0.56, 0.87, 1.60e-03),
    ("STable 3", "top 10% vs bottom 10%", "persistent contact"): (0.71, 0.50, 1.00, 5.17e-02),
    ("STable 3", "top 10% vs bottom 90%", "prolonged initial contact"): (0.83, 0.70, 0.99, 3.49e-02),
    ("STable 3", "top 10% vs bottom 90%", "persistent contact"): (0.88, 0.67, 1.15, 3.35e-01),
    ("STable 3", "top 10% vs bottom 50%", "prolonged initial contact"): (0.79, 0.66, 0.94, 8.95e-03),
    ("STable 3", "top 10% vs bottom 50%", "persistent contact"): (0.80, 0.61, 1.06, 1.22e-01),
    ("STable 4", "top 5% vs bottom 5%", "prolonged initial contact"): (0.70, 0.51, 0.96, 2.80e-02),
    ("STable 4", "top 5% vs bottom 5%", "persistent contact"): (1.04, 0.62, 1.74, 8.77e-01),
    ("STable 4", "top 5% vs bottom 95%", "prolonged initial contact"): (0.82, 0.65, 1.04, 1.04e-01),
    ("STable 4", "top 5% vs bottom 95%", "persistent contact"): (0.97, 0.68, 1.39, 8.82e-01),
    ("STable 4", "top 5% vs bottom 50%", "prolonged initial contact"): (0.77, 0.61, 0.99, 3.80e-02),
    ("STable 4", "top 5% vs bottom 50%", "persistent contact"): (0.88, 0.61, 1.28, 5.12e-01),
}

# ------------------------------------------------------------------ 1. abstract values
print(line)
print(f"Bonferroni alpha for the PGS = 0.05/6 = {ALPHA_BONF:.5f}")
print(line)
print("1. Abstract values: direction and p implied by each printed CI")
abs_calc = {}
for k, (o, lo, hi) in ABSTRACT.items():
    se, z, p = wald(o, lo, hi)
    abs_calc[k] = (se, z, p)
    print(f"   {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}]  direction "
          f"{'increased' if o > 1 else 'decreased'}; SE {se:.4f}; z {z:+.2f}; "
          f"p {p:.4f}; passes Bonferroni: {p < ALPHA_BONF}; "
          f"change in odds per SD {100 * (o - 1):+.0f}%")

# ------------------------------------------------------------------ 2. Results values
print(line)
print("2. Results-text values: p implied by each printed CI vs printed p")
res_calc = {}
for k, (o, lo, hi, pr) in RESULTS.items():
    se, z, p = wald(o, lo, hi)
    res_calc[k] = (se, z, p)
    print(f"   {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr}  | "
          f"SE {se:.4f}; z {z:+.2f}; implied p {p:.4f}; direction "
          f"{'increased' if o > 1 else 'decreased'}; change in odds per SD {100 * (o - 1):+.0f}%")

# ------------------------------------------------------------------ 3. reference swap
print(line)
print("3. Reciprocals of the Results ADHD values (what a swapped reference category would give)")
recip = {}
for k in ("ADHD, prolonged initial contact", "ADHD, persistent contact"):
    o, lo, hi, _ = RESULTS[k]
    r = (1 / o, 1 / hi, 1 / lo)
    recip[k] = r
    a = ABSTRACT[k]
    print(f"   {k:32s} 1/OR = {r[0]:.2f} [{r[1]:.2f}-{r[2]:.2f}]   abstract {a[0]:.2f} "
          f"[{a[1]:.2f}-{a[2]:.2f}]   matches to 2 dp: "
          f"{tuple(round(x, 2) for x in r) == a}")

# ------------------------------------------------------------------ 4. matching to STable 2
print(line)
print("4. Which STable 2 rows carry exactly the abstract's OR and CI (2 dp)?")
matches = {}
for k, (o, lo, hi) in ABSTRACT.items():
    m = [f"{c} / PGS-{g}" for (c, g), v in ST2.items() if (v[0], v[3], v[4]) == (o, lo, hi)]
    matches[k] = m
    print(f"   abstract {k:32s} {o:.2f} [{lo:.2f}-{hi:.2f}] -> {m}")
print("   STable 2 rows labelled PGS-ADHD for the same classes:")
for c in ("Prolonged initial contact", "Persistent contact"):
    v = ST2[(c, "ADHD")]
    print(f"     {c:28s} OR {v[0]:.2f} [{v[3]:.2f}-{v[4]:.2f}] p {v[2]:.2e}")
print("   STable 2 printed SE vs SE recovered from its CI, and printed p vs implied p:")
for (c, g), (o, se_p, p_p, lo, hi) in ST2.items():
    if g in ("ADHD", "AN") and c != "Later re-entry":
        se, z, p = wald(o, lo, hi)
        print(f"     {c:28s} PGS-{g:4s} printed SE {se_p:.2f} vs {se:.3f}; printed p {p_p:.2e} vs {p:.2e}")
n_any = sum(1 for v in ST2.values() if (v[0], v[3], v[4]) == ABSTRACT["ADHD, prolonged initial contact"])
print(f"   rows in STable 2 equal to 1.06 [1.01-1.11]: {n_any}")

# ------------------------------------------------------------------ 5. mutually adjusted model
print(line)
print("5. STable 5 (mutually adjusted) PGS-ADHD rows")
for k, (o, lo, hi, pr) in ST5_ADHD.items():
    se, z, p = wald(o, lo, hi)
    print(f"   {k:28s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.2e}; below 1: {hi < 1}")
o, lo, hi, pr = ST5_AN_PRO
print(f"   STable 5 PGS-AN, prolonged initial contact: OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; "
      f"equals the Abstract's 'ADHD, prolonged' triple: {(o, lo, hi) == ABSTRACT['ADHD, prolonged initial contact']}")

# ------------------------------------------------------------------ 6. other PGS-ADHD estimates
print(line)
print("6. Other PGS-ADHD estimates for the two classes (STable 18 age strata, STable 3 and 4 top-10%/5% contrasts)")
abs_triples = {ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]}
for (tab, sub, c), (o, lo, hi, pr) in OTHER_ADHD.items():
    se, z, p = wald(o, lo, hi)
    print(f"   {tab:9s} {sub:22s} {c:26s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.3f}; "
          f"OR>1 {o > 1}; CI wholly above 1 {lo > 1}; equals an Abstract triple {(o, lo, hi) in abs_triples}")
n_gt1 = sum(v[0] > 1 for v in OTHER_ADHD.values())
n_ci_gt1 = sum(v[1] > 1 for v in OTHER_ADHD.values())
n_match = sum(v[:3] in abs_triples for v in OTHER_ADHD.values())
print(f"   point estimates above 1: {n_gt1}; CIs wholly above 1: {n_ci_gt1}; equal to an Abstract triple: {n_match}")
print(line)

# ================================================================== figures
import matplotlib  # noqa: E402

matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
LW, MS = 1.5, 8   # 1.5 pt = 2 px; marker 8 pt
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
    plt.close(fig)


def fmt(t):
    return f"{t[0]:.2f} [{t[1]:.2f}–{t[2]:.2f}]"


def log_axis(ax, lo=0.78, hi=1.30, ticks=(0.8, 0.85, 0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2)):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    ticks = [t for t in ticks if lo <= t <= hi]
    ax.set_xticks(ticks)
    ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.minorticks_off()
    ax.grid(axis="x", color=GRID, lw=0.6)
    ax.set_axisbelow(True)
    ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


A_PRO, A_PER = ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]
R_PRO, R_PER = RESULTS["ADHD, prolonged initial contact"], RESULTS["ADHD, persistent contact"]
AN_PRO = ST2[("Prolonged initial contact", "AN")]
AN_PER = ST2[("Persistent contact", "AN")]

# ------------------------------------------------------------------ A: two text panels
def cand_A():
    fig = plt.figure(figsize=(W, 4.3))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
    ax.set_xlim(0, 100); ax.set_ylim(0, 60)

    def panel(x, w, title, sub):
        ax.add_patch(FancyBboxPatch((x, 4), w, 50, boxstyle="round,pad=0,rounding_size=1.2",
                                    fc="white", ec=INK2, lw=0.8))
        ax.text(x + 2, 50.5, title, fontsize=11.5, weight="bold", va="top")
        ax.text(x + 2, 46.3, sub, fontsize=10, color=INK2, va="top")

    panel(1, 38, "Abstract", "Results paragraph")
    panel(57, 42, "Results text", "“PGS and trajectory patterns”")
    ax.text(50, 58.5, "PGS-ADHD vs the brief contact trajectory (reference): the same two comparisons",
            ha="center", fontsize=10.5, va="top")
    rows = [("Prolonged initial contact", A_PRO, (R_PRO[0], R_PRO[1], R_PRO[2]), f"p = {R_PRO[3]}", 34),
            ("Persistent contact", A_PER, (R_PER[0], R_PER[1], R_PER[2]), f"p = {R_PER[3]}", 19)]
    for lab, a, r, ptxt, y in rows:
        ax.text(3, y + 4.2, lab, fontsize=10, color=INK2)
        ax.text(3, y, f"OR {fmt(a)}", fontsize=12.5, color=ORANGE, weight="bold")
        ax.text(3, y - 4, "OR > 1: increased odds", fontsize=10, color=INK2)
        ax.text(59.3, y + 4.2, lab, fontsize=10, color=INK2)
        ax.text(59.3, y, f"OR {fmt(r)}", fontsize=12.5, color=ORANGE, weight="bold")
        ax.text(59.3, y - 4, f"{ptxt}; OR < 1: decreased odds", fontsize=10, color=INK2)
        ax.annotate("", xy=(56.6, y + 1), xytext=(39.4, y + 1),
                    arrowprops=dict(arrowstyle="<->", color=INK, lw=LW))
        ax.text(48, y + 2.6, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.1)
        ax.text(48, y - 1.0, "both CIs\nexclude 1", ha="center", va="top", fontsize=10, color=INK2, linespacing=1.1)
    ax.text(3, 9.8, "Wording: “decreased odds”", fontsize=10)
    ax.text(3, 6.0, "Values equal the PGS-AN rows", fontsize=10, color=INK2)
    ax.text(59.3, 9.8, "Wording: “decreased odds”", fontsize=10)
    ax.text(59.3, 6.0, "Matches PGS-ADHD rows of STable 2", fontsize=10, color=INK2)
    ax.text(1, 1.0, "Orange: values as printed in the paper.", fontsize=10, color=INK2)
    save(fig, "cand_A")


# ------------------------------------------------------------------ B: dot-and-interval, 2 rows
def cand_B():
    fig, ax = plt.subplots(figsize=(W, 3.4))
    log_axis(ax, 0.80, 1.36)
    items = [  # y, (o, lo, hi), style
        (3.25, A_PRO, "abs"), (2.95, (R_PRO[0], R_PRO[1], R_PRO[2]), "res"), (2.65, recip["ADHD, prolonged initial contact"], "rec"),
        (1.35, A_PER, "abs"), (1.05, (R_PER[0], R_PER[1], R_PER[2]), "res"), (0.75, recip["ADHD, persistent contact"], "rec"),
    ]
    for y, (o, lo, hi), st in items:
        col = BLUE if st == "rec" else ORANGE
        mk = {"abs": "D", "res": "o", "rec": "s"}[st]
        fc = "white" if st == "abs" else col
        ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="round", zorder=2)
        ax.plot(o, y, mk, ms=MS, mfc=fc, mec=col, mew=LW, zorder=3)
        ax.text(1.232, y, fmt((o, lo, hi)), va="center", fontsize=10, color=INK)
    ax.set_yticks([2.95, 1.05])
    ax.set_yticklabels(["Prolonged\ninitial contact", "Persistent\ncontact"], fontsize=10.5, color=INK)
    ax.set_ylim(0.4, 3.75)
    ax.set_xlabel("Odds ratio per SD of PGS-ADHD vs brief contact (log scale)")
    ax.text(0.995, 3.68, "← decreased odds", ha="right", va="top", fontsize=10, color=INK2)
    ax.text(1.005, 3.68, "increased odds →", ha="left", va="top", fontsize=10, color=INK2)
    h = [Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, mew=LW, label="Abstract (printed)"),
         Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Results text and STable 2 (printed)"),
         Line2D([], [], color=BLUE, marker="s", ms=MS, lw=LW, label="1/OR of Results values (recomputed)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_B")


# ------------------------------------------------------------------ C: source-matching diagram
def cand_C():
    fig = plt.figure(figsize=(W, 4.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
    ax.set_xlim(0, 100); ax.set_ylim(0, 56)
    ax.text(2, 53, "Abstract, as printed", fontsize=11.5, weight="bold", va="top")
    ax.text(58, 53, "Supplementary Table 2, as printed", fontsize=11.5, weight="bold", va="top")
    left = [("PGS-ADHD, prolonged", A_PRO, 40), ("PGS-ADHD, persistent", A_PER, 28), ("PGS-AN, persistent", ABSTRACT["AN, persistent contact"], 16)]
    right = {("Prolonged initial contact", "ADHD"): 44, ("Persistent contact", "ADHD"): 35,
             ("Prolonged initial contact", "AN"): 23, ("Persistent contact", "AN"): 14}
    for (c, g), y in right.items():
        v = ST2[(c, g)]
        ax.add_patch(FancyBboxPatch((58, y - 3.2), 40, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
                                    fc="white", ec=INK2, lw=0.8))
        ax.text(60, y + 0.9, f"PGS-{g}, {c.split()[0].lower()}", fontsize=10, color=INK2, va="center")
        ax.text(60, y - 1.5, f"OR {v[0]:.2f} [{v[3]:.2f}–{v[4]:.2f}]", fontsize=10.5, color=ORANGE, weight="bold", va="center")
    for lab, t, y in left:
        ax.add_patch(FancyBboxPatch((2, y - 3.2), 36, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
                                    fc="white", ec=INK2, lw=0.8))
        ax.text(4, y + 0.9, lab, fontsize=10, color=INK2, va="center")
        ax.text(4, y - 1.5, f"OR {fmt(t)}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
    # identical-value links
    for yl, yr in ((40, 23), (28, 14), (16, 14)):
        ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
                    arrowprops=dict(arrowstyle="-|>", color=INK, lw=LW, shrinkA=0, shrinkB=0))
    # rows with the same label
    for yl, yr in ((40, 44), (28, 35)):
        ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
                    arrowprops=dict(arrowstyle="-", color=INK2, lw=1.0, ls=(0, (3, 2)), shrinkA=0, shrinkB=0))
    h = [Line2D([], [], color=INK, lw=LW, label="identical OR and both CI bounds"),
         Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="same PGS and class label")]
    ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.01), ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_C")


# ------------------------------------------------------------------ D: forest of STable 2 with abstract overlay
def cand_D():
    order = ["MDD", "ADHD", "ASD", "BD", "SCZ", "AN"]
    classes = ["Prolonged initial contact", "Persistent contact"]
    fig, axs = plt.subplots(1, 2, figsize=(W, 3.9), sharey=True)
    for ax, c in zip(axs, classes):
        log_axis(ax, 0.80, 1.26, ticks=(0.8, 0.9, 1.0, 1.1, 1.2))
        for i, g in enumerate(order):
            y = len(order) - i
            o, _, p, lo, hi = ST2[(c, g)]
            ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="round", zorder=2)
            ax.plot(o, y, "o", ms=MS, color=ORANGE, zorder=3)
        a = A_PRO if c.startswith("Prolonged") else A_PER
        yA = len(order) - order.index("ADHD") + 0.38
        ax.plot([a[1], a[2]], [yA, yA], color=ORANGE, lw=LW, ls=(0, (2, 1.5)), zorder=2)
        ax.plot(a[0], yA, "D", ms=MS, mfc="white", mec=ORANGE, mew=LW, zorder=3)
        yAN = len(order) - order.index("AN")
        ax.annotate("", xy=(a[0], yAN + 0.22), xytext=(a[0], yA - 0.2),
                    arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.0, ls=(0, (3, 2))))
        ax.set_title(c, fontsize=10.5, color=INK)
        ax.set_xlabel("OR per SD vs brief contact (log)")
    axs[0].set_yticks(range(len(order), 0, -1))
    axs[0].set_yticklabels([f"PGS-{g}" for g in order], fontsize=10, color=INK)
    axs[0].set_ylim(0.4, 6.9)
    h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="STable 2 (printed)"),
         Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, ls=(0, (2, 1.5)), mew=LW,
                label="Abstract's PGS-ADHD value (printed)"),
         Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="lands on the PGS-AN row")]
    fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.07), ncol=3, frameon=False, fontsize=10)
    fig.tight_layout(rect=(0, 0.06, 1, 1))
    save(fig, "cand_D")


# ------------------------------------------------------------------ E: p-value strip
def cand_E():
    fig, ax = plt.subplots(figsize=(W, 3.2))
    rows = [
        ("Prolonged, Abstract\n1.06 [1.01–1.11]", None, abs_calc["ADHD, prolonged initial contact"][2]),
        ("Prolonged, Results\n0.91 [0.87–0.96]", R_PRO[3], res_calc["ADHD, prolonged initial contact"][2]),
        ("Persistent, Abstract\n1.12 [1.03–1.21]", None, abs_calc["ADHD, persistent contact"][2]),
        ("Persistent, Results\n0.90 [0.83–0.97]", R_PER[3], res_calc["ADHD, persistent contact"][2]),
    ]
    ax.set_xscale("log"); ax.set_xlim(5e-5, 0.2)
    ax.set_xticks([1e-4, 1e-3, 1e-2, 1e-1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1"]); ax.minorticks_off()
    ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.axvline(ALPHA_BONF, color=INK2, lw=0.8, ls=(0, (4, 2)))
    ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (1, 2)))
    ax.text(ALPHA_BONF / 1.08, 4.62, "Bonferroni α 0.0083", fontsize=10, color=INK2, va="top", ha="right")
    ax.text(0.05 * 1.08, 4.62, "0.05", fontsize=10, color=INK2, va="top")
    for i, (lab, pr, pc) in enumerate(rows):
        y = len(rows) - i
        if pr is not None:
            ax.plot(pr, y, "o", ms=MS, color=ORANGE, zorder=3)
        ax.plot(pc, y, "D", ms=MS, color=BLUE, zorder=4, mec="white", mew=1.0)
        ax.text(max(pc, pr or 0) * 1.35, y - 0.05, f"{pc:.4f}", fontsize=10, va="center", color=INK)
    ax.set_yticks(range(len(rows), 0, -1)); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
    ax.set_ylim(0.4, 4.7)
    ax.set_xlabel("Two-sided p for PGS-ADHD vs brief contact (log scale)")
    h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=0, label="p printed in Results"),
         Line2D([], [], color=BLUE, marker="D", ms=MS, lw=0, label="p recomputed from the printed CI")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
    save(fig, "cand_E")


# Which values each design draws, so the Figure 1 caption can name them correctly
CAPTION_KEY = {
    "A": "All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn.",
    "B": "Orange: odds ratios and 95% CIs as printed (Abstract; Results text and STable 2). Blue: reciprocals of the Results values, recomputed here to show what a swapped reference category would give.",
    "C": "All numbers are copied from the paper (orange: Abstract and STable 2 as printed); connectors mark identical OR-CI triples and shared labels.",
    "D": "All numbers are copied from the paper (orange: STable 2 rows, and the Abstract's PGS-ADHD values as open diamonds).",
    "E": "Orange: p values as printed in the Results text (the Abstract prints none). Blue: two-sided p recomputed from each printed 95% CI.",
}
print(f"Figure 1 caption key for candidate {FIG1}: {CAPTION_KEY[FIG1]}")

for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
    f()
for ext in ("svg", "png"):
    shutil.copyfile(os.path.join(CAND, f"cand_{FIG1}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"figures written to {CAND}; fig1 = candidate {FIG1}")

Cohort 2 is reported as 361,258 participants in Table 2 but 32,220 in the flowchart

Psychol Med (2026) · doi:10.1017/S0033291726103328 · PMC12902173

Summary

The paper analyses two matched cohorts from Taiwan's one-million-person Longitudinal Health Insurance Database (LHID): psychological distress (PD, meaning depression or anxiety) followed for incident IBS (Cohort 1), and IBS followed for incident PD (Cohort 2). For Cohort 2, the Results text and Table 2 describe 361,258 participants (180,629 per arm), while the sample-selection flowchart (Figure 1) gives a final follow-up N of 32,220 (16,110 per arm), 11.2 times smaller. The paper's own follow-up event percentages (2,031 = 12.6% and 1,308 = 8.1%) fit only the flowchart's 16,110 per arm, and the width of the headline hazard-ratio interval fits the roughly 3,300 events in that smaller sample. Table 2 therefore cannot describe the cohort that was analysed. It also reports 45.77% of the IBS arm "With PD" at baseline in a cohort defined by excluding prior PD. Cohort 1 shows the same pattern (Table 1: 234,361 per arm; flowchart: 18,402 per arm).

What the paper reports

Results, "Baseline characteristics: Cohort 2 (IBS predicting psychological distress)":

Table 2 summarizes the baseline characteristics of 361,258 participants, divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).

Table 2, column headers and the first block (rows "Without PD" and "With PD"):

Variable | IBS(n = 180,629) | Without IBS(n = 180,629) | p-value

Without PD | 97,948 (54.23%) | 148,971 (82.47%) |

With PD | 82,681 (45.77%) | 31,658 (17.53%) |

Results, the same section, "Psychological distress outcomes":

The prevalence of psychological distress was substantially higher among individuals with IBS (45.77%) compared with controls (17.53%, p < .001).

Methods, "Cohort 2: IBS predicting psychological distress", on the definition of the cohort:

Those with any depressive or anxiety disorder before the index date were excluded to ensure incident outcome assessment.

A consolidated flowchart including all exclusions (prior IBS, prior psychological distress, and missing data) and final analytic sample sizes is shown in Figure 1.

Figure 1 flowchart, Cohort 2 (list items, rendered as text in PMC and in the publisher's HTML):

Newly diagnosed IBS cases (2002–2012): 16,110

Excluded prior depressive or anxiety disorders: 3,231 excluded

IBS cohort: 16,110

Matched controls: 16,110 Final follow-up N for PD outcomes: 32,220

Results, "Cohort 2: IBS predicting incident depression or anxiety":

In the reverse analysis, during a mean follow-up of 6.9 years, 2,031 IBS patients (12.6%) developed either depression or anxiety, compared to 1,308 (8.1%) in the control group.

Cohort 1 has the same kind of discrepancy. Results, "Baseline characteristics: Cohort 1":

A total of 468,722 participants were included, with 234,361 individuals in each group after matching.

Figure 1 flowchart, Cohort 1:

Newly diagnosed depressive/anxiety disorders (2002–2012): 18,402

Psychological distress group: 18,402

Matched controls: 18,402 Final follow-up N for IBS outcome: 36,804

Results, Cohort 1 follow-up, and the Cohort 1 baseline IBS prevalence:

During a mean follow-up period of 7.3 years, 1,498 individuals (8.1%) in the psychological distress group developed IBS, compared to 921 (5.0%) in the control group.

The prevalence of IBS at baseline was significantly higher in the psychological distress group compared to controls (32.11% vs. 12.04%, p < .001)

The Cohort 1 definition (Methods) and the Discussion:

Individuals with a prior diagnosis of IBS (ICD-9-CM: 564.1) before the index date were excluded to ensure incident outcome assessment.

Our findings apply only to incident cases, as individuals with any history of IBS or prior depressive/anxiety disorders were excluded from both cohorts.

The large sample enabled robust subgroup and interaction analyses.

What the paper's own numbers show

Method. I took every sample size, count and percentage for both cohorts from the Results text, the Figure 1 flowchart text, Tables 1–4 and the Methods description of the LHID. I checked (i) each stated total against twice the per-arm N; (ii) that each block of Table 2 sums to its column header; (iii) which denominators reproduce the printed follow-up event percentages, by finding every per-arm N for which the printed event count divided by N rounds to the printed percentage; (iv) the flowchart's own subtraction; (v) the number of events implied by the width of each overall adjusted hazard ratio's 95% CI. For (v) I used the standard approximation for a 1:1 comparison, var(log HR) ≈ 4/D, and also computed the interval expected from the reported event counts, SE ≈ √(1/d₁ + 1/d₂), and the interval expected if the Table N had produced events at the printed rates. Finally, I compared the cohort sizes with the size of the LHID. Patient-level data are not available, so the models cannot be refitted. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Cohort 2 total N Results text; Table 2 headers 361,258 (180,629 per arm) 180,629 × 2 = 361,258, internally consistent
Cohort 2 final follow-up N Figure 1 flowchart 32,220 (16,110 per arm) 16,110 × 2 = 32,220, internally consistent
Ratio of the two Cohort 2 totals — — 361,258 / 32,220 = 11.21; difference 329,038 (164,519 per arm)
Table 2 block sums (PD status, sex, age group, urbanization, income) Table 2 n = 180,629 per column every block sums to 180,629 in both columns
Depression + anxiety = "With PD" Table 2 82,681 and 31,658 15,806 + 66,875 = 82,681; 7,339 + 24,319 = 31,658
IBS-arm follow-up events Results 2,031 (12.6%) 2,031 / 16,110 = 12.61%; 2,031 / 180,629 = 1.12%
Control-arm follow-up events Results 1,308 (8.1%) 1,308 / 16,110 = 8.12%; 1,308 / 180,629 = 0.72%
Per-arm N that reproduces both printed percentages derived — 16,056 to 16,183: contains 16,110, excludes 180,629
IBS cohort after excluding prior PD Figure 1 flowchart 16,110 new cases, 3,231 excluded, cohort 16,110 16,110 − 3,231 = 12,879, not 16,110
IBS arm "With PD" at baseline Table 2; Results 82,681 (45.77%) 82,681 / 180,629 = 45.77%, in a cohort that excluded prior PD; 25.6 times the flowchart's 3,231 exclusions and 40.7 times the 2,031 follow-up events
Cohort 2 aHR 1.46 (1.37–1.56): events implied by CI width Results; Table 4 — SE(log HR) = 0.0331; 4/SE² ≈ 3,644 events (3,288–4,061 allowing for rounding of the limits); reported events 2,031 + 1,308 = 3,339, inside that range
CI expected from 3,339 events derived 1.37–1.56 1.36–1.57 (matches)
CI expected if 180,629 per arm had the printed rates derived — 22,759 + 14,631 = 37,390 events; CI 1.43–1.49 (does not match; the reported interval is 3.1 times as wide)
Cohort 1 total N Results text; Table 1 headers 468,722 (234,361 per arm) 234,361 × 2 = 468,722
Cohort 1 final follow-up N Figure 1 flowchart 36,804 (18,402 per arm) 18,402 × 2 = 36,804; ratio 468,722 / 36,804 = 12.74
Cohort 1 follow-up events Results 1,498 (8.1%) and 921 (5.0%) /18,402 = 8.14% and 5.00%; /234,361 = 0.64% and 0.39%; N reproducing both: 18,381–18,606
Cohort 1 aHR 1.55 (1.42–1.69): events implied by CI width Results; Table 3 — ≈ 2,028 (1,885–2,188), slightly fewer than the 2,419 reported (1,498 + 921), i.e. the reported interval is a little wider than the events alone imply; both are far from the 30,701 implied by the Table N. CI from 2,419 events 1.43–1.68; from Table N at printed rates (30,701 events) 1.51–1.59, 3.8 times narrower than reported
PD arm with IBS at baseline Table 1; Results 75,244 (32.11%) 75,244 / 234,361 = 32.11%, in a cohort that excluded prior IBS
Adults in the LHID Methods 1,000,000 − 1,982 − 2,614 995,404
Cohort 1 eligible population Figure 1 flowchart "990,0xx" 995,404 − 5,412 = 989,992, which is not of the form 990,0xx either
Exposed arms of both cohorts Tables 1 and 2 234,361 + 180,629 414,990 = 41.7% of LHID adults
Exposed arms of both cohorts Figure 1 flowchart 18,402 + 16,110 34,512 = 3.5% of LHID adults

The two exposed groups essentially cannot share people. Cohort 1 excludes anyone with IBS before the PD diagnosis, and Cohort 2 excludes anyone with PD before the IBS diagnosis, so each person can be exposed in at most one cohort (apart from anyone first diagnosed with both on the same date). Taken at face value, Tables 1 and 2 would mean that 41.7% of all adults in the database received a first diagnosis of PD or of IBS between 2002 and 2012, with the other condition absent beforehand. The flowchart's figures correspond to 3.5%.

Both totals for each cohort are internally consistent. Table 2's blocks all sum to 180,629, and the flowchart's numbers multiply out to 32,220. The two descriptions cannot both refer to the same matched sample. The follow-up percentages and the CI widths fit the flowchart, so the survival analyses appear to have been run on about 16,110 patients per arm in Cohort 2 (and about 18,402 in Cohort 1). It is not stated what population Tables 1 and 2 describe.

Figure

Per-arm sample sizes printed in the Tables and in the flowchart, with the range implied by the printed event percentages

Figure 1. Participants per arm for each cohort on one linear axis starting at zero. Orange markers are printed in the paper: circles are the Table and Results-text values (234,361 for Cohort 1, 180,629 for Cohort 2), and squares are the Figure 1 flowchart values (18,402 and 16,110). The grey bar and the label on it give the ratio between the two printed values (12.7 and 11.2). The blue diamond marks the range of per-arm N (narrower than the marker at this scale) for which both printed follow-up percentages are reproduced from the printed event counts (Cohort 2: 2,031 at 12.6% and 1,308 at 8.1% give 16,056–16,183; Cohort 1: 1,498 at 8.1% and 921 at 5.0% give 18,381–18,606), computed by recompute.py.txt. In both cohorts this range contains the flowchart value and excludes the Table value.

Overall adjusted hazard ratios with the reported 95% CI and the intervals expected from the reported events and from the Table sample sizes

Figure 2. Overall adjusted hazard ratios on one log-scale axis; the vertical line marks HR = 1. Orange circles are the reported aHRs and 95% CIs (Table 4: 1.46, 1.37–1.56; Table 3: 1.55, 1.42–1.69). Filled blue diamonds are the intervals expected from the reported numbers of events (3,339 and 2,419), with SE(log HR) = √(1/d₁ + 1/d₂); they agree closely with the reported intervals (for Cohort 1 the reported interval is slightly wider than 2,419 events alone imply, which corresponds to about 2,028 events). Open blue diamonds are the intervals expected if the Table sample sizes had produced events at the printed rates (37,390 and 30,701 events); they are about three to four times narrower than reported (3.1x for Cohort 2, 3.8x for Cohort 1, measured as SE(log HR) and identical for the linear width). The reported precision is what the flowchart-sized cohorts would give, not what the Table-sized cohorts would give.

Why it matters

Cohort 2 is one of the paper's two primary analyses. Its sample size is given in the Results text and in Table 2 as 361,258, about eleven times the 32,220 in the flowchart that the Methods identify as showing the "final analytic sample sizes". The follow-up events and the precision of the hazard ratio fit the smaller number, so the hazard ratios were most likely estimated on about 32,000 people. Their direction and statistical significance are probably not affected by the misstated N.

What is affected is the description of the study population. Table 2, the Results paragraphs that summarize it ("well matched", the income and comorbidity contrasts, the 45.77% vs 17.53% PD prevalence) and the corresponding Table 1 material for Cohort 1 cannot describe the analysed cohorts. They report N values 11–13 times too large and a large prevalence of the very condition that each cohort was defined to exclude: 82,681 IBS patients "With PD" in Cohort 2 and 75,244 PD patients "With IBS" in Cohort 1. A reader cannot use these tables to judge baseline balance, comorbidity burden or generalizability, and the Discussion's "The large sample enabled robust subgroup and interaction analyses" rests in part on the larger figures. Anyone extracting the sample size for a systematic review or meta-analysis would take 361,258 (or 468,722) from the text, overstating the size of these cohorts eleven- to thirteen-fold.

The flowchart has its own problem. It lists 16,110 newly diagnosed IBS cases, then 3,231 exclusions for prior PD, and then an IBS cohort of 16,110 again, where 12,879 would be expected. So it is not clear whether 16,110 is the number before or after the exclusion.

A separate inconsistency adds to the uncertainty. The abstract gives "IBS was associated with increased risks of depression (adjusted hazard ratio [aHR] = 1.55) and anxiety (aHR = 1.68)", but 1.55 and 1.68 are the overall and female HRs for the opposite direction (PD → IBS) in Table 3. The body reports no depression- or anxiety-specific HRs, and the abstract's 1.45 and 1.51 for "depression and anxiety ... developing IBS" do not appear as point estimates anywhere in the body. Table 4's overall HR for IBS → PD is 1.46.

Explanations considered

Status

Suggested correction

  1. State which population Tables 1 and 2 describe. If the analysed cohorts are those in Figure 1 (18,402 and 16,110 per arm, as the event percentages and hazard-ratio precision indicate), replace Tables 1 and 2 and the Results paragraphs that summarize them with the baseline characteristics of those cohorts, and correct "468,722" / "234,361" and "361,258" / "180,629" in the Results text.
  2. Explain why Tables 1 and 2 show 75,244 PD patients with IBS and 82,681 IBS patients with PD at baseline in cohorts defined by excluding those conditions, or remove these rows if they do not describe baseline status.
  3. Reconcile the Cohort 2 flowchart: 16,110 newly diagnosed IBS cases minus 3,231 exclusions should not leave an IBS cohort of 16,110. Give the counts before and after exclusion, and give the Cohort 1 eligible population as a number instead of "990,0xx" (995,404 − 5,412 = 989,992, which does not fit "990,0xx" either).
  4. Correct the abstract's hazard ratios so that each one matches a reported estimate in the body for the stated direction and outcome (for IBS → PD, Table 4 gives 1.46 overall), or add the depression- and anxiety-specific estimates to the Results.
  5. Reword statements that rest on the larger sample sizes, such as "The large sample enabled robust subgroup and interaction analyses", if the analysed cohorts are the smaller ones.
Recomputation script (python recompute.py.txt)
"""Recompute the cohort sizes of PMC12902173
(Psychol Med 2026, doi:10.1017/S0033291726103328).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five design candidates in candidates/.

All inputs are copied from the article as archived in PubMed Central (the same numbers are
in the publisher's HTML and the Europe PMC XML): Methods (data source and the Figure 1
flowchart text), Results, the Abstract and Tables 1-4. No patient-level data are used.
"""
import math
import os
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (from the paper)
LHID = 1_000_000            # Methods, data source
MISSING_DEMO = 1_982        # Methods / flowchart
UNDER_18 = 2_614            # Methods / flowchart
PRIOR_IBS_C1 = 5_412        # flowchart, Cohort 1

# Cohort 2: IBS -> psychological distress (PD)
C2_TABLE_ARM = 180_629      # Table 2 headers and Results text ("divided evenly")
C2_TEXT_TOTAL = 361_258     # Results text
C2_FC_NEW = 16_110          # flowchart: newly diagnosed IBS cases
C2_FC_EXCL = 3_231          # flowchart: excluded prior depressive or anxiety disorders
C2_FC_ARM = 16_110          # flowchart: IBS cohort / matched controls
C2_FC_TOTAL = 32_220        # flowchart: final follow-up N for PD outcomes
C2_EVENTS = (2_031, 1_308)  # Results: IBS group, control group
C2_EVENT_PCT = (12.6, 8.1)  # Results
C2_AHR = (1.46, 1.37, 1.56)  # Results / Table 4, overall
TABLE2 = {  # (IBS, without IBS) counts per block of Table 2
    "Psychological distress (without PD, with PD)": ((97_948, 82_681), (148_971, 31_658)),
    "Gender (male, female)": ((91_509, 89_120), (91_509, 89_120)),
    "Age group (18-24, 25-44, 45-64, >=65)": ((17_211, 67_837, 66_504, 29_077),) * 2,
    "Urbanization (levels 1-4)": ((55_995, 68_639, 28_900, 27_095),) * 2,
    "Monthly income (3 bands)": ((43_194, 85_106, 52_329), (99_860, 55_902, 24_867)),
}
T2_WITH_PD = (82_681, 31_658)
T2_PD_PCT = (45.77, 17.53)
T2_DEP_ANX = ((15_806, 66_875), (7_339, 24_319))

# Cohort 1: PD -> IBS
C1_TABLE_ARM = 234_361      # Table 1 headers and Results text
C1_TEXT_TOTAL = 468_722
C1_FC_ARM = 18_402          # flowchart
C1_FC_TOTAL = 36_804
C1_EVENTS = (1_498, 921)
C1_EVENT_PCT = (8.1, 5.0)
C1_AHR = (1.55, 1.42, 1.69)  # Results / Table 3, overall
T1_WITH_IBS = (75_244, 28_210)
T1_IBS_PCT = (32.11, 12.04)

TABLE3 = {"Overall": 1.55, "Female": 1.68, "Male": 1.39}
TABLE4 = {"Overall": 1.46, "Female": 1.52, "Male": 1.38}
ABSTRACT_HR = {"IBS -> depression": 1.55, "IBS -> anxiety": 1.68,
               "depression -> IBS": 1.45, "anxiety -> IBS": 1.51}

Z = 1.959964


def pct(a, b, d=2):
    return round(100 * a / b, d)


def n_range(events, pct_printed, dec=1):
    """Denominators N for which 100*events/N rounds to pct_printed at `dec` decimals."""
    h = 0.5 * 10 ** (-dec)
    return math.floor(events / ((pct_printed + h) / 100)) + 1, math.floor(events / ((pct_printed - h) / 100))


def se_from_ci(lo, hi):
    return (math.log(hi) - math.log(lo)) / (2 * Z)


# ------------------------------------------------------------------ 1. Cohort 2 sizes
print("1. Cohort 2 (IBS -> psychological distress)")
print(f"   Table 2 / text: {C2_TABLE_ARM:,} x 2 = {2 * C2_TABLE_ARM:,} (text says {C2_TEXT_TOTAL:,})")
print(f"   Flowchart:      {C2_FC_ARM:,} x 2 = {2 * C2_FC_ARM:,} (flowchart says {C2_FC_TOTAL:,})")
c2_ratio = C2_TEXT_TOTAL / C2_FC_TOTAL
print(f"   Ratio text/flowchart = {c2_ratio:.2f}; difference = {C2_TEXT_TOTAL - C2_FC_TOTAL:,} "
      f"({C2_TABLE_ARM - C2_FC_ARM:,} per arm)")
c2_after_excl = C2_FC_NEW - C2_FC_EXCL
print(f"   Flowchart internal: {C2_FC_NEW:,} new IBS - {C2_FC_EXCL:,} excluded = {c2_after_excl:,}, "
      f"but IBS cohort printed as {C2_FC_ARM:,}")
print("   Table 2 block sums (IBS column, control column):")
for k, (a, b) in TABLE2.items():
    print(f"     {k}: {sum(a):,}, {sum(b):,}")
print(f"   With PD: {T2_WITH_PD[0]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[0], C2_TABLE_ARM)}%, "
      f"{T2_WITH_PD[1]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[1], C2_TABLE_ARM)}% "
      f"(printed {T2_PD_PCT[0]}% and {T2_PD_PCT[1]}%)")
for lab, (d, a), wp in zip(("IBS", "control"), T2_DEP_ANX, T2_WITH_PD):
    print(f"   {lab}: depression + anxiety = {d:,} + {a:,} = {d + a:,} (= 'With PD' {wp:,}); "
          f"depression share {pct(d, wp)}%")
print("   Follow-up events:")
c2_pct_fc, c2_pct_tab = [], []
for lab, e, rp in zip(("IBS", "control"), C2_EVENTS, C2_EVENT_PCT):
    a, b = 100 * e / C2_FC_ARM, 100 * e / C2_TABLE_ARM
    c2_pct_fc.append(a); c2_pct_tab.append(b)
    lo, hi = n_range(e, rp)
    print(f"     {lab}: {e:,}/{C2_FC_ARM:,} = {a:.2f}%; {e:,}/{C2_TABLE_ARM:,} = {b:.2f}% "
          f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c2_lo = max(n_range(e, p)[0] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
c2_hi = min(n_range(e, p)[1] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
print(f"     Both percentages hold only for N per arm in {c2_lo:,}-{c2_hi:,}: contains "
      f"{C2_FC_ARM:,}, excludes {C2_TABLE_ARM:,}")
print(f"   Prevalent PD at baseline in a cohort that excluded prior PD: {T2_WITH_PD[0]:,} "
      f"vs {C2_FC_EXCL:,} excluded in the flowchart ({T2_WITH_PD[0] / C2_FC_EXCL:.1f}x); "
      f"vs {C2_EVENTS[0]:,} follow-up events ({T2_WITH_PD[0] / C2_EVENTS[0]:.1f}x)")

# ------------------------------------------------------------------ 2. Cohort 1 sizes
print("\n2. Cohort 1 (psychological distress -> IBS)")
print(f"   Table 1 / text: {C1_TABLE_ARM:,} x 2 = {2 * C1_TABLE_ARM:,} (text says {C1_TEXT_TOTAL:,})")
print(f"   Flowchart:      {C1_FC_ARM:,} x 2 = {2 * C1_FC_ARM:,} (flowchart says {C1_FC_TOTAL:,})")
c1_ratio = C1_TEXT_TOTAL / C1_FC_TOTAL
print(f"   Ratio text/flowchart = {c1_ratio:.2f}")
c1_pct_fc, c1_pct_tab = [], []
for lab, e, rp in zip(("PD", "control"), C1_EVENTS, C1_EVENT_PCT):
    a, b = 100 * e / C1_FC_ARM, 100 * e / C1_TABLE_ARM
    c1_pct_fc.append(a); c1_pct_tab.append(b)
    lo, hi = n_range(e, rp)
    print(f"     {lab}: {e:,}/{C1_FC_ARM:,} = {a:.2f}%; {e:,}/{C1_TABLE_ARM:,} = {b:.2f}% "
          f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c1_lo = max(n_range(e, p)[0] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
c1_hi = min(n_range(e, p)[1] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
print(f"     Both percentages hold only for N per arm in {c1_lo:,}-{c1_hi:,}: contains "
      f"{C1_FC_ARM:,}, excludes {C1_TABLE_ARM:,}")
print(f"   With IBS at baseline (Table 1): {T1_WITH_IBS[0]:,}/{C1_TABLE_ARM:,} = "
      f"{pct(T1_WITH_IBS[0], C1_TABLE_ARM)}%, {T1_WITH_IBS[1]:,}/{C1_TABLE_ARM:,} = "
      f"{pct(T1_WITH_IBS[1], C1_TABLE_ARM)}% (printed {T1_IBS_PCT[0]}%, {T1_IBS_PCT[1]}%), in a "
      f"cohort that excluded prior IBS (flowchart: {PRIOR_IBS_C1:,} excluded)")

# ------------------------------------------------------------------ 3. source population
print("\n3. Source population (LHID)")
adults = LHID - MISSING_DEMO - UNDER_18
elig_c1 = adults - PRIOR_IBS_C1
print(f"   {LHID:,} - {MISSING_DEMO:,} - {UNDER_18:,} = {adults:,} adults with complete data")
print(f"   minus {PRIOR_IBS_C1:,} prior IBS = {elig_c1:,} (flowchart prints '990,0xx'; "
      f"{elig_c1:,} is {'' if 990_000 <= elig_c1 < 990_100 else 'not '}of that form)")
exp_tab = C1_TABLE_ARM + C2_TABLE_ARM
exp_fc = C1_FC_ARM + C2_FC_ARM
print(f"   Exposed arms, Tables 1+2: {C1_TABLE_ARM:,} + {C2_TABLE_ARM:,} = {exp_tab:,} = "
      f"{pct(exp_tab, adults, 1)}% of adults")
print(f"   Exposed arms, flowchart:  {C1_FC_ARM:,} + {C2_FC_ARM:,} = {exp_fc:,} = "
      f"{pct(exp_fc, adults, 1)}% of adults")
all_tab = C1_TEXT_TOTAL + C2_TEXT_TOTAL
print(f"   All four Table arms: {C1_TEXT_TOTAL:,} + {C2_TEXT_TOTAL:,} = {all_tab:,} = "
      f"{pct(all_tab, adults, 1)}% of adults (controls may overlap between cohorts)")
print("   The two exposed groups essentially cannot overlap: Cohort 1 excludes prior IBS and"
      " Cohort 2 excludes prior PD, so a person is exposed in at most one of them (apart from"
      " anyone first diagnosed with both on the same date).")

# ------------------------------------------------------------------ 4. CI width vs events
print("\n4. Number of events implied by the width of the overall aHR interval")
print("   Schoenfeld approximation for 1:1 allocation: var(log HR) = 4 / D")
ci_rows = []
for lab, (hr, lo, hi), ev, ev_pct, arm_tab in (
        ("Cohort 2 (Table 4)", C2_AHR, C2_EVENTS, C2_EVENT_PCT, C2_TABLE_ARM),
        ("Cohort 1 (Table 3)", C1_AHR, C1_EVENTS, C1_EVENT_PCT, C1_TABLE_ARM)):
    se = se_from_ci(lo, hi)
    se_min = se_from_ci(lo - 0.005, hi + 0.005)
    se_max = se_from_ci(lo + 0.005, hi - 0.005)
    d_impl = 4 / se ** 2
    d_rep = sum(ev)
    se_rep = math.sqrt(1 / ev[0] + 1 / ev[1])
    ev_big = [p / 100 * arm_tab for p in ev_pct]
    se_big = math.sqrt(1 / ev_big[0] + 1 / ev_big[1])
    ci_rep = (hr * math.exp(-Z * se_rep), hr * math.exp(Z * se_rep))
    ci_big = (hr * math.exp(-Z * se_big), hr * math.exp(Z * se_big))
    print(f"   {lab}: aHR {hr} ({lo}-{hi}); SE(log HR) = {se:.4f} "
          f"(rounding range {se_max:.4f}-{se_min:.4f})")
    print(f"     implied events 4/SE^2 = {d_impl:,.0f} (range {4 / se_min ** 2:,.0f}-"
          f"{4 / se_max ** 2:,.0f}); reported events {ev[0]:,} + {ev[1]:,} = {d_rep:,}")
    print(f"     95% CI expected from the reported events (SE = sqrt(1/d1 + 1/d2) = {se_rep:.4f}): "
          f"{ci_rep[0]:.2f}-{ci_rep[1]:.2f}")
    print(f"     if {arm_tab:,} per arm had the printed event rates: events "
          f"{ev_big[0]:,.0f} + {ev_big[1]:,.0f} = {sum(ev_big):,.0f}; SE {se_big:.4f}; "
          f"95% CI {ci_big[0]:.2f}-{ci_big[1]:.2f}")
    d_lo, d_hi = 4 / se_min ** 2, 4 / se_max ** 2
    inside = d_lo <= d_rep <= d_hi
    print(f"     reported events {d_rep:,} {'lie inside' if inside else 'lie outside'} the implied "
          f"range {d_lo:,.0f}-{d_hi:,.0f}")
    w_rep = hi - lo
    w_big = ci_big[1] - ci_big[0]
    print(f"     reported interval vs interval expected from Table N: SE {se:.4f} / {se_big:.4f} "
          f"= {se / se_big:.2f}x; linear width {w_rep:.2f} / {w_big:.4f} = {w_rep / w_big:.2f}x")
    ci_rows.append((lab, hr, (lo, hi), ci_rep, ci_big, d_rep, sum(ev_big)))

# ------------------------------------------------------------------ 5. abstract HRs
print("\n5. Abstract hazard ratios vs Tables 3 and 4")
for k, v in ABSTRACT_HR.items():
    t3 = [s for s, x in TABLE3.items() if x == v]
    t4 = [s for s, x in TABLE4.items() if x == v]
    print(f"   Abstract {k} = {v}: Table 3 (PD -> IBS) rows {t3 or 'none'}; "
          f"Table 4 (IBS -> PD) rows {t4 or 'none'}")
print(f"   Table 4 overall (IBS -> PD) = {TABLE4['Overall']}; Table 3 overall (PD -> IBS) = "
      f"{TABLE3['Overall']}. No depression- or anxiety-specific HRs are reported in the body.")

# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
W = 7.2
LW = 1.5  # points; 1.5 pt = 2 px at 96 dpi
plt.rcParams.update({
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK, "text.color": INK,
    "xtick.color": INK2, "ytick.color": INK2, "axes.spines.top": False,
    "axes.spines.right": False, "font.family": "DejaVu Sans", "svg.fonttype": "none",
})


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def wrap(s, w):
    return "\n".join(textwrap.wrap(s, w))


def fmt(n):
    return f"{n:,.0f}"


# ---------------------------------------------------------------- candidate A: N per arm dot plot
def cand_A():
    """Per-arm N: Tables/text vs flowchart (both printed), with the N range implied by the
    printed event counts and percentages (recomputed)."""
    fig, ax = plt.subplots(figsize=(W, 3.3))
    rows = [("Cohort 1\nPD \u2192 IBS", C1_TABLE_ARM, C1_FC_ARM, (c1_lo, c1_hi), C1_EVENTS,
             C1_EVENT_PCT, "Table 1"),
            ("Cohort 2\nIBS \u2192 PD", C2_TABLE_ARM, C2_FC_ARM, (c2_lo, c2_hi), C2_EVENTS,
             C2_EVENT_PCT, "Table 2")]
    ys = [1, 0]
    for y, (lab, tab, fc, (lo, hi), ev, ep, tname) in zip(ys, rows):
        ax.plot([fc, tab], [y, y], color=GRID, lw=4, zorder=1, solid_capstyle="round")
        ax.scatter(tab, y, s=150, color=ORANGE, zorder=3, edgecolor="white", lw=0.8)
        ax.scatter(fc, y, s=150, color=ORANGE, marker="s", zorder=3, edgecolor="white", lw=0.8)
        ax.plot([lo, hi], [y - 0.2, y - 0.2], color=BLUE, lw=LW * 2, zorder=2)
        ax.scatter((lo + hi) / 2, y - 0.2, s=70, color=BLUE, marker="D", zorder=3,
                   edgecolor="white", lw=0.6)
        ax.text(tab, y + 0.17, f"{tname} / text: {fmt(tab)}", ha="center", va="bottom",
                color=ORANGE, fontsize=10)
        ax.text(fc, y + 0.17, f"Figure 1 flowchart: {fmt(fc)}", ha="left", va="bottom",
                color=ORANGE, fontsize=10)
        ax.text(hi + 4000, y - 0.2,
                f"N implied by {fmt(ev[0])} ({ep[0]}%) and {fmt(ev[1])} ({ep[1]}%) events: "
                f"{fmt(lo)}\u2013{fmt(hi)}", ha="left", va="center", color=BLUE, fontsize=10)
        ax.text((fc + tab) / 2, y, f"\u00d7{tab / fc:.1f}", ha="center", va="center",
                color=INK2, fontsize=10, zorder=4,
                bbox=dict(boxstyle="square,pad=0.25", fc=BG, ec="none"))
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-0.55, 1.55); ax.set_xlim(0, 255_000)
    ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v / 1000:.0f}k"))
    ax.set_xlabel("Participants per arm (exposed group; controls are matched 1:1)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported: Table / Results text"),
         Line2D([], [], marker="s", ls="", color=ORANGE, ms=9, label="reported: flowchart"),
         Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
                label="recomputed: N consistent with the printed event %")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2,
              frameon=False, fontsize=10)
    return fig


# ---------------------------------------------------------------- candidate B: text panels
def cand_B():
    fig = plt.figure(figsize=(W, 5.4))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 80); ax.axis("off")

    def panel(x, y, w, h, title, lines):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                    fc="white", ec=ORANGE, lw=LW))
        ax.text(x + 1.5, y + h - 1.5, title, fontsize=11, fontweight="bold", va="top")
        ax.text(x + 1.5, y + h - 6.5, lines, fontsize=10, va="top", linespacing=1.4)

    panel(2, 30, 43, 46, "Results text and Table 2",
          wrap("\u201cTable 2 summarizes the baseline characteristics of 361,258 participants, "
               "divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).\u201d", 40)
          + "\n\nTable 2 column headers:\n\u201cIBS(n = 180,629)\u201d\n\u201cWithout IBS(n = 180,629)\u201d"
          + "\n\nTable 2, \u201cWith PD\u201d:\n\u201c82,681 (45.77%) | 31,658 (17.53%)\u201d")
    panel(55, 30, 43, 46, "Figure 1 flowchart, Cohort 2",
          "\u201cNewly diagnosed IBS cases\n(2002\u20132012): 16,110\u201d\n"
          "\u201cExcluded prior depressive or\nanxiety disorders: 3,231 excluded\u201d\n"
          "\u201cIBS cohort: 16,110\u201d\n\u201cMatched controls: 16,110\u201d\n"
          "\u201cFinal follow-up N for PD\noutcomes: 32,220\u201d\n\nResults, follow-up:\n"
          + wrap("\u201c2,031 IBS patients (12.6%) \u2026 compared to 1,308 (8.1%)\u201d", 40))
    ax.annotate("", xy=(54.2, 53), xytext=(45.8, 53),
                arrowprops=dict(arrowstyle="<->", color=BLUE, lw=LW))
    ax.text(50, 56, "\u00d711.2", ha="center", color=BLUE, fontsize=11, fontweight="bold")
    ax.text(2, 23, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
            fontweight="bold", color=BLUE)
    notes = [f"180,629 \u00d7 2 = {2 * C2_TABLE_ARM:,}, but 16,110 \u00d7 2 = {2 * C2_FC_ARM:,}"
             f" (ratio {c2_ratio:.2f})",
             f"2,031 / 16,110 = {c2_pct_fc[0]:.1f}% and 1,308 / 16,110 = {c2_pct_fc[1]:.1f}%"
             f" (as printed); over 180,629: {c2_pct_tab[0]:.2f}% and {c2_pct_tab[1]:.2f}%",
             f"16,110 \u2212 3,231 = {c2_after_excl:,}, not 16,110",
             f"82,681 IBS patients \u201cWith PD\u201d in a cohort that excluded prior PD"]
    for i, s in enumerate(notes):
        ax.text(3, 18 - i * 4.4, "\u2022 " + s, fontsize=10, color=BLUE, va="center")
    return fig


# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")

    def box(x, y, w, h, s, ec, fs=10, bold=False):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
                                    fc="white", ec=ec, lw=LW))
        ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs,
                fontweight="bold" if bold else "normal")

    def arrow(x0, y0, x1, y1, c=INK2):
        ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
                    arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))

    ax.text(1, 61, "Cohort 2 as printed in the Methods and the Figure 1 flowchart", fontsize=10.5,
            fontweight="bold")
    box(1, 44, 21, 12, "Methods: LHID\n1,000,000 \u2212 1,982\n\u2212 2,614", ORANGE)
    box(27, 44, 20, 12, "new IBS cases\n2002\u20132012\n16,110", ORANGE)
    box(52, 44, 20, 12, "excluded prior\nPD: 3,231\nIBS cohort: 16,110", ORANGE)
    box(77, 44, 22, 12, "matched 1:1\n16,110 + 16,110\n= 32,220", ORANGE)
    arrow(22.6, 50, 26.4, 50); arrow(47.6, 50, 51.4, 50); arrow(72.6, 50, 76.4, 50)

    ax.text(24, 38.5, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
            fontweight="bold", color=BLUE, va="center")
    box(1, 22, 21, 11, f"{adults:,}\nadults", BLUE)
    box(52, 22, 20, 11, f"16,110 \u2212 3,231\n= {c2_after_excl:,}", BLUE)
    box(77, 22, 22, 11, f"2,031 (12.6%), 1,308\n(8.1%) events imply\nN = {c2_lo:,}\u2013{c2_hi:,}", BLUE)
    ax.plot([11.5, 11.5], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
    ax.plot([62, 62], [43.4, 41.0], color=BLUE, lw=LW, ls=(0, (3, 2)))
    ax.plot([62, 62], [36.0, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
    ax.plot([88, 88], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))

    ax.text(1, 15, "Cohort 2 as printed in the Results text and Table 2", fontsize=10.5,
            fontweight="bold")
    box(52, 1, 47, 11, f"180,629 + 180,629 = 361,258\n({c2_ratio:.1f} times the flowchart\u2019s 32,220)",
        ORANGE, bold=True)
    box(1, 1, 46, 11, "82,681 (45.77%) of the IBS arm \u201cWith PD\u201d\nalthough prior PD was "
        "an exclusion", ORANGE)
    return fig


# ---------------------------------------------------------------- candidate D: event % dot plot
def cand_D():
    fig, ax = plt.subplots(figsize=(W, 3.5))
    rows = [("Cohort 1, PD arm\n1,498 events", C1_EVENT_PCT[0], c1_pct_fc[0], c1_pct_tab[0]),
            ("Cohort 1, controls\n921 events", C1_EVENT_PCT[1], c1_pct_fc[1], c1_pct_tab[1]),
            ("Cohort 2, IBS arm\n2,031 events", C2_EVENT_PCT[0], c2_pct_fc[0], c2_pct_tab[0]),
            ("Cohort 2, controls\n1,308 events", C2_EVENT_PCT[1], c2_pct_fc[1], c2_pct_tab[1])]
    ys = [3, 2, 1, 0]
    for y, (lab, rep, fc, tab) in zip(ys, rows):
        ax.plot([tab, rep], [y, y], color=GRID, lw=4, zorder=1)
        ax.scatter(rep, y, s=170, color=ORANGE, zorder=2, edgecolor="white", lw=0.8)
        ax.scatter(fc, y, s=60, color=BLUE, marker="D", zorder=3, edgecolor="white", lw=0.6)
        ax.scatter(tab, y, s=110, facecolor=BG, edgecolor=BLUE, marker="D", lw=LW, zorder=3)
        ax.text(rep + 0.3, y, f"{rep}%", va="center", color=ORANGE, fontsize=10)
        ax.text(tab + 0.3, y + 0.22, f"{tab:.2f}%", va="bottom", ha="left", color=BLUE,
                fontsize=10)
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_xlim(0, 14.5); ax.set_ylim(-0.6, 3.6)
    ax.set_xlabel("Share of the arm with an outcome event during follow-up (%)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="reported %"),
         Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
                label="recomputed: events / flowchart N"),
         Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=LW, ms=9,
                label="recomputed: events / Table N")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3,
              frameon=False, fontsize=10)
    return fig


# ---------------------------------------------------------------- candidate E: CI width check
def cand_E():
    fig, ax = plt.subplots(figsize=(3.4, 3.8))
    y = 0
    yt, yl = [], []
    for lab, hr, rep_ci, ci_rep, ci_big, d_rep, d_big in ci_rows:
        tname = "Table 2" if "Cohort 2" in lab else "Table 1"
        items = [("reported aHR, 95% CI", rep_ci, ORANGE, "o", 0.28),
                 (f"recomputed: {d_rep:,} reported events", ci_rep, BLUE, "D", 0.0),
                 (f"recomputed: {tname} N at printed rates\n({d_big:,.0f} events)",
                  ci_big, BLUE, "D", -0.28)]
        for txt, (lo, hi), c, m, dy in items:
            mfc = c if dy >= 0 else BG
            ax.plot([lo, hi], [y + dy, y + dy], color=c, lw=LW)
            ax.plot([hr], [y + dy], marker=m, color=c, mfc=mfc, mec=c, mew=LW, ms=8, ls="")
            ax.text(1.02, y + dy, f"{lo:.2f}\u2013{hi:.2f}  {txt}", va="center", fontsize=10,
                    color=c, transform=ax.get_yaxis_transform(), clip_on=False)
        yt.append(y); yl.append(lab.replace(" (", "\n("))
        y -= 1.2
    ax.axvline(1, color=INK2, lw=0.8)
    ax.set_xscale("log")
    ax.set_xticks([1, 1.2, 1.4, 1.6, 1.8])
    ax.get_xaxis().set_major_formatter(matplotlib.ticker.FormatStrFormatter("%.1f"))
    ax.get_xaxis().set_minor_formatter(matplotlib.ticker.NullFormatter())
    ax.set_xlim(0.95, 1.8)
    ax.minorticks_off()
    ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
    ax.set_ylim(y + 0.75, 0.55)
    ax.set_xlabel("Overall adjusted hazard ratio\n(log scale; line at 1 = no association)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
    return fig


for name, fn in (("A_n_per_arm", cand_A), ("B_text_panels", cand_B),
                 ("C_calc_flow", cand_C), ("D_event_pct", cand_D), ("E_ci_width", cand_E)):
    save(fn(), os.path.join(CAND, name))
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_E(), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1 (= candidate A), fig2 (= candidate E), candidates/A-E")

The logistic-regression "odds ratios", including those in the abstract, appear to be log-odds coefficients

Psychol Med 2022;52(14):3289-3296 (published online 18 March 2021) · doi:10.1017/S0033291721000088 · PMC9693673

Summary

The paper reports two multivariable logistic regressions for clinically elevated hsCRP (⩾3 mg/L) and labels every estimate an odds ratio (OR). One of them is negative ("OR −4.39, 95% CI −7.04 to −1.97"), and six more have negative lower confidence limits (four distinct intervals; the age and gender intervals appear in both models). An odds ratio cannot be negative. Read as log-odds coefficients, the printed values hang together: every printed CI agrees with its printed p-value on significance at 0.05 (the interval excludes 0 exactly when p < 0.05), and the paper's own Table 1 counts reproduce the estimates closely (unadjusted log-odds 2.39 against the printed 2.44). Read as ORs, the two main-effect intervals in model 2 contradict their p-values. On the coefficient reading, the abstract's headline odds ratios are understated about 2.7- to 4.7-fold: "OR 2.44" corresponds to an OR of about 11.5 (95% CI 3.6 to 51.4), "OR 1.52" to about 4.6 and "OR 1.09" to about 3.0. The direction and significance of each finding are unchanged, but the effect sizes as printed are on the wrong scale, and read literally as ORs, the CI given for depression (0.27 to 2.01) includes 1 and would contradict the stated significance.

What the paper reports

Methods, "Statistical analysis":

To address the second hypothesis, two models of multivariable logistic regression were used. The first assessed how much risk comorbid depression and overweight contributed to the hsCRP levels ⩾3 mg/L relative to other participant groups, with age and gender as covariates. The second assessed how much risk depression and overweight status (BMI ⩾ 25) separately contributed to hsCRP levels ⩾3 mg/L, with age and gender as covariates.

Results, "High sensitivity C-Reactive Protein", first logistic paragraph (model 1):

Overweight patients with depression were at a significantly increased risk of having hsCRP ⩾3 mg/L [OR 2.44, 95% confidence interval (CI) 1.28 to 3.94, p < 0.001], overweight controls did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.51, 95% CI −0.33 to 2.87, p = 0.13), normal-weight patients with depression did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.08, 95% CI −0.6 to 2.4, p = 0.28), while normal-weight controls were at a significantly decreased risk of hsCRP ⩾3 mg/L (OR −4.39, 95% CI −7.04 to −1.97, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.01, 95% CI −0.04 to 0.06, p = 0.621, OR 0.61, 95% CI −0.19 to 1.47, p = 0.145, respectively].

Results, same section, second logistic paragraph (model 2):

Depression status was significantly associated with hsCRP ⩾3 mg/L (OR 1.09, 95% CI 0.27 to 2.01, p = 0.013) and overweight status was significantly associated with hsCRP ⩾3 mg/L (OR 1.52, 95% CI 0.7 to 2.41, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.62, 95% CI −0.04 to 0.06, p = 0.647, OR 0.01, 95% CI −0.19 to 1.47, p = 0.137, respectively].

Abstract, "Results":

Multivariable logistic regression showed that comorbid depression and overweight significantly increased the risk of clinically elevated hsCRP levels ⩾3 mg/L (OR 2.44, 1.28–3.94). In a separate multivariable logistic regression model, overweight status contributed most to the risk of having hsCRP levels ⩾3 mg/L (OR 1.52, 0.7–2.41), while depression also contributed a significant risk (OR 1.09, 0.27–2).

Table 1, row "hsCRP ⩾3 mg/L (%)". The columns are overweight & depressed (N = 69), overweight control (N = 35), normal weight & depressed (N = 55), normal weight control (N = 57), statistic, p:

hsCRP ⩾3 mg/L (%) | 26 (37.7%) | 5 (14.3%) | 6 (10.9%) | 3 (5.3%) | x2 = 25.96 | <0.001

The conclusions that rest on these models. Abstract, "Conclusion":

Comorbid depression and overweight status are associated with increased hsCRP, and the coexistence of these conditions amplified the risk of clinically elevated hsCRP levels. Overweight status contributed most to the risk of clinically elevated hsCRP levels, but depression also contributed to a significant risk.

Discussion, first paragraph:

Of clinical relevance, overweight patients with depression had the highest risk for hsCRP levels ⩾3 mg/L, indicating clinically elevated peripheral inflammation.

What the paper's own numbers show

Method. In logistic regression a coefficient β is a log odds ratio, and the odds ratio is exp(β). An OR must be positive and its null is 1, while a coefficient can take any sign and its null is 0. I made three checks:

  1. Sign. I checked each printed estimate and CI limit for values ⩽ 0, which no OR can take.
  2. CI against p-value. I took each printed CI as a 95% Wald interval and back-calculated the two-sided p-value it implies under two readings: as a coefficient (null 0) and as an OR (null 1, computed on the log scale). I also checked, more simply, whether each interval excludes the null exactly when the printed p is below 0.05. The printed intervals are not symmetric about their estimates, and most look like profile-likelihood intervals (see check 3), so a Wald back-calculation from them is only approximate. The significance check does not depend on that assumption.
  3. Refit from Table 1. I refitted both models without the age and gender covariates, using only the grouped counts in Table 1 (40 cases with hsCRP ⩾3 mg/L among 216 participants). The paper's covariate-adjusted models need individual data, so the refits show the scale and approximate size of the estimates, not their exact values. I report both Wald and profile-likelihood intervals for the refits, because most printed intervals are asymmetric about their estimates in the way profile-likelihood intervals are (2.44 lies 1.16 above its lower limit of 1.28 and 1.50 below its upper limit of 3.94).

Every number below is printed by recompute.py.txt. Quoted passages keep the paper's minus signs (U+2212); elsewhere minus signs are typed as hyphens.

Quantity Location Reported Recomputed from the paper's numbers
hsCRP ⩾3 mg/L by group Table 1 26/69, 5/35, 6/55, 3/57; χ² = 25.96, p < 0.001 37.7%, 14.3%, 10.9%, 5.3%; Pearson χ² = 25.96, df 3, p = 9.7 × 10⁻⁶ (matches)
Overweight + depressed v. reference Abstract; Results, model 1 "OR 2.44 (1.28 to 3.94)", p < 0.001 As a coefficient, the CI implies p = 0.0003 (matches p < 0.001); as an OR, it implies p = 0.002. Unadjusted refit: log-odds 2.39 (profile 1.26 to 3.87; Wald 1.13 to 3.65), p = 0.0002. Crude OR = (26/43)/(3/54) = 10.9. If the printed value is a coefficient, OR = exp(2.44) = 11.5 (3.6 to 51.4)
Overweight controls v. reference Results, model 1 "OR 1.51 (−0.33 to 2.87)", p = 0.13 Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.13), but a Wald back-calculation gives p = 0.064, about half the printed 0.13. Refit: log-odds 1.10 (profile -0.37 to 2.74; Wald -0.40 to 2.60), Wald p = 0.15. exp(printed) = 4.5 (0.72 to 17.6)
Normal-weight depressed v. reference Results, model 1 "OR 1.08 (−0.6 to 2.4)", p = 0.28 Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.28), but a Wald back-calculation gives p = 0.16, about half the printed 0.28. Refit: log-odds 0.79 (profile -0.60 to 2.39; Wald -0.65 to 2.23), Wald p = 0.28. exp(printed) = 2.9 (0.55 to 11.0)
"Normal-weight controls" Results, model 1 "OR −4.39 (−7.04 to −1.97)", p < 0.001 Impossible as an OR. With four groups and an intercept, one group must be the reference and gets no coefficient of its own. Normal-weight controls are the reference implied by the other three contrasts. Their observed log-odds is ln(3/54) = -2.89, and the refit intercept is -2.89 (profile -4.30 to -1.89), so −4.39 is most consistent with the model intercept, which in the adjusted model refers to age 0
Depression Abstract; Results, model 2 "OR 1.09 (0.27 to 2.01)", p = 0.013 Read as an OR, the CI contains 1 and implies p = 0.87, contradicting p = 0.013. Read as a coefficient, it implies p = 0.014 (matches). Refit: log-odds 1.12 (profile 0.31 to 2.03), p = 0.010. exp(printed) = 3.0 (1.31 to 7.46)
Overweight Abstract; Results, model 2 "OR 1.52 (0.7 to 2.41)", p < 0.001 Read as an OR, the CI contains 1 and implies p = 0.18, contradicting p < 0.001. Read as a coefficient, it implies p = 0.0005 (matches). Refit: log-odds 1.46 (profile 0.68 to 2.32), p = 0.00045. exp(printed) = 4.6 (2.01 to 11.1)
Age, gender Results, model 1 "OR 0.01 (−0.04 to 0.06)", p = 0.621; "OR 0.61 (−0.19 to 1.47)", p = 0.145 Negative lower limits are impossible for ORs. As coefficients, implied p = 0.70 and 0.15. exp(printed) = 1.01 (0.96 to 1.06) per year and 1.84 (0.83 to 4.35)
Age, gender Results, model 2 "OR 0.62 (−0.04 to 0.06)", p = 0.647; "OR 0.01 (−0.19 to 1.47)", p = 0.137 The age estimate 0.62 lies outside its own CI (-0.04 to 0.06). The two point estimates appear to be transposed relative to their CIs and to model 1 (0.01 with -0.04 to 0.06; 0.61 with -0.19 to 1.47). Both CIs include 0 and both p-values are above 0.05, so significance still agrees on the coefficient reading
Size of understatement Abstract 2.44, 1.52, 1.09 exp(printed) / printed = 4.7×, 3.0×, 2.7×

In the refit of model 1, the printed intervals and the profile-likelihood intervals line up closely: 1.28 to 3.94 against 1.26 to 3.87, -0.33 to 2.87 against -0.37 to 2.74, and -0.6 to 2.4 against -0.60 to 2.39. For overweight + depressed, depression and overweight, the printed intervals also share the refit's upward skew (for example 1.16 below and 1.50 above the estimate, against 1.13 and 1.48 in the refit). That pattern is what R's confint() gives for a glm fit on the coefficient scale. The Methods state that all analyses used R 4.0.0.

The Wald back-calculation is less exact. On the coefficient reading it matches the printed p for overweight + depressed (0.0003 v. < 0.001), "normal-weight controls" (0.0007 v. < 0.001), depression (0.014 v. 0.013), overweight (0.0005 v. < 0.001) and model 1 gender (0.15 v. 0.145), and is close to it for model 1 age (0.70 v. 0.621). For overweight controls (0.064 v. 0.13) and normal-weight depressed (0.16 v. 0.28) it gives about half the printed value, a gap a reader rerunning the script will see. In these two rows the printed limits are close to the refit's limits, but the printed estimates (1.51, 1.08) are higher than the refit estimates (1.10, 0.79). The printed estimates therefore sit nearer the upper limit (1.84 below and 1.36 above; 1.68 below and 1.32 above). The back-calculation divides the estimate by an SE taken from the interval width, so a larger estimate with the same width gives a larger z and a smaller p-value. The printed p-values for these two rows are close to the unadjusted refit Wald p-values (0.15 and 0.28). Because the printed intervals look like asymmetric profile intervals, a Wald back-calculation from them is only approximate, and these two rows cannot be resolved further without the model output. The gap is in size only: on the coefficient reading, no printed interval contradicts its printed p-value on significance at 0.05.

Figure

Printed logistic-regression values, all labelled 'OR', on their own linear scale, with refits from Table 1

Each row is one group or main-effect estimate the paper prints with the label "OR"; the age and gender rows are not shown. Orange circles and lines are the printed estimate and 95% CI, on the linear scale on which they are printed. Blue squares and lines are log-odds coefficients refitted, without the age and gender covariates, from the hsCRP ⩾3 mg/L counts in Table 1, with profile-likelihood 95% intervals. The shaded region holds values below 0, which no odds ratio can take. The solid line at 0 is the null for a log-odds coefficient, and the dashed line at 1 is the null for an odds ratio. Whether each printed p-value (right) is below 0.05 matches whether its interval crosses 0, not whether it crosses 1. That is clearest for depression and overweight in model 2, whose intervals cross 1 but are reported as significant. The row labelled "normal-weight controls" has no blue refit because the reference group of a categorical predictor has no contrast of its own.

Printed values read as odds ratios v. exp(printed values), on a log odds-ratio axis

The same estimates on a log odds-ratio axis. Orange is the printed value taken literally as an OR. Lower limits below 0 cannot be placed on this axis and are drawn as arrows. Blue is exp(printed value), the odds ratio implied if the printed numbers are coefficients, labelled with its point estimate. The intercept-like "normal-weight controls" row is omitted because its printed value is negative. The age and gender rows of both models are not shown either.

Why it matters

The three figures in the abstract are the paper's quantitative answer to its second aim, "what extent overweight status and depression status contributed to clinically elevated hsCRP levels". As printed, they suggest modest associations: odds roughly 2.4, 1.5 and 1.1 times higher. If they are coefficients, the implied odds ratios are about 11.5, 4.6 and 3.0, a 2.7- to 4.7-fold difference. A reader citing these ORs, or pooling them in a meta-analysis, would carry the wrong magnitude. Read literally as ORs, the printed intervals also contradict the paper's own significance statements: the depression interval (0.27 to 2.01) and the overweight interval (0.7 to 2.41) both include 1. A reader who takes the label at face value could therefore conclude that neither main effect is significant, which is the opposite of what the abstract states.

The qualitative conclusions do not depend on the scale. Overweight-and-depressed participants have the highest odds, overweight contributes more than depression, and both are significant in model 2. Those orderings and significance calls are the same on either scale, and the Table 1 refits support them. The second-paragraph age and gender estimates are a separate, smaller transcription problem.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. State that the values reported in the logistic-regression paragraphs of the Results and in the abstract are log-odds coefficients (β), or replace them with odds ratios, exp(β), and their exponentiated CIs. If the printed values are the model coefficients, the abstract would read approximately: - "OR 11.5, 95% CI 3.6–51.4" for comorbid depression and overweight; - "OR 4.6, 2.0–11.1" for overweight; - "OR 3.0, 1.3–7.5" for depression.

These values should be confirmed from the original model output. 2. In model 1, report the reference group explicitly (apparently normal-weight controls), and either drop the "−4.39" or label it as the intercept rather than as a risk estimate for normal-weight controls. 3. In the model 2 paragraph, check the age and gender estimates, which appear transposed (0.62 is printed with the interval -0.04 to 0.06). 4. Consider giving the full model output (coefficients, SEs, intercept and coding) in a table or supplement.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9693673 (Psychol Med 2022;52(14):3289-3296, online 18 March 2021;
#                     doi:10.1017/S0033291721000088)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG) plus five candidate figure designs in ./candidates/.
# Inputs are only numbers printed in the paper (Results text and Table 1).
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975)

# ---------------------------------------------------------------- inputs
# Table 1, row "hsCRP >=3 mg/L (%)": events / group N
groups = {"OW_dep": (26, 69), "OW_ctl": (5, 35), "NW_dep": (6, 55), "NW_ctl": (3, 57)}
# Results, hsCRP paragraphs 1 and 2: (estimate, CI low, CI high, reported p) as printed with label "OR"
rep1 = {
    "OW_dep": (2.44, 1.28, 3.94, "<0.001"),
    "OW_ctl": (1.51, -0.33, 2.87, "0.13"),
    "NW_dep": (1.08, -0.6, 2.4, "0.28"),
    "NW_ctl": (-4.39, -7.04, -1.97, "<0.001"),
    "age": (0.01, -0.04, 0.06, "0.621"),
    "gender": (0.61, -0.19, 1.47, "0.145"),
}
rep2 = {
    "depression": (1.09, 0.27, 2.01, "0.013"),
    "overweight": (1.52, 0.7, 2.41, "<0.001"),
    "age (as printed)": (0.62, -0.04, 0.06, "0.647"),
    "gender (as printed)": (0.01, -0.19, 1.47, "0.137"),
}

def pnum(s):
    return 0.001 if s.startswith("<") else float(s)

def fmt(x, d=2):
    return f"{x:.{d}f}"

# ---------------------------------------------------------------- 1. Table 1 arithmetic
print("1. Table 1 hsCRP >=3 mg/L row")
for g, (k, n) in groups.items():
    print(f"   {g}: {k}/{n} = {100*k/n:.1f}%")
obs = np.array([[k, n - k] for k, n in groups.values()])
chi2, p_chi, dof, _ = stats.chi2_contingency(obs, correction=False)
print(f"   Pearson chi2 = {chi2:.2f}, df = {dof}, p = {p_chi:.2g}   (reported x2 = 25.96, p < 0.001)")
print(f"   total events = {obs[:,0].sum()}, non-events = {obs[:,1].sum()}, N = {obs.sum()}")

# ---------------------------------------------------------------- 2. reading the printed values two ways
def wald_from_ci(b, lo, hi):
    se = (hi - lo) / (2 * Z)
    return se, 2 * stats.norm.sf(abs(b / se))

def as_or(b, lo, hi):
    """p implied if (b, lo, hi) were an OR and its CI (Wald on log scale)."""
    if b <= 0 or lo <= 0:
        return None
    se = (np.log(hi) - np.log(lo)) / (2 * Z)
    return 2 * stats.norm.sf(abs(np.log(b) / se))

readings = {}
for title, rep in (("Model 1 (group status + age + gender)", rep1), ("Model 2 (depression + overweight + age + gender)", rep2)):
    print(f"\n2. {title}")
    print("   name | printed 'OR' (CI), p | impossible as OR? | p if log-odds | p if OR | exp(printed) = OR if log-odds")
    for k, (b, lo, hi, pr) in rep.items():
        _, pl = wald_from_ci(b, lo, hi)
        po = as_or(b, lo, hi)
        imp = (b <= 0) or (lo <= 0)
        ci_has_1 = (lo < 1 < hi)
        readings[k] = dict(b=b, lo=lo, hi=hi, pr=pr, p_logodds=pl, p_or=po, exp=(np.exp(b), np.exp(lo), np.exp(hi)))
        e = f"{np.exp(b):.3g} ({np.exp(lo):.3g} to {np.exp(hi):.3g})"
        pos = "impossible" if po is None else f"{po:.3f} (CI contains 1: {ci_has_1})"
        print(f"   {k}: {b} ({lo} to {hi}), p {pr} | {'YES' if imp else 'no'} | {pl:.3g} | {pos} | {e}")
    print("   Significance at 0.05 on the coefficient reading: does 'CI excludes 0' agree with 'printed p < 0.05'?")
    for k, (b, lo, hi, pr) in rep.items():
        excl0 = not (lo < 0 < hi)
        sig = pnum(pr) < 0.05
        print(f"      {k}: CI excludes 0 = {excl0}; printed p < 0.05 = {sig}; agree = {excl0 == sig}")
    print("   Point estimate inside its own printed CI?",
          {k: bool(v[1] <= v[0] <= v[2]) for k, v in rep.items()})

# CI asymmetry of the printed headline interval (Wald would be symmetric)
b, lo, hi, _ = rep1["OW_dep"]
print(f"\n   OW_dep printed interval: distance below = {b-lo:.2f}, above = {hi-b:.2f} (asymmetric; midpoint {(lo+hi)/2:.2f})")

# ---------------------------------------------------------------- 3. unadjusted logistic fits from Table 1 counts
names = list(groups)
y = np.array([groups[g][0] for g in names], float)
n = np.array([groups[g][1] for g in names], float)

def nll(beta, X):
    eta = X @ beta
    return -np.sum(y * eta - n * np.logaddexp(0, eta))

def fit(X, fixed=None):
    if fixed is None:
        r = optimize.minimize(nll, np.zeros(X.shape[1]), args=(X,), method="BFGS", options={"gtol": 1e-10})
        return r.x, r.fun
    j, val = fixed
    free = [i for i in range(X.shape[1]) if i != j]
    def f(bf):
        bb = np.empty(X.shape[1]); bb[free] = bf; bb[j] = val
        return nll(bb, X)
    r = optimize.minimize(f, np.zeros(len(free)), method="BFGS", options={"gtol": 1e-10})
    return None, r.fun

def summarize(X, labels):
    beta, f0 = fit(X)
    p = 1 / (1 + np.exp(-X @ beta))
    W = n * p * (1 - p)
    se = np.sqrt(np.diag(np.linalg.inv(X.T @ (W[:, None] * X))))
    out = {}
    crit = stats.chi2.ppf(0.95, 1) / 2
    for j, lab in enumerate(labels):
        g = lambda t: fit(X, (j, t))[1] - f0 - crit
        plo = optimize.brentq(g, beta[j] - 12 * se[j], beta[j])
        phi = optimize.brentq(g, beta[j], beta[j] + 12 * se[j])
        pw = 2 * stats.norm.sf(abs(beta[j] / se[j]))
        out[lab] = dict(b=beta[j], se=se[j], wlo=beta[j] - Z * se[j], whi=beta[j] + Z * se[j], plo=plo, phi=phi, p=pw)
        print(f"   {lab}: log-odds {beta[j]:.2f} (Wald {beta[j]-Z*se[j]:.2f} to {beta[j]+Z*se[j]:.2f}; "
              f"profile {plo:.2f} to {phi:.2f}), p = {pw:.2g}; OR = {np.exp(beta[j]):.2f} "
              f"(profile {np.exp(plo):.2f} to {np.exp(phi):.2f})")
    return out

print("\n3a. Unadjusted model 1 from Table 1 counts (reference = normal-weight controls)")
X1 = np.array([[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1], [1, 0, 0, 0]], float)
m1 = summarize(X1, ["intercept", "OW_dep", "OW_ctl", "NW_dep"])
print("\n3b. Unadjusted model 2 from Table 1 counts (main effects, no interaction)")
X2 = np.array([[1, 1, 1], [1, 0, 1], [1, 1, 0], [1, 0, 0]], float)  # intercept, depression, overweight
m2 = summarize(X2, ["intercept", "depression", "overweight"])

print("\n3b2. Interval skew: distance from estimate to lower / upper limit, printed v. unadjusted refit (profile)")
for k_rep, k_fit, mm in (("OW_dep", "OW_dep", m1), ("OW_ctl", "OW_ctl", m1), ("NW_dep", "NW_dep", m1),
                         ("depression", "depression", m2), ("overweight", "overweight", m2)):
    b_, lo_, hi_, _ = {**rep1, **rep2}[k_rep]
    f_ = mm[k_fit]
    print(f"   {k_rep}: printed {b_-lo_:.2f} below / {hi_-b_:.2f} above; "
          f"refit {f_['b']-f_['plo']:.2f} below / {f_['phi']-f_['b']:.2f} above")

print("\n3c. Crude ORs v. normal-weight controls, (a/b)/(c/d)")
crude = {}
for g in ["OW_dep", "OW_ctl", "NW_dep"]:
    k, N = groups[g]
    crude[g] = (k / (N - k)) / (3 / 54)
    print(f"   {g}: OR = {crude[g]:.2f}, log OR = {np.log(crude[g]):.2f}")

# intercept plausibility: NW controls' observed log-odds and a rough age-0 shift
lo_nw = np.log(3 / 54)
shift = lo_nw - 0.01 * 30 - 0.61 * 44 / 57
print(f"\n3d. Normal-weight controls: observed log-odds ln(3/54) = {lo_nw:.2f}; "
      f"shifted to age 0 and the other gender using printed age 0.01/yr (median age 30) "
      f"and gender 0.61 (77.2% female): about {shift:.2f}  (printed '-4.39', CI -7.04 to -1.97)")

print("\n3e. Headline ratios, exp(printed) / printed")
for k, r in (("OW_dep", rep1["OW_dep"]), ("overweight", rep2["overweight"]), ("depression", rep2["depression"])):
    print(f"   {k}: exp({r[0]}) = {np.exp(r[0]):.2f}; ratio to printed = {np.exp(r[0])/r[0]:.1f}x")

# ================================================================ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "axes.edgecolor": FG2, "axes.labelcolor": FG,
    "xtick.color": FG2, "ytick.color": FG, "text.color": FG, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})

def style(ax, grid_axis="x"):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
    ax.tick_params(axis="y", length=0)

def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)

ROWS = [  # (label, printed tuple, Table-1 refit dict or None)
    ("Overweight + depressed\n(model 1)", rep1["OW_dep"], m1["OW_dep"]),
    ("Overweight controls\n(model 1)", rep1["OW_ctl"], m1["OW_ctl"]),
    ("Normal-weight depressed\n(model 1)", rep1["NW_dep"], m1["NW_dep"]),
    ("'Normal-weight controls'\n(model 1)", rep1["NW_ctl"], None),
    ("Depression\n(model 2)", rep2["depression"], m2["depression"]),
    ("Overweight\n(model 2)", rep2["overweight"], m2["overweight"]),
]

# ---- design A: printed values on their own (linear) scale, both candidate nulls
def design_A(path):
    fig, ax = plt.subplots(figsize=(7.2, 4.9))
    ypos = np.arange(len(ROWS))[::-1].astype(float)
    ax.axvspan(-7.6, 0, color="#f1efea", zorder=0.5, lw=0)
    ax.text(-7.45, len(ROWS) - 0.35, "values < 0: impossible for an OR", color=FG2, fontsize=10, va="center")
    ax.axvline(0, color=FG, lw=1, zorder=1)
    ax.axvline(1, color=FG2, lw=1, ls=(0, (4, 3)), zorder=1)
    ax.text(-0.12, -0.75, "0 = null for a\nlog-odds coefficient", fontsize=10, color=FG, ha="right", va="center")
    ax.text(1.12, -0.75, "1 = null for\nan odds ratio", fontsize=10, color=FG2, ha="left", va="center")
    for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, ROWS):
        ax.plot([lo, hi], [yy + 0.12] * 2, color=ORANGE, lw=2, zorder=3, solid_capstyle="butt")
        ax.plot(b, yy + 0.12, "o", ms=8, color=ORANGE, zorder=4)
        if m is None:
            ax.text(-7.45, yy - 0.42, "no refit: a reference group\nhas no contrast of its own",
                    fontsize=10, color=BLUE, va="center", ha="left", linespacing=1.1,
                    bbox=dict(fc="#f1efea", ec="none", pad=1.5), zorder=2)
        if m is not None:
            ax.plot([m["plo"], m["phi"]], [yy - 0.14] * 2, color=BLUE, lw=2, zorder=3, solid_capstyle="butt")
            ax.plot(m["b"], yy - 0.14, "s", ms=7.5, color=BLUE, zorder=4)
        ax.text(4.75, yy, f"p {pr}" if pr.startswith("<") else f"p = {pr}", fontsize=10, color=FG2, va="center")
    ax.set_yticks(ypos)
    ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
    ax.set_xlim(-7.6, 5.6)
    ax.set_ylim(-1.6, len(ROWS) - 0.1)
    ax.set_xticks(range(-7, 5))
    ax.set_xlabel("Value on the scale the paper prints (labelled 'OR', 95% CI)")
    style(ax)
    ax.text(4.75, len(ROWS) - 0.35, "printed p", fontsize=10, color=FG2, va="center", weight="bold")
    h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed in the paper as 'OR (95% CI)'")
    h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="Log-odds refitted from Table 1 counts (unadjusted)")
    ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.36, 1.0), frameon=False, ncol=1, fontsize=10)
    fig.tight_layout()
    save(fig, path)

# ---- design B: odds-ratio (log) axis, printed-as-OR v. exp(printed)
def design_B(path):
    rows = [r for r in ROWS if r[1][0] > 0]
    fig, ax = plt.subplots(figsize=(7.2, 4.4))
    ypos = np.arange(len(rows))[::-1].astype(float)
    left = 0.1
    ax.axvline(1, color=FG2, lw=1, zorder=1)
    for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, rows):
        y0 = yy + 0.15
        if lo > 0:
            ax.plot([lo, hi], [y0] * 2, color=ORANGE, lw=2, zorder=3)
        else:
            ax.annotate("", xy=(left * 1.03, y0), xytext=(hi, y0),
                        arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12), zorder=3)
            ax.text(left * 1.12, y0 + 0.2, f"lower limit {str(lo).replace('-', '\u2212')} < 0", fontsize=10, color=ORANGE, va="center")
        ax.plot(b, y0, "o", ms=8, color=ORANGE, zorder=4)
        e, elo, ehi = np.exp(b), np.exp(lo), np.exp(hi)
        y1 = yy - 0.15
        ax.plot([elo, ehi], [y1] * 2, color=BLUE, lw=2, zorder=3)
        ax.plot(e, y1, "s", ms=7.5, color=BLUE, zorder=4)
        ax.text(ehi * 1.08, y1, f"{e:.1f}", fontsize=10, color=BLUE, va="center")
    ax.set_xscale("log")
    ax.set_xlim(left, 120)
    ax.set_xticks([0.1, 0.3, 1, 3, 10, 30, 100])
    ax.set_xticklabels(["0.1", "0.3", "1", "3", "10", "30", "100"])
    ax.set_yticks(ypos)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-0.7, len(rows) - 0.3)
    ax.set_xlabel("Odds ratio for hsCRP ⩾3 mg/L (log scale, 95% CI)")
    style(ax)
    h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed value read as an OR")
    h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="exp(printed value): the OR if it is a log-odds coefficient")
    ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.33, 1.0), frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)

# ---- design C: p-value consistency under the two readings
def design_C(path):
    keys = [("Overweight + depressed (m1)", "OW_dep"), ("Overweight controls (m1)", "OW_ctl"),
            ("Normal-weight depressed (m1)", "NW_dep"), ("'Normal-weight controls' (m1)", "NW_ctl"),
            ("Depression (m2)", "depression"), ("Overweight (m2)", "overweight")]
    fig, ax = plt.subplots(figsize=(7.2, 4.3))
    ypos = np.arange(len(keys))[::-1].astype(float)
    ax.axvline(0.05, color=FG2, lw=1, ls=(0, (4, 3)))
    ax.text(0.052, len(keys) - 0.45, "p = 0.05", fontsize=10, color=FG2)
    for yy, (lab, k) in zip(ypos, keys):
        r = readings[k]
        pr = pnum(r["pr"])
        ax.plot(pr, yy + 0.18, "o", ms=9, color=ORANGE, zorder=4)
        if r["pr"].startswith("<"):
            ax.annotate("", xy=(pr / 3, yy + 0.18), xytext=(pr, yy + 0.18),
                        arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=10))
        ax.plot(r["p_logodds"], yy, "s", ms=8, color=BLUE, zorder=4)
        if r["p_or"] is None:
            ax.text(1.5e-5, yy - 0.3, "OR reading impossible", fontsize=10, color=FG2, va="center", ha="left")
        else:
            ax.plot(r["p_or"], yy - 0.2, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
    ax.set_xscale("log")
    ax.set_xlim(1e-5, 1.5)
    ax.set_yticks(ypos)
    ax.set_yticklabels([k[0] for k in keys], fontsize=10)
    ax.set_xlabel("Two-sided p-value (log scale)")
    style(ax)
    h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="p printed in the paper"),
         plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=8, label="p implied by the printed CI read as log-odds"),
         plt.Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="p implied by the printed CI read as an OR")]
    ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.4, 1.0), frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)

# ---- design D: calculation flow for the headline abstract figure
def box(ax, x, y, w, h, txt, ec, fs=10.5, weight="normal"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.02",
                                fc=BG, ec=ec, lw=2))
    ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs, color=FG, weight=weight)

def arrow(ax, a, b, txt=None, off=(0, 0.03)):
    ax.annotate("", xy=b, xytext=a, arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.6, mutation_scale=14))
    if txt:
        ax.text((a[0] + b[0]) / 2 + off[0], (a[1] + b[1]) / 2 + off[1], txt, ha="center", fontsize=10, color=FG2)

def design_D(path):
    fig = plt.figure(figsize=(7.2, 4.0))
    ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
    ax.set_xlim(-0.02, 1.02); ax.set_ylim(0, 1); ax.axis("off")
    lr = np.log(crude["OW_dep"])
    b, lo, hi, _ = rep1["OW_dep"]
    X = [(0.0, 0.34), (0.40, 0.25), (0.71, 0.29)]
    top = ["Table 1, hsCRP ⩾3 mg/L\noverweight + depressed: 26/69\nnormal-weight controls: 3/57",
           f"log odds ratio\nln[(26/43)/(3/54)]\n= {lr:.2f}",
           f"crude odds ratio\n= {crude['OW_dep']:.1f}"]
    bot = ["Abstract and Results print\n'OR 2.44 (1.28–3.94)'",
           f"read as a log-odds\ncoefficient: {b:.2f}\n({lo:.2f} to {hi:.2f})",
           f"odds ratio = exp(2.44)\n= {np.exp(b):.1f}\n(95% CI {np.exp(lo):.1f} to {np.exp(hi):.1f})"]
    for (x, w), t in zip(X, top):
        box(ax, x, 0.62, w, 0.27, t, BLUE, fs=10)
    for (x, w), t, ec in zip(X, bot, [ORANGE, ORANGE, BLUE]):
        box(ax, x, 0.08, w, 0.27, t, ec, fs=10)
    for yy in (0.755, 0.215):
        arrow(ax, (0.355, yy), (0.385, yy))
        arrow(ax, (0.665, yy), (0.695, yy), "exp", off=(0, 0.035))
    ax.annotate("", xy=(0.525, 0.365), xytext=(0.525, 0.605),
                arrowprops=dict(arrowstyle="<|-|>", color=FG2, lw=1.6, mutation_scale=14))
    ax.text(0.545, 0.485, f"{lr:.2f} v. {b:.2f}: same (log-odds) scale;\nthe printed value is also adjusted\nfor age and gender",
            fontsize=10, color=FG2, va="center")
    ax.text(0.0, 0.985, "Blue boxes: computed from the paper's own numbers.  Orange boxes: as printed.", fontsize=10, color=FG2, va="top")
    save(fig, path)

# ---- design E: dumbbell, printed headline 'ORs' v. exp(printed)
def design_E(path):
    rows = [("Overweight + depressed v. normal-weight\ncontrols (abstract, model 1)", rep1["OW_dep"]),
            ("Overweight (abstract, model 2)", rep2["overweight"]),
            ("Depression (abstract, model 2)", rep2["depression"])]
    fig, ax = plt.subplots(figsize=(7.2, 3.3))
    ypos = np.arange(len(rows))[::-1].astype(float)
    ax.axvline(1, color=FG2, lw=1)
    for yy, (lab, (b, lo, hi, pr)) in zip(ypos, rows):
        e = np.exp(b)
        ax.annotate("", xy=(e / 1.08, yy), xytext=(b * 1.06, yy),
                    arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4, mutation_scale=12))
        ax.plot(b, yy, "o", ms=10, color=ORANGE, zorder=4)
        ax.plot(e, yy, "s", ms=9.5, color=BLUE, zorder=4)
        ax.text(b, yy + 0.27, f"{b:.2f}", ha="center", fontsize=10, color=ORANGE)
        ax.text(e, yy + 0.27, f"{e:.1f}", ha="center", fontsize=10, color=BLUE)
        ax.text(np.sqrt(b * e), yy - 0.33, f"x{e/b:.1f}", ha="center", fontsize=10, color=FG2)
    ax.set_xscale("log")
    ax.set_xlim(0.8, 20)
    ax.set_xticks([1, 2, 5, 10, 20]); ax.set_xticklabels(["1", "2", "5", "10", "20"])
    ax.set_yticks(ypos); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_ylim(-0.7, len(rows) - 0.3)
    ax.set_xlabel("Odds ratio (log scale)")
    style(ax)
    h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="'OR' printed in the abstract"),
         plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=9.5, label="exp(printed value): the OR if it is a log-odds coefficient")]
    ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.45, 1.0), frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)

designs = [("A_printed_scale_two_nulls", design_A), ("B_odds_ratio_axis", design_B),
           ("C_pvalue_consistency", design_C), ("D_calculation_flow", design_D),
           ("E_headline_dumbbell", design_E)]
for i, (nm, f) in enumerate(designs, 1):
    f(os.path.join(CAND, f"cand{i}_{nm}"))
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))

# contact sheet of the five candidates
imgs = [plt.imread(os.path.join(CAND, f"cand{i}_{nm}.png")) for i, (nm, _) in enumerate(designs, 1)]
fig, axs = plt.subplots(3, 2, figsize=(14.4, 15))
for i, ax in enumerate(axs.flat):
    ax.axis("off")
    if i < len(imgs):
        ax.imshow(imgs[i])
        ax.set_title(f"Candidate {i+1}: {designs[i][0][2:].replace('_', ' ')}", fontsize=13, loc="left")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=110)
plt.close(fig)
print("\nFigures written: fig1.svg/png, fig2.svg/png, candidates/cand1..5 (.svg/.png), candidates/contact_sheet.png")

Reported positive predictive value of 55% conflicts with the paper's own sensitivity, specificity and prevalence

Sci Adv (2023) · doi:10.1126/sciadv.adg7545 · PMC10732528

Summary

The paper reports that its serial panel of three bivariate miRNA ratios separates preeclampsia (PE) cases from controls with sensitivity 93%, specificity 79%, positive predictive value (PPV) 55% and negative predictive value (NPV) 89%. It states these figures in the Abstract, the Results and the Discussion, and the Results and Discussion add "given a 57.7% prevalence of PE in our study" (71 of 123 samples). At that prevalence, a sensitivity of 93% and a specificity of 79% give a PPV of 85.8%. The per-sample classifications in the paper's data file S5 give 66 true positives and 11 false positives, so PPV = 66/77 = 85.7%. The reported NPV and both likelihood ratios agree with the same inputs; only the PPV does not. The error understates the panel's performance in a headline figure, and it affects the paper's comparison with the sFlt1:PlGF ratio, whose main limitation the paper gives as a low PPV.

What the paper reports

Abstract:

Applying an iterative machine learning method identified three bivariate miRNA biomarkers (miR-522-3p/miR-4732-5p, miR-516a-5p/miR-144-3p, and miR-27b-3p/let-7b-5p) that, when applied serially, distinguished between PE cases of different severity and differentiated cases from controls with a sensitivity of 93%, specificity of 79%, positive predictive value (PPV) of 55%, and negative predictive value (NPV) of 89%.

Results, "Identification of a small panel of candidate bivariate biomarkers":

We then re-ran the algorithm an additional two times, correctly identifying all but five PE cases, three of which were diagnosed as superimposed PE and one mild PE (Fig. 7A).

This iterative panel of three bivariate biomarkers achieved a PPV of 55% at a sensitivity of 93%, specificity of 79%, a positive likelihood ratio (+LR) of 4.43, and a negative likelihood ratio (-LR) of 0.09, given a 57.7% prevalence of PE in our study.

Discussion:

Overall, this iterative process was able to identify over 90% of cases with a false-positive rate of less than 10% at each iteration and achieved a PPV of 55% at a sensitivity of 93% and specificity of 79% (+LR = 4.43, -LR = 0.09) given a prevalence of PE of 57.7% in our study.

Results, "Characteristics of the study population" (the cohort the panel was applied to):

Cases were defined as pregnancies with an adjudicated PE diagnosis (n = 71), and controls (n = 52) were from participants with either hypertension that did not meet the criteria for PE or normal (nonhypertensive) pregnancy outcomes. The pass-filter 71 cases and 52 controls were subjected to further analysis.

Materials and Methods, "Candidate bivariate marker selection" (the definition used):

PPV was calculated as PPV = True Positives/(True Positives + False Positives), and NPV was calculated as NPV = True Negatives/(True Negatives + False Negatives).

Introduction (the comparison the PPV feeds into):

The primary limitation of the sFlt1:PlGF ratio is its relatively low positive predictive value (PPV), which numerous studies have reported to be between 30 and 65% (9–12).

What the paper's own numbers show

Method. I recomputed the PPV in two independent ways. (1) From the paper's rounded rates, using Bayes' rule, PPV = Se·p / (Se·p + (1 − Sp)(1 − p)), with Se = 0.93, Sp = 0.79 and p = 0.577. The same formula is stated in the medRxiv preprint of this paper. (2) From counts. Data file S5 ("Refers to Figure 7") lists all 123 samples with the iteration (1, 2 or 3) at which each one was flagged as PE, or "Not Removed". Flagged cases are true positives, unflagged cases false negatives, flagged controls false positives and unflagged controls true negatives. I applied the Methods definitions to those counts. I also computed what a PPV of 55% would require, both in prevalence and in false positives. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Prevalence Results; Discussion 57.7% 71/123 = 57.72%
Classification counts Data file S5 cases flagged at iterations 1/2/3: 40/15/11, not flagged 5; controls flagged 3/4/4, not flagged 41 TP = 66, FN = 5, FP = 11, TN = 41
Sensitivity Abstract; Results; Discussion 93% 66/71 = 92.96%
Specificity Abstract; Results; Discussion 79% 41/52 = 78.85%
PPV Abstract; Results; Discussion 55% 85.8% from the rounded rates; 66/77 = 85.71% from the counts
NPV Abstract 89% 89.2% from the rounded rates; 41/46 = 89.13% from the counts
+LR Results; Discussion 4.43 0.93/0.21 = 4.43; 4.39 from the counts
−LR Results; Discussion 0.09 0.07/0.79 = 0.089; 0.089 from the counts
PPV if specificity were anywhere from 38/52 to 44/52 (TP fixed at 66) derived — 82.5% to 89.2%
Prevalence at which Se 93% and Sp 79% give PPV 55% derived — 21.6%; the NPV there would be 97.6%, not 89%
False positives needed for PPV 55% with 66 true positives derived — 66/0.55 − 66 = 54, more than the 52 controls
PPV of the miRNA panel alone in the 70-patient sFlt1:PlGF subset (42 TP, 6 FP) Results not stated 42/48 = 87.5%
Combined miRNA then sFlt1:PlGF (42 TP, 2 FP, 21 TN, 5 FN) Abstract; Results Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8% 89.36%, 91.3%, 95.5%, 80.8% (consistent)
Preprint inputs (prevalence 57.8%, Sp 78.8%) medRxiv preprint, Methods PPV 55% 85.7% with the preprint's own formula

Every other metric of the panel agrees with its inputs to the printed precision. The PPV is the only one that does not, and it is about 31 percentage points below the value the paper's own numbers give.

Figure

PPV as a function of prevalence at the reported sensitivity and specificity, with the reported and recomputed PPV at the study prevalence

Figure 1. The blue curve is the PPV that a test with the paper's reported sensitivity (93%) and specificity (79%) has at each prevalence (Bayes' rule). The thin vertical line marks the study prevalence the paper states, 71/123 = 57.7%. At that prevalence the curve gives 85.8% (blue diamond), and the counts in data file S5 give 66/77 = 85.7%. The paper's reported 55% (orange circle) lies well below the curve. The curve reaches 55% only at a prevalence of 21.6% (open circle), and at that prevalence the NPV would be 97.6%, not the 89% the paper reports. All values come from the paper or from recompute.py.txt.

Calculation flow from data file S5 counts to sensitivity, specificity, NPV and PPV

Figure 2. The count reconstruction from data file S5. Summing the cases and controls flagged at each of the three iterations gives TP 66, FN 5, FP 11 and TN 41. These counts reproduce the reported sensitivity, specificity and NPV, but give a PPV of 85.7% against the reported 55%. The lower panel shows that a 55% PPV would need 54 false positives, more than the 52 controls in the cohort.

Why it matters

Explanations considered

None of these explanations reconciles 55% with the paper's other figures. I have not found what calculation produced it.

A related observation (not part of the main claim). For the 18-patient independent validation cohort (11 cases, 7 controls), the Results report "a PPV of 85% at a sensitivity of 91% and specificity of 57%". A sensitivity of 91% with 11 cases is 10/11, which agrees with the Discussion's "we found all but one PE case". A specificity of 57% with 7 controls is 4/7, which leaves 3 false positives. That gives PPV = 10/13 = 76.9%. A PPV of 85% (11/13 = 84.6%) would require all 11 cases to be detected. Data file S6 gives only the cohort's demographics, not per-sample calls, so I could not check this from counts.

Status

As of 2026-10-23:

Sources: article (DOI); PMC10732528; medRxiv preprint; supplementary files via the Europe PMC supplementaryFiles service.

Suggested correction

In the Abstract, the Results ("Identification of a small panel of candidate bivariate biomarkers") and the Discussion, the PPV of the three-bivariate panel in the 123-sample cohort should read 86% (66/77 = 85.7% from data file S5; 85.8% from the rounded sensitivity, specificity and prevalence) instead of 55%. Sensitivity, specificity, NPV, +LR and −LR need no change. If the 55% was meant to describe a different population or prevalence, the text should say which one, and the NPV should then be recomputed for it. The authors may also wish to check the validation-cohort PPV of 85% in the Results. With the stated sensitivity (10/11) and specificity (4/7) it computes to 10/13 = 77%.

Recomputation script (python recompute.py.txt)
# Recompute the diagnostic metrics of the three-bivariate miRNA panel in
# Sci Adv 2023, doi:10.1126/sciadv.adg7545 (PMC10732528), and draw the figures.
#
# Run:  python recompute.py.txt
# Every input below is a number printed in the article or in its data file S5
# (per-sample iteration at which each of the 123 samples was flagged as PE).
# Outputs: fig1.svg/png, fig2.svg/png and five candidate designs in candidates/.

import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
import numpy as np

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs
CASES, CONTROLS = 71, 52                 # Results, "Characteristics of the study population"
N = CASES + CONTROLS
REP = dict(se=0.93, sp=0.79, ppv=0.55, npv=0.89, plr=4.43, nlr=0.09, prev=0.577)  # Abstract/Results/Discussion

# Data file S5: iteration at which each sample was flagged (1, 2, 3) or "Not Removed"
S5 = {"case": {1: 40, 2: 15, 3: 11, "not": 5}, "control": {1: 3, 2: 4, 3: 4, "not": 41}}
TP = sum(v for k, v in S5["case"].items() if k != "not")
FN = S5["case"]["not"]
FP = sum(v for k, v in S5["control"].items() if k != "not")
TN = S5["control"]["not"]
assert TP + FN == CASES and FP + TN == CONTROLS


def ppv_f(se, sp, p):
    return se * p / (se * p + (1 - sp) * (1 - p))


def npv_f(se, sp, p):
    return sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)


def pct(x, d=1):
    return f"{100 * x:.{d}f}%"


print("=== 1. Prevalence")
prev = CASES / N
print(f"prevalence = {CASES}/{N} = {pct(prev, 2)}  (reported 57.7%)")

print("\n=== 2. From the paper's rounded rates (Se 93%, Sp 79%, prevalence 57.7%)")
se, sp, p = REP["se"], REP["sp"], REP["prev"]
ppv_r, npv_r = ppv_f(se, sp, p), npv_f(se, sp, p)
plr_r, nlr_r = se / (1 - sp), (1 - se) / sp
print(f"PPV = {se}*{p} / ({se}*{p} + {1-sp:.2f}*{1-p:.3f}) = {pct(ppv_r)}   (reported 55%)")
print(f"NPV = {pct(npv_r)} (reported 89%)   +LR = {plr_r:.2f} (reported 4.43)   -LR = {nlr_r:.3f} (reported 0.09)")

print("\n=== 3. From the counts in data file S5")
se_c, sp_c = TP / CASES, TN / CONTROLS
ppv_c, npv_c = TP / (TP + FP), TN / (TN + FN)
plr_c, nlr_c = se_c / (1 - sp_c), (1 - se_c) / sp_c
print(f"cases flagged at iterations 1/2/3 = {S5['case'][1]}/{S5['case'][2]}/{S5['case'][3]}, not flagged = {FN}")
print(f"controls flagged at iterations 1/2/3 = {S5['control'][1]}/{S5['control'][2]}/{S5['control'][3]}, not flagged = {TN}")
print(f"TP={TP} FN={FN} FP={FP} TN={TN}")
print(f"Se = {TP}/{CASES} = {pct(se_c, 2)}   Sp = {TN}/{CONTROLS} = {pct(sp_c, 2)}")
print(f"PPV = {TP}/{TP+FP} = {pct(ppv_c, 2)}   NPV = {TN}/{TN+FN} = {pct(npv_c, 2)}")
print(f"+LR = {plr_c:.2f}   -LR = {nlr_c:.3f}")

print(f"gap between recomputed and reported PPV = {100*(ppv_c-REP['ppv']):.1f} percentage points")
for it in (1, 2, 3):
    t, f_ = S5["case"][it], S5["control"][it]
    print(f"  iteration {it} alone: PPV = {t}/{t+f_} = {pct(t/(t+f_))}")

print("\n=== 4. Sensitivity of the PPV to the exact specificity count (TP fixed at 66)")
sweep = []
for tn in range(38, 45):
    fp = CONTROLS - tn
    sweep.append(TP / (TP + fp))
    print(f"  TN={tn} FP={fp}: Sp={pct(tn/CONTROLS)} PPV={pct(TP/(TP+fp))}")
print(f"lowest PPV in this range = {pct(min(sweep))}")

print("\n=== 5. What would a PPV of 55% require?")
target = REP["ppv"]
p55 = target * (1 - sp) / (se * (1 - target) + target * (1 - sp))
print(f"prevalence giving PPV 55% at Se 93%/Sp 79% = {pct(p55)}; NPV at that prevalence = {pct(npv_f(se, sp, p55))} (reported 89%)")
fp55 = TP / target - TP
print(f"false positives needed with TP={TP}: {fp55:.1f} (only {CONTROLS} controls exist)")

print("\n=== 6. Other ratios tried, none gives 55%")
print(f"TP/(TP+FP+FN) = {pct(TP/(TP+FP+FN))}   accuracy = {pct((TP+TN)/N)}   TN/(TN+FP+FN) = {pct(TN/(TN+FP+FN))}")
print(f"1 - PPV = {pct(1-ppv_c)}   FP/(FP+TN) = {pct(FP/CONTROLS)}   TP/N = {pct(TP/N)}")

print("\n=== 7. Preprint inputs (medRxiv 10.1101/2023.01.26.23285072: prevalence 57.8%, Sp 78.8%)")
print(f"PPV formula stated in the preprint, applied to Se 0.93, Sp 0.788, p 0.578 = {pct(ppv_f(0.93, 0.788, 0.578))}")

print("\n=== 8. sFlt1:PlGF subset (n = 70), Results")
subset = {"miRNA panel alone": (42, 6, 17, 5),
          "miRNA then sFlt1:PlGF": (42, 2, 21, 5),
          "sFlt1:PlGF only on miRNA positives": (39, 0, 23, 8)}
for lab, (tp, fp, tn, fn) in subset.items():
    print(f"  {lab}: Se={pct(tp/(tp+fn), 2)} Sp={pct(tn/(tn+fp))} PPV={pct(tp/(tp+fp))} NPV={pct(tn/(tn+fn))}")
print(f"  PPV gain from adding sFlt1:PlGF: {100*(0.955-ppv_c):.1f} points vs full-cohort {pct(ppv_c)}, "
      f"{100*(42/44-42/48):.1f} points vs subset {pct(42/48)}")
print("  (reported for 'then': Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8%; for 'only on positives': Se 82.9%, Sp 100%)")

print("\n=== 9. Side observation: independent validation cohort (11 cases, 7 controls)")
vtp, vfn, vtn, vfp = 10, 1, 4, 3      # Se 91% = 10/11 ('all but one PE case'), Sp 57% = 4/7
print(f"Se={pct(vtp/11)} Sp={pct(vtn/7)} PPV={vtp}/{vtp+vfp} = {pct(vtp/(vtp+vfp))} (reported 85%); 11/13 = {pct(11/13)}")

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK2,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "axes.titlesize": 11.5, "axes.titleweight": "bold", "axes.titlelocation": "left",
    "svg.fonttype": "none", "lines.linewidth": 2,
})


def style_axes(ax, grid_axis="both"):
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.grid(True, axis=grid_axis, color=GRID, linewidth=0.8)
    ax.set_axisbelow(True)


def save(fig, path):
    fig.savefig(path + ".svg")
    fig.savefig(path + ".png", dpi=200)
    plt.close(fig)
    print("wrote", os.path.relpath(path, HERE) + ".svg/.png")


print("\n=== figures")

# A. Dot comparison: reported vs recomputed for the four percentage metrics
def fig_A(path):
    fig, ax = plt.subplots(figsize=(W, 3.8))
    rows = [("Sensitivity", REP["se"], se_c, f"{TP}/{CASES}"),
            ("Specificity", REP["sp"], sp_c, f"{TN}/{CONTROLS}"),
            ("NPV", REP["npv"], npv_c, f"{TN}/{TN+FN}"),
            ("PPV", REP["ppv"], ppv_c, f"{TP}/{TP+FP}")]
    d = 0.14
    for i, (lab, r, c, frac) in enumerate(rows):
        y = len(rows) - 1 - i
        ax.scatter(r * 100, y + d, s=80, color=ORANGE, marker="o", zorder=3, edgecolor=BG, linewidth=1.2)
        ax.scatter(c * 100, y - d, s=80, color=BLUE, marker="D", zorder=4, edgecolor=BG, linewidth=1.2)
        if lab == "PPV":
            ax.text(r * 100 - 2.5, y + d, f"reported {r*100:.0f}%", ha="right", va="center", fontsize=10)
            ax.text(c * 100 - 2.5, y - d, f"{frac} = {c*100:.1f}%", ha="right", va="center", fontsize=10)
        else:
            ax.text(min(r, c) * 100 - 2.5, y, f"reported {r*100:.0f}%;  {frac} = {c*100:.1f}%",
                    ha="right", va="center", fontsize=10, color=INK2)
    ax.set_yticks(range(len(rows)))
    ax.set_yticklabels([r[0] for r in rows][::-1])
    ax.set_xlim(0, 100)
    ax.set_ylim(-0.5, len(rows) - 0.5)
    ax.set_xlabel("Percent (whole cohort: 71 cases, 52 controls)")
    style_axes(ax, "x")
    ax.scatter([], [], s=70, color=ORANGE, marker="o", label="Reported in the paper")
    ax.scatter([], [], s=70, color=BLUE, marker="D", label="Recomputed from data file S5 counts")
    ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2, frameon=False, fontsize=10)
    ax.set_title("Only the PPV disagrees with the panel's own classification counts")
    fig.tight_layout()
    save(fig, path)


# B. PPV as a function of prevalence at the reported Se/Sp
def fig_B(path):
    fig, ax = plt.subplots(figsize=(W, 4.3))
    ps = np.linspace(0.005, 0.995, 400)
    ax.plot(ps * 100, ppv_f(se, sp, ps) * 100, color=BLUE, lw=2, zorder=2,
            label="PPV implied by Se 93% and Sp 79%")
    ax.axvline(p * 100, color=INK2, lw=0.8, zorder=1)
    ax.text(p * 100 + 1, 3, f"study prevalence\n71/123 = {prev*100:.1f}%", color=INK2, fontsize=10, va="bottom")
    ax.plot([p55 * 100, p * 100], [55, 55], color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
    ax.scatter(p55 * 100, 55, s=64, facecolor=BG, edgecolor=BLUE, linewidth=2, zorder=4)
    ax.annotate(f"the curve reaches 55% only at a prevalence\nof {p55*100:.1f}%, where NPV would be "
                f"{npv_f(se, sp, p55)*100:.1f}%, not 89%",
                (p55 * 100, 55), xytext=(8, -10), textcoords="offset points", ha="left", va="top",
                fontsize=10, color=INK2)
    ax.scatter(p * 100, ppv_r * 100, s=110, color=BLUE, marker="D", zorder=5, edgecolor=BG, linewidth=1.5,
               label="Recomputed PPV at 57.7%")
    ax.annotate(f"recomputed {ppv_r*100:.1f}%\n(counts: {TP}/{TP+FP} = {ppv_c*100:.1f}%)", (p * 100, ppv_r * 100),
                xytext=(12, -8), textcoords="offset points", ha="left", va="top", fontsize=10)
    ax.scatter(p * 100, 55, s=110, color=ORANGE, marker="o", zorder=5, edgecolor=BG, linewidth=1.5,
               label="Reported PPV at 57.7%")
    ax.annotate("reported 55%", (p * 100, 55), xytext=(10, 0), textcoords="offset points",
                ha="left", va="center", fontsize=10)
    ax.set_xlim(0, 100); ax.set_ylim(0, 100)
    ax.set_xlabel("Prevalence of preeclampsia (%)")
    ax.set_ylabel("Positive predictive value (%)")
    style_axes(ax)
    ax.legend(loc="upper left", frameon=False, fontsize=10)
    ax.set_title("At the study's prevalence, Se 93% and Sp 79% give a PPV of 86%")
    fig.tight_layout()
    save(fig, path)


def box(ax, x, y, w, h, text, edge, fs=10.5, weight="normal", fc=BG, color=INK):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
                                linewidth=2, edgecolor=edge, facecolor=fc))
    if text:
        ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, fontweight=weight,
                color=color, linespacing=1.3)


def arrow(ax, a, b, color=INK2):
    ax.add_patch(FancyArrowPatch(a, b, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=color))


# C. Calculation flow from data file S5 counts
def fig_C(path):
    fig = plt.figure(figsize=(W, 4.2))
    ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
    ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    box(ax, 0.005, 0.56, 0.19, 0.34, "Data file S5\n123 samples,\neach with the\niteration that\nflagged it as PE", INK2, fs=10)
    box(ax, 0.235, 0.75, 0.30, 0.19,
        f"71 cases\nflagged {S5['case'][1]}+{S5['case'][2]}+{S5['case'][3]} = {TP}  (TP)\nnot flagged: {FN}  (FN)", BLUE, fs=10)
    box(ax, 0.235, 0.52, 0.30, 0.19,
        f"52 controls\nflagged {S5['control'][1]}+{S5['control'][2]}+{S5['control'][3]} = {FP}  (FP)\nnot flagged: {TN}  (TN)", BLUE, fs=10)
    arrow(ax, (0.20, 0.78), (0.23, 0.83)); arrow(ax, (0.20, 0.68), (0.23, 0.62))
    arrow(ax, (0.545, 0.73), (0.575, 0.73))
    ax.text(0.59, 0.96, "Recomputed", fontsize=10, color=BLUE, fontweight="bold", va="center")
    ax.text(0.995, 0.96, "Reported", fontsize=10, color=ORANGE, fontweight="bold", va="center", ha="right")
    outs = [(0.87, f"Se {TP}/{CASES} = {se_c*100:.1f}%", "93%  (matches)", True),
            (0.77, f"Sp {TN}/{CONTROLS} = {sp_c*100:.1f}%", "79%  (matches)", True),
            (0.67, f"NPV {TN}/{TN+FN} = {npv_c*100:.1f}%", "89%  (matches)", True),
            (0.57, f"PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", "55%  (no match)", False)]
    for y, t, r, ok in outs:
        ax.text(0.59, y, t, fontsize=10, color=INK, va="center")
        ax.text(0.995, y, r, fontsize=10, ha="right", va="center", color=INK2 if ok else INK)
    ax.add_patch(FancyBboxPatch((0.585, 0.53), 0.405, 0.08, boxstyle="round,pad=0.004,rounding_size=0.01",
                                linewidth=1.5, edgecolor=ORANGE, facecolor="none"))
    ax.plot([0.005, 0.995], [0.42, 0.42], color=GRID, lw=1)
    ax.text(0.005, 0.34, "What a PPV of 55% would require", fontsize=10.5, fontweight="bold", color=INK, va="center")
    ax.text(0.005, 0.23, f"With the same {TP} true positives: {TP}/0.55 - {TP} = {fp55:.0f} false positives, "
            f"but there are only {CONTROLS} controls.", fontsize=10, color=INK, va="center")
    ax.text(0.005, 0.13, f"With Se 93% and Sp 79%: a prevalence of {p55*100:.1f}% (the paper states "
            f"{REP['prev']*100:.1f}%),", fontsize=10, color=INK, va="center")
    ax.text(0.005, 0.05, f"at which the NPV would be {npv_f(se, sp, p55)*100:.1f}% rather than the reported 89%.",
            fontsize=10, color=INK, va="center")
    save(fig, path)


# D. False positives observed vs required
def fig_D(path):
    fig, ax = plt.subplots(figsize=(W, 2.8))
    vals = [("Observed\n(data file S5)", FP, BLUE, None, f"{FP}   PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%"),
            ("Needed for the\nreported PPV of 55%", fp55, ORANGE, "//", f"{fp55:.0f}   PPV {TP}/{TP+fp55:.0f} = 55%")]
    for i, (lab, v, c, h, txt) in enumerate(vals):
        y = 1 - i
        ax.barh(y, v, height=0.46, color=c if h is None else BG, edgecolor=c, hatch=h, linewidth=2)
        ax.text(v + 1.5, y, txt, va="center", ha="left", fontsize=10)
    ax.axvline(CONTROLS, color=INK, lw=1)
    ax.text(CONTROLS - 1, 1.48, f"all {CONTROLS} controls", ha="right", va="center", fontsize=10, color=INK2)
    ax.set_yticks([1, 0]); ax.set_yticklabels([v[0] for v in vals])
    ax.set_xlim(0, 100); ax.set_ylim(-0.45, 1.7)
    ax.set_xlabel(f"Controls classified as preeclampsia (true positives fixed at {TP})")
    style_axes(ax, "x")
    fig.tight_layout(rect=(0, 0, 1, 0.9))
    fig.suptitle(f"A 55% PPV would need {fp55:.0f} false positives; the cohort has only {CONTROLS} controls",
                 x=0.01, ha="left", fontsize=11.5, fontweight="bold")
    save(fig, path)


# E. Text panels: the three reported locations vs the paper's own inputs
def fig_E(path):
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
    ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    quotes = [("Abstract", "“...sensitivity of 93%, specificity of 79%,\npositive predictive value (PPV) of 55%,\nand negative predictive value (NPV) of 89%.”"),
              ("Results (Fig. 7 paragraph)", "“...achieved a PPV of 55% at a sensitivity of\n93%, specificity of 79%, ... (+LR) of 4.43, ...\n(-LR) of 0.09, given a 57.7% prevalence...”"),
              ("Discussion", "“...achieved a PPV of 55% at a sensitivity of\n93% and specificity of 79% (+LR = 4.43,\n-LR = 0.09) given a prevalence of PE of 57.7%...”")]
    ys = [0.65, 0.345, 0.04]
    for (h, q), y in zip(quotes, ys):
        box(ax, 0.005, y, 0.53, 0.26, "", ORANGE)
        ax.text(0.02, y + 0.215, h, fontsize=10.5, fontweight="bold", va="center", color=INK)
        ax.text(0.02, y + 0.095, q, fontsize=10, va="center", color=INK, linespacing=1.3)
    bx, by, bw, bh = 0.60, 0.22, 0.385, 0.56
    box(ax, bx, by, bw, bh, "", BLUE)
    ax.text(bx + 0.02, by + bh - 0.05, "The paper's own inputs", fontsize=10.5, fontweight="bold", va="center")
    lines = [("Se 0.93, Sp 0.79,", False), ("prevalence 71/123 = 0.577", False),
             (f"PPV = {ppv_r*100:.1f}%  (reported 55%)", True),
             (f"NPV = {npv_r*100:.1f}%  (reported 89%)", False),
             (f"+LR = {plr_r:.2f}  (reported 4.43)", False),
             (f"-LR = {nlr_r:.3f}  (reported 0.09)", False),
             (f"S5 counts: PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", True)]
    for k, (t, b) in enumerate(lines):
        ax.text(bx + 0.02, by + bh - 0.12 - k * 0.062, t, fontsize=10, va="center",
                fontweight="bold" if b else "normal", color=INK)
    for y in ys:
        ax.add_patch(FancyArrowPatch((0.54, y + 0.13), (bx - 0.005, 0.50), arrowstyle="-|>", mutation_scale=12,
                                     lw=1.3, color=INK2))
    ax.text(bx + bw / 2, 0.86, "same 55% in all three places;\nonly the PPV fails the check", fontsize=10,
            color=INK2, ha="center", va="center")
    ax.text(0.005, 0.975, "Orange boxes: the paper's text", fontsize=10, color=INK2, va="center")
    ax.text(0.995, 0.975, "Blue box: recomputed", fontsize=10, color=INK2, va="center", ha="right")
    save(fig, path)


fig_A(os.path.join(CAND, "A_metric_dots"))
fig_B(os.path.join(CAND, "B_ppv_vs_prevalence"))
fig_C(os.path.join(CAND, "C_count_flow"))
fig_D(os.path.join(CAND, "D_false_positives_needed"))
fig_E(os.path.join(CAND, "E_text_panels"))
# provisional choice for the write-up, pending a pick among candidates/A-E
fig_B(os.path.join(HERE, "fig1"))
fig_C(os.path.join(HERE, "fig2"))

Reported CO2 storage capacities per km³ are about 920 times what the paper's inputs give

Sci Adv (2024) · doi:10.1126/sciadv.adq0515 · PMC11566992

Summary

The paper's only quantitative estimate of reservoir storage capacity is a first-order calculation. It fills the 5.7% effective porosity of a basalt reservoir with a fluid carrying 0.65 M CO2 (6.5 M once the chelating agent GLDA has fully degraded) and reports capacities of 1.5 and 15 Gt CO2 per km³. Multiplying the paper's own inputs (10⁹ m³ × 0.057 × 650 mol/m³ × 44.01 g/mol) gives 0.0016 and 0.016 Gt per km³ (1.6 and 16 Mt). The reported values are about 920 times larger, which fits a megatonne-to-gigatonne unit slip, and they would require a pore volume 52 times the volume of the rock. The same inflated figure produces the claim that "up to 20 Gt of CO2 annually" fits in 1.3 km³ (1/25,000 of Iceland's active rift zone): with the paper's inputs, 1.3 km³ holds about 0.021 Gt, and the stated "two orders of magnitude higher" comparison with the cited 0.18 Gt/km³ becomes about 11 times lower.

What the paper reports

All passages are from the article as archived in PubMed Central. The archive text writes km3 and CO2 for km³ and CO₂. Where the paper names the cited study (its reference 17) by author, the name is replaced here by "[the cited study]"; everything else is verbatim.

The inputs:

"Specifically, the total porosity ranged from 10.99 to 15.23%, of which only 18.7 to 57.5% were connected porosity, resulting in a mean effective porosity of 4.8%. Assuming an enhancement in pore properties comparable to that observed in the experiment with injection of the acidic GLDA solution, the effective porosity was anticipated to increase from 4.8 to 5.7%." (Results, "CO2 mineralization with GLDA biodegradation and overall evaluation", para 2)

"if a solution containing 20 wt % GLDA-Na4 (C9H9NNa4O8) was used for injection, then CO2 concentration is expected to increase 10-fold, from 0.65 M to 0.65 + 0.65 × 9 (number of carbon atoms in GLDA) = 6.5 M." (Same subsection, para 1)

The capacity calculation and the claims built on it:

"When filling all effective porosity with a CO2-charged 20 wt % GLDA-Na4 seawater solution (CO2 concentration estimated as 0.65 M), the reservoir exhibited a CO2 storage capacity of 1.5 Gt/km3." (Same subsection, para 2)

"With complete degradation of GLDA, the concentration of CO2 in the fluid would attain a value of 6.5 M, resulting in a reservoir capable of storing 15 Gt of CO2/km3, two orders of magnitude higher than the value reported by [the cited study] (17) based on CO2 capture and storage using fresh water (a mean value of 0.18 Gt of CO2/km3)." (Same subsection, para 2)

"Up to 20 Gt of CO2 annually could potentially be sequestered in a 1.3 km3 reservoir volume, which is 1/25,000 of the active rift zone in Iceland (area of 33,000 km2, thickness of 1 km)." (Same subsection, para 2)

The abstract's closing claim:

"Promising CO2 storage and mineralization capacities two orders higher than current techniques, this approach reduces required reservoir volume while enhancing efficiency." (Abstract)

The paper uses M for mol/L throughout. For example, the Methods give "~0.0325, 0.065, 0.162, 0.325, and 0.65 M" for 1 to 20 wt % GLDA solutions (Materials and Methods, "Powder experiments", para 2).

What the paper's own numbers show

Method. The paper's stated rule is to fill all effective porosity of the rock with fluid at the stated CO2 concentration. For 1 km³ of rock (10⁹ m³) this gives:

capacity = 10⁹ m³ × effective porosity × concentration (mol/L × 1,000 L/m³) × 44.01 g/mol ÷ 10¹⁵ g/Gt.

The only constant in the capacity calculation not taken from the paper is the molar mass of CO2. One ceiling uses an assumed dense-CO2 density of 700 kg/m³, and one comparison uses an assumed basalt density of 2,950 kg/m³. recompute.py.txt evaluates the expression, inverts it to find the porosity the reported values would need, and recomputes the 1.3 km³ and 1/25,000 figures. It also takes the cited 0.18 Gt/km³ back to the cited study's own tables.

Quantity Location Reported Recomputed
Pore fluid in 1 km³ at 5.7% effective porosity Results, "CO2 mineralization …", para 2 not stated 5.7 × 10⁷ m³
CO2 in that fluid at 0.65 M same not stated 3.7 × 10¹⁰ mol = 1.63 × 10¹² g
Storage capacity at 0.65 M same 1.5 Gt/km³ 0.0016 Gt/km³ (1.6 Mt/km³); reported is 920× larger
Concentration after full GLDA degradation same, para 1 0.65 + 0.65 × 9 = 6.5 M 6.5 M (consistent)
Storage capacity at 6.5 M same, para 2 15 Gt/km³ 0.016 Gt/km³ (16 Mt/km³); reported is 920× larger
Effective porosity the reported values would need derived 5.7% stated 5,244% (52 m³ of fluid per m³ of rock), at either concentration
CO2 mass per m³ of rock implied by the reported values derived 1.5 / 15 Gt/km³ 1,500 / 15,000 kg per m³ of rock
Ceiling: 6.5 M fluid filling 100% of the rock derived – 0.29 Gt/km³
Ceiling: dense CO2 (700 kg/m³, assumed) filling all 5.7% pores derived – 0.040 Gt/km³
Reservoir volume for 20 Gt same, para 2 1.3 km³ = 1/25,000 of 33,000 km³ 20 / 15 = 1.33 km³ and 33,000 / 25,000 = 1.32 km³ (the reported chain is self-consistent with 15 Gt/km³)
CO2 held in 1.3 km³ at 6.5 M same "up to 20 Gt" 0.021 Gt (21 Mt)
Reservoir volume for 20 Gt at the recomputed capacity derived 1.3 km³ 1,227 km³ = 1/27 of the 33,000 km³ rift zone
Cited comparator, from the cited study's own tables cited study, Tables 7 and 8 0.18 Gt/km³ 0.33 Gt / 1.8 km³ = 0.183; 6,100 Gt / 33,000 km³ = 0.185 Gt/km³
Ratio of the 6.5 M capacity to the comparator same, para 2; Abstract "two orders of magnitude higher" (15 / 0.18 = 83×) 0.016 / 0.18 = 0.09, about 11× lower

Each later number follows arithmetically from the 15 Gt/km³ figure: 1.3 km³ × 15 Gt/km³ = 19.5 Gt ("up to 20 Gt"), and 1.3 km³ is 1/25,000 of the 33,000 km³ rift zone. The error therefore enters once, at the step that converts grams of CO2 to gigatonnes, and is carried through the rest of the paragraph.

Figure

Arithmetic chain from the paper's inputs to storage capacity, with the step where the reported values diverge highlighted

Figure 1. The top row gives the paper's stated inputs, multiplied in sequence for 1 km³ of rock: 5.7% effective porosity gives 5.7 × 10⁷ m³ of pore fluid, 0.65 M (650 mol/m³) gives 3.7 × 10¹⁰ mol of CO2, and 44.01 g/mol gives 1.6 × 10¹² g, or 1.63 Mt. The dashed box marks the conversion from grams to gigatonnes, where the reported values become about 920 times larger than the product of the inputs. The table below gives each downstream result twice: recomputed from the paper's inputs (blue marker, middle column) and as reported in the paper (orange marker, right column). All recomputed numbers come from recompute.py.txt.

Reported and recomputed capacities per km³ on a log scale, with physical ceilings and the cited comparator

Figure 2. Storage capacity in Gt CO2 per km³ of rock, on a log scale. For each scenario, the orange dot is the value reported in the paper and the blue dot is the value recomputed from the paper's own porosity and concentration; the labelled factor is the ratio of the two. The dotted lines are two ceilings, also computed in recompute.py.txt. Ceiling A (0.040 Gt/km³) is pure dense CO2 at an assumed 700 kg/m³ filling all of the 5.7% effective porosity. Ceiling B (0.29 Gt/km³) is the 6.5 M fluid filling 100% of the rock volume. Both reported values lie above both ceilings. The grey diamond is the 0.18 Gt/km³ comparator the paper cites, a mineral-carbonation estimate that the cited study derives as 0.33 Gt for a 1.8 km³ formation. The recomputed 6.5 M value lies below it, not two orders of magnitude above it.

Why it matters

The two capacities are the paper's only quantitative estimate of storage at reservoir scale, and they support the paper's scale-up claims: that 20 Gt of CO2 per year (the 2100 removal target the Introduction cites) could be stored in 1.3 km³, a small fraction of Iceland's rift zone, and, in the abstract, that the approach offers capacities "two orders higher than current techniques" and "reduces required reservoir volume". With the paper's own inputs, the reservoir volume needed for 20 Gt is about 1,230 km³ rather than 1.3 km³, roughly 1/27 of the rift zone rather than 1/25,000. The comparison with the cited value is not two orders of magnitude higher: in the same units it is about 11 times lower.

The abstract gives no numbers and does not say which comparison its "two orders higher" rests on. The only quantitative comparison in the body is the one recomputed here, so with the paper's inputs the abstract claim is not supported by the body.

The experimental results do not depend on this calculation. These are the measured changes in basalt cores after acidic GLDA flooding (16% gain in effective porosity, 26-fold gain in permeability), the CO2 absorption of GLDA seawater solutions, and the batch and flooding chemistry. The stoichiometric step from 0.65 M to 6.5 M after GLDA degradation is arithmetically consistent and is not in question here.

Explanations considered

A different meaning of "M". The paper uses M as mol/L, and 1 mol/L equals 1 kmol/m³, which is what the recomputation uses. The value 0.65 M for 20 wt % GLDA-Na4 (molar mass 351.1 g/mol, so 0.570 mol per kg of solution) implies a solution density of 1.14 kg/L, which is plausible. No reading of M makes the concentration 1,000 times larger.

A different porosity. To give 1.5 Gt/km³ at 0.65 M, or 15 Gt/km³ at 6.5 M, the pore fluid would have to be 52 times the volume of the rock (an effective porosity of about 5,200%). The paper's stated values are 4.8% and 5.7%. In its own experiments, the flooding core's effective porosity rose from 12.4 to 14.4%, and its basalt samples' total porosity is 14–20%. The cited study gives 10.99–15.23% total. None of these comes close. Even if the whole rock volume were 6.5 M fluid, the capacity would be 0.29 Gt/km³.

A megatonne/gigatonne slip. The correct results are 1.63 and 16.3 Mt/km³. Reporting them as 1.5 and 15 Gt/km³ fits a factor-of-1,000 unit slip combined with rounding or slightly different inputs (the remaining factor is 0.92). This is the most economical arithmetic explanation of how the numbers arose. It does not make the Gt figures correct, and every downstream statement in the paragraph inherits the factor.

A different basis (per km³ of pore space, or including mineralized CO2). The text says the calculation fills "all effective porosity" with the solution at the stated concentration. Even 1 km³ of pure pore fluid at 6.5 M holds only 0.29 Gt, not 15. Mineralization converts CO2 already in the fluid into carbonate and adds no CO2. Filling all 5.7% of the pore space with dense CO2 is capped at about 0.04 Gt/km³. The reported 15 Gt/km³ means 15,000 kg of CO2 per m³ of rock, about five times (5.1×) the density of basalt itself (taking a typical 2,950 kg/m³, an assumed value).

"Annually" meaning repeated injection. The paper gives no injection rate, pore-fluid turnover or number of cycles per year. The 20 Gt figure equals the static capacity times the volume (1.3 km³ × 15 Gt/km³ = 19.5 Gt), so "annually" adds no factor to the calculation.

The comparator is in Mt as well. If the cited 0.18 Gt/km³ were also inflated by a factor of 1,000, the paper's ratio would survive. The cited study's own tables rule this out. Its Table 7 gives a mean CO2 sequestration potential of 0.33 Gt for a CarbFix formation of 1.8 km³ (0.4 km × 2.5 km × 1.8 km), which is 0.18 Gt/km³. Its Table 8 gives 6,100 Gt for the 33,000 km³ rift zone, which is 0.185 Gt/km³. The paper takes its 33,000 km² × 1 km rift-zone dimensions from the same table. The value is computed from reactive surface area and a mineral-carbonation fixation rate per unit area, not from CO2 dissolved in pore fluid. It is in gigatonnes in the source, and it rests on a different basis from the paper's fluid-filling calculation. With consistent units, the paper's 0.016 Gt/km³ is about 11 times below it.

Conversion or archiving artifact. Ruled out. The publisher-typeset PDF (Science Advances, deposited in the PMC Open Access collection) carries the same numbers and units as the PMC text and the Europe PMC XML, with km³ correctly marked up. The quantities are not affected by formatting.

A derivation in the supplement. The only supplementary file (Tables S1–S2, Figs. S1–S11) contains rock compositions, the seawater recipe and experimental figures. It contains no capacity calculation, no Gt or km³ values, and no alternative units.

Status

As of 2026-10-23:

Sources: PMC11566992 (PubMed Central archive, read with src/show.py); https://doi.org/10.1126/sciadv.adq0515; publisher PDF and supplement from s3://pmc-oa-opendata/PMC11566992.1/; cited study, doi:10.1016/j.ijggc.2017.12.008 (accepted manuscript https://www.osti.gov/servlets/purl/1538337); https://api.crossref.org/works?filter=updates:10.1126/sciadv.adq0515.

Suggested correction

In the Results paragraph:

In the abstract, revise "capacities two orders higher than current techniques" and "reduces required reservoir volume" to match the corrected calculation, or name the comparison and basis that support them.

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC11566992 and regenerate fig1/fig2.

Run:  python recompute.py.txt   (writes fig1.svg/png and fig2.svg/png next to this file)

Inputs are taken from the article as archived in PubMed Central (PMC11566992),
Results subsection "CO2 mineralization with GLDA biodegradation and overall
evaluation", paragraphs 1-2:
  - effective porosity after GLDA treatment: 5.7 % (from 4.8 %)
  - CO2 concentration of the injected fluid: 0.65 M; 6.5 M after complete GLDA degradation
  - reported capacities: 1.5 and 15 Gt CO2 per km3; up to 20 Gt in 1.3 km3
  - active rift zone in Iceland: 33,000 km2 x 1 km; comparator 0.18 Gt CO2/km3
The comparator's own basis is taken from the cited study (ref. 17 of the paper,
Int. J. Greenh. Gas Control 70:146-156 (2018), doi:10.1016/j.ijggc.2017.12.008, accepted manuscript, Tables 7 and 8).
Three constants are not from either paper: the molar mass of CO2 (used in the
capacity calculation); for one physical ceiling, an assumed dense-CO2 density of
700 kg/m3; and, for one comparison only, a typical basalt density of 2,950 kg/m3
(both assumed values are labelled as such).
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (paper)
PHI0, PHI = 0.048, 0.057          # effective porosity before / after GLDA treatment
C1, C2 = 0.65, 6.5                # mol/L CO2 (GLDA-loaded; after full degradation)
REP1, REP2 = 1.5, 15.0            # reported Gt CO2 per km3
REP_ANNUAL, REP_VOL = 20.0, 1.3   # "Up to 20 Gt ... in a 1.3 km3 reservoir volume"
RIFT_KM3 = 33000 * 1              # 33,000 km2 x 1 km
RIFT_FRACTION_REP = 25000         # "1/25,000 of the active rift zone"
CITED = 0.18                      # Gt CO2 per km3, as quoted from the cited study
# Cited study (ref. 17), Table 7 (mean row) and Table 8, as printed in that study
CITED_SITE_GT, CITED_SITE_KM3 = 0.33, 1.8       # CarbFix site: 0.4 km x 2.5 km x 1.8 km
CITED_RIFT_GT = 6100                            # active rift zone, 33,000 km3

# ---------------------------------------------------------------- constants
M_CO2 = 44.01          # g/mol
KM3 = 1e9              # m3
G_PER_GT = 1e15        # g per Gt
RHO_DENSE_CO2 = 700    # kg/m3, assumed (generous) dense/supercritical CO2 density
RHO_BASALT = 2950      # kg/m3, assumed typical basalt grain/bulk density (~2,900-3,000)


def gt_per_km3(conc_M, phi):
    """CO2 (Gt) held in 1 km3 of rock whose effective pores are filled with fluid at conc_M mol/L."""
    mol = conc_M * 1000 * phi * KM3          # mol/L -> mol/m3, times pore volume
    return mol * M_CO2 / G_PER_GT


pore_m3 = PHI * KM3
mol1 = C1 * 1000 * pore_m3
g1 = mol1 * M_CO2
cap1, cap2 = gt_per_km3(C1, PHI), gt_per_km3(C2, PHI)
ratio1, ratio2 = REP1 / cap1, REP2 / cap2

print("== Capacity per km3 of rock, paper's own inputs ==")
print(f"pore fluid in 1 km3 at {PHI:.1%} effective porosity: {pore_m3:.3g} m3")
print(f"0.65 M: {mol1:.4g} mol CO2 = {g1:.4g} g = {g1/1e12:.3f} Mt = {cap1:.5f} Gt/km3; "
      f"reported {REP1} Gt/km3; reported/recomputed = {ratio1:.0f}x")
print(f"6.5 M:  {cap2*1e3:.2f} Mt = {cap2:.4f} Gt/km3; reported {REP2} Gt/km3; "
      f"reported/recomputed = {ratio2:.0f}x")
print(f"10x step from 0.65 M to 6.5 M (0.65 + 0.65 x 9): {0.65 + 0.65*9:.2f} M (paper's arithmetic is consistent)")
print()

print("== What the reported values would require ==")
for c, rep in [(C1, REP1), (C2, REP2)]:
    phi_needed = rep * G_PER_GT / (c * 1000 * M_CO2 * KM3)
    print(f"{rep} Gt/km3 at {c} M needs effective porosity {phi_needed:.1%}; "
          f"implied CO2 mass per m3 of rock = {rep*1e12/KM3:,.0f} kg/m3")
print(f"15 Gt/km3 = {REP2*1e12/KM3:,.0f} kg CO2 per m3 of rock = "
      f"{REP2*1e12/KM3/RHO_BASALT:.1f}x a basalt density of {RHO_BASALT:,} kg/m3 (assumed)")
print()

print("== Physical ceilings (per km3 of rock) ==")
ceil_100 = gt_per_km3(C2, 1.0)
ceil_dense = PHI * KM3 * RHO_DENSE_CO2 / 1e12
print(f"6.5 M fluid filling 100% of the rock volume: {ceil_100:.3f} Gt/km3")
print(f"pure dense CO2 ({RHO_DENSE_CO2} kg/m3, assumed) filling all 5.7% pores: {ceil_dense:.4f} Gt/km3")
print()

print("== 20 Gt in 1.3 km3 ==")
print(f"reported chain: 20 / 15 = {REP_ANNUAL/REP2:.2f} km3; 33,000 / 25,000 = {RIFT_KM3/RIFT_FRACTION_REP:.2f} km3; "
      f"1.3 x 15 = {REP_VOL*REP2:.1f} Gt")
held_13 = REP_VOL * cap2
vol_20 = REP_ANNUAL / cap2
print(f"CO2 held in 1.3 km3 at 6.5 M (recomputed): {held_13:.4f} Gt = {held_13*1e3:.1f} Mt")
print(f"volume needed for 20 Gt at the recomputed 6.5 M capacity: {vol_20:,.0f} km3 "
      f"= 1/{RIFT_KM3/vol_20:.0f} of the {RIFT_KM3:,} km3 rift zone")
print()

print("== Comparison with the cited value (0.18 Gt/km3) ==")
print(f"cited study, CarbFix site: {CITED_SITE_GT} Gt / {CITED_SITE_KM3} km3 = {CITED_SITE_GT/CITED_SITE_KM3:.3f} Gt/km3")
print(f"cited study, rift zone:    {CITED_RIFT_GT} Gt / {RIFT_KM3:,} km3 = {CITED_RIFT_GT/RIFT_KM3:.3f} Gt/km3")
print(f"reported 15 / 0.18 = {REP2/CITED:.0f}x  ('two orders of magnitude higher')")
print(f"recomputed {cap2:.4f} / 0.18 = {cap2/CITED:.3f}  (about {CITED/cap2:.0f}x lower)")
print()

print("== Side check: 0.65 M for 20 wt % GLDA-Na4 ==")
MW_GLDA_NA4 = 9*12.011 + 9*1.008 + 14.007 + 4*22.990 + 8*15.999   # C9H9NNa4O8
molal = 200 / MW_GLDA_NA4
print(f"MW C9H9NNa4O8 = {MW_GLDA_NA4:.1f} g/mol; 20 wt % = {molal:.3f} mol/kg solution; "
      f"0.65 M implies a solution density of {C1/molal:.2f} kg/L (plausible; M is mol/L)")

# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, HL = "#eb6834", "#2a78d6", "#e4e3df", "#fbe9e1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
                     "text.color": INK, "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)


def fmt_sci(x):
    e = int(f"{x:e}".split("e")[1])
    m = x / 10**e
    return f"{m:.2g}×10$^{{{e}}}$"


# ---------------------------------------------------------------- fig1: arithmetic chain
W, H = 7.2, 6.65
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")


def box(x, y, w, h, text, ec=INK2, fc="white", color=INK, weight="normal", lw=1.5, ls="-", size=10):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06",
                                fc=fc, ec=ec, lw=lw, ls=ls))
    ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=size, color=color,
            weight=weight, linespacing=1.35)


def arrow(x0, y0, x1, y1, color=INK2):
    ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12,
                                 color=color, lw=1.5, shrinkA=0, shrinkB=0))


ax.text(0.15, H - 0.18, "Capacity of 1 km³ of rock, using only the paper's inputs",
        ha="left", va="top", fontsize=11, weight="bold")
ax.text(0.15, H - 0.45, "Shared steps (the paper's stated inputs)", ha="left", va="top",
        fontsize=10, color=INK2)

bw, bh, gap, y0 = 1.55, 0.95, 0.22, H - 1.65
xs = [0.15 + i*(bw + gap) for i in range(4)]
steps = ["Rock volume\n1 km³ = 10$^{9}$ m³",
         f"× 5.7% effective\nporosity\n= {fmt_sci(pore_m3)} m³ fluid",
         f"× 0.65 M\n(650 mol/m³)\n= {fmt_sci(mol1)} mol",
         f"× 44.01 g/mol\n= {fmt_sci(g1)} g CO$_2$\n= {g1/1e12:.2f} Mt"]
for x, s in zip(xs, steps):
    box(x, y0, bw, bh, s)
for i in range(3):
    arrow(xs[i] + bw, y0 + bh/2, xs[i+1], y0 + bh/2)

# divergence step
dx, dy, dw, dh = xs[2] - 0.9, y0 - 1.05, W - 0.15 - (xs[2] - 0.9), 0.62
ax.add_patch(FancyBboxPatch((dx, dy), dw, dh, boxstyle="round,pad=0.02,rounding_size=0.06",
                            fc=HL, ec=ORANGE, lw=2, ls="--"))
ax.text(dx + dw/2, dy + dh/2 + 0.11, "Unit step: grams → gigatonnes (÷10$^{15}$)",
        ha="center", va="center", fontsize=10, weight="bold")
ax.text(dx + dw/2, dy + dh/2 - 0.15, f"the reported values diverge here by ≈{ratio1:.0f}×",
        ha="center", va="center", fontsize=10, color=INK2)
arrow(xs[3] + bw/2, y0, xs[3] + bw/2, dy + dh)

# outcome table
ty = dy - 0.42
cx_lab, cx_b, cx_o = 0.15, 3.75, 5.95
ax.text(cx_lab, ty, "Result", ha="left", va="center", fontsize=10, color=INK2, weight="bold")
ax.plot([cx_b - 1.05], [ty], "o", ms=8, color=BLUE)
ax.plot([cx_o - 0.85], [ty], "o", ms=8, color=ORANGE)
ax.text(cx_b, ty, "Recomputed from\nthe paper's inputs", ha="center", va="center", fontsize=10,
        color=BLUE, weight="bold")
ax.text(cx_o, ty, "Reported in\nthe paper", ha="center", va="center", fontsize=10,
        color=ORANGE, weight="bold")
rows = [("Capacity per km³, 0.65 M", f"{cap1:.4f} Gt ({cap1*1e3:.1f} Mt)", "1.5 Gt"),
        ("Capacity per km³, 6.5 M\n(after GLDA degradation, ×10)", f"{cap2:.3f} Gt ({cap2*1e3:.0f} Mt)", "15 Gt"),
        ("CO$_2$ held in 1.3 km³ at 6.5 M", f"{held_13:.3f} Gt ({held_13*1e3:.0f} Mt)", "up to 20 Gt"),
        ("Rock volume needed for 20 Gt", f"{vol_20:,.0f} km³\n(1/{RIFT_KM3/vol_20:.0f} of the rift zone)",
         "1.3 km³\n(1/25,000 of the rift zone)"),
        ("Ratio to the cited\n0.18 Gt/km³ (ref. 17)", f"{cap2/CITED:.2f}×\n(about {CITED/cap2:.0f}× lower)",
         "“two orders of\nmagnitude higher”")]
rh = 0.62
for i, (lab, b, o) in enumerate(rows):
    yy = ty - 0.55 - i*rh
    ax.plot([0.15, W - 0.15], [yy + rh/2, yy + rh/2], color=GRID, lw=1)
    ax.text(cx_lab, yy, lab, ha="left", va="center", fontsize=10, linespacing=1.2)
    ax.text(cx_b, yy, b, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
    ax.text(cx_o, yy, o, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
save(fig, "fig1")

# ---------------------------------------------------------------- fig2: log-scale comparison
fig, ax = plt.subplots(figsize=(7.2, 3.7))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
rows2 = [("0.65 M fluid in\n5.7% porosity", REP1, cap1),
         ("6.5 M fluid in 5.7% porosity\n(after GLDA degradation)", REP2, cap2)]
yp = [2, 1]
for y, (lab, rep, rec) in zip(yp, rows2):
    ax.plot([rec, rep], [y, y], color=INK2, lw=1, zorder=1)
    ax.plot(rep, y, "o", ms=9, color=ORANGE, zorder=3)
    ax.plot(rec, y, "o", ms=9, color=BLUE, zorder=3)
    gm = 0.11 if y == 2 else 1.6
    ax.text(gm, y + 0.13, f"×{rep/rec:.0f}", ha="center", va="bottom", fontsize=10, color=INK2)
    ax.text(rep * 1.25, y, f"{rep:g}", ha="left", va="center", fontsize=10, color=INK)
    ax.text(rec / 1.25, y, f"{rec:.4f}" if rec < 0.01 else f"{rec:.3f}", ha="right", va="center",
            fontsize=10, color=INK)
ax.plot(CITED, 0, "D", ms=8, color=INK2, zorder=3)
ax.text(CITED * 1.3, 0, "0.18 (cited mineral-carbonation\nestimate: 0.33 Gt / 1.8 km³)",
        ha="left", va="center", fontsize=10, color=INK)
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2] + ["Comparator cited in\nthe paper (ref. 17)"])
ax.set_ylim(-0.6, 2.6)
ax.set_xscale("log")
ax.set_xlim(2e-4, 2e2)
for xv, lab, ha in [(ceil_dense, f"A: {ceil_dense:.3f} ", "right"), (ceil_100, f" B: {ceil_100:.2f}", "left")]:
    ax.axvline(xv, color=INK2, lw=1, ls=":", zorder=0)
    ax.text(xv, 2.62, lab, ha=ha, va="bottom", fontsize=10, color=INK2)
ax.set_xlabel("Gt CO$_2$ per km³ of rock (log scale)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelsize=10)
ax.tick_params(axis="x", labelsize=10, colors=INK2)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in the paper"),
           Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from the paper's inputs"),
           Line2D([], [], ls=":", color=INK2, lw=1, label="Ceilings (see caption)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=3, frameon=False,
           fontsize=10, handletextpad=0.4, columnspacing=1.2)
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))

The abstract gives the top tolerated monkey dose in mg/kg; the body gives μg/kg

Sci Adv (2021) · doi:10.1126/sciadv.abi8193 · PMC8357232

Summary

The abstract states that the T cell–engaging bispecific CCW702 "was well tolerated up to 34.1 mg/kg per dose" in cynomolgus monkeys. The Results, the Methods and the Fig. 6 legend all describe a single repeat-dose monkey study with doses of 2, 9.8 and 34.1 μg/kg, and they put the no-observed-adverse-effect level (NOAEL) at 34.1 μg/kg. Read literally, the abstract's 34.1 mg/kg is 34,100 μg/kg, 1,000 times the highest dose tested. The paper's own exposure value for that group (day-1 AUC 1.02 h·μg/mL) gives an apparent clearance of 33 mL/h/kg for a μg/kg dose. For a mg/kg dose it gives about 33,400 mL/h/kg, which is 2.6 times total cardiac output and physically impossible. This is the abstract's only primate safety number. Read as printed, it puts the tolerated monkey dose level with the highest mouse efficacy dose (35.2 mg/kg), when the tested dose was 33 to 1,000 times lower than the mouse efficacy doses.

What the paper reports

Abstract:

In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose, with near-complete subcutaneous bioavailability and a PK profile supporting testing of a weekly dosing regimen in patients.

Results, "Pharmacokinetics and pharmacodynamics in nonhuman primate", second paragraph:

Doses of 2, 9.8, and 34.1 μg/kg were administered to animals QAD for 10 days (five total doses).

CCW702 administration was generally well tolerated at these dose levels with no changes in animal body weight, food consumption, or body temperature and the no-observed-adverse-effect level (NOAEL) determined to be the highest dose in the study of 34.1 μg/kg.

This occurred despite similar exposure of CCW702 on day 1 and on day 9 (1.02 hours·μg/ml versus 0.705 hours·μg/ml for the 34.1 μg/kg group) (fig. S17).

Same section, first paragraph (single-dose PK studies):

CCW702 exposure was detectable by electrochemiluminescence bioanalytical assay at doses of ≥9.8 μg/kg.

Subcutaneous dosing had near complete bioavailability (F = 108 ± 20.2%) with a maximum concentration (Cmax) and half-life (t1/2) of 1.5 (ng/ml)/(mg/kg) and 11.4 hours compared with the intravenous route of administration Cmax of 28.3 (ng/ml)/(mg/kg) and t1/2 of 1.6 hours, respectively (fig. S17).

Fig. 6 legend, panel A (panel B gives the same doses):

Heatmaps of cytokines measured from peripheral blood of animals administrated CCW702 at 2, 9.8, and 34.1 μg/kg, every other day for 10 days (a total of five total doses).

Methods, "Pharmacokinetic and pharmacodynamic 10-day study in cyno":

Nine naïve male cynomolgus monkeys (Macaca fascicularis, Chinese origin), age 2 to 4 years, weighing 2.5 to 3.2 kg were used for the study.

The NOAEL was determined to be 34.1 μg/kg under the study conditions.

For context, Results, "In vivo efficacy in xenograft and patient-derived xenograft models":

every 7 days (Q7D) at four-dose levels (1.13, 3.5, 11.1, and 35.2 mg/kg; groups G to J)

What the paper's own numbers show

Method. I converted the abstract's figure to μg/kg and compared it with the doses tested in the body. To find which unit the data support, I divided the top dose by the paper's day-1 AUC for that group. That gives the apparent clearance CL/F, and since F ≈ 1 (108%) it is close to systemic clearance. I compared CL/F under each reading with standard physiological values for the monkey (Davies & Morris, Pharm Res 1993;10:1093–5): cardiac output 217 mL/min/kg (13,020 mL/h/kg), hepatic blood flow 43.6 mL/min/kg (2,616 mL/h/kg) and plasma volume 44.8 mL/kg. Plasma clearance cannot exceed the blood flow that carries the drug to the organs that eliminate it, so it cannot exceed cardiac output. The AUC interval is given in fig. S17, which I could not access. A partial AUC would make CL/F an overestimate, but by much less than the factor of 1,000 at issue here. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Highest tolerated monkey dose Abstract 34.1 mg/kg per dose = 34,100 μg/kg
Monkey repeat-dose levels Results NHP para 2; Fig. 6 legend 2, 9.8, 34.1 μg/kg, QAD × 5 highest = 34.1 μg/kg = 0.0341 mg/kg
NOAEL Results NHP para 2; Methods 34.1 μg/kg equals the top dose tested
Abstract value / top dose tested derived (implied equal) 1,000
Cumulative dose, top group derived — 5 × 34.1 = 170.5 μg/kg = 0.171 mg/kg
Amount per dose, 2.5–3.2 kg animal Methods (weights) — 0.085–0.109 mg if μg/kg; 85–109 mg if mg/kg
CL/F if top dose = 34.1 μg/kg Results NHP para 2 (AUC 1.02 h·μg/mL) — 33.4 mL/h/kg, 0.26% of cardiac output (day 9: 48.4 mL/h/kg)
CL/F if top dose = 34.1 mg/kg same — 33,431 mL/h/kg, 2.6 × total cardiac output
SC Cmax and AUC at top dose Results NHP para 1 (1.5 per unit dose, t1/2 11.4 h) AUC 1.02 h·μg/mL Cmax 51.2 ng/mL; crude AUC ≈ Cmax/k = 841 ng·h/mL, close to the reported 1,020. This check holds under either reading, so it does not settle the unit on its own
Mouse Q7D efficacy doses / top monkey dose Results (mouse efficacy) 1.13–35.2 mg/kg 33× to 1,032× the top monkey dose tested
Abstract value / top mouse dose derived — 34.1 / 35.2 = 0.97

Figure

The abstract gives the top monkey dose as 34.1 mg/kg; the Results, Methods and Fig. 6 give 34.1 μg/kg.

Figure 1. The two statements of the top cynomolgus dose. The left panel (orange, the value as printed in the abstract) quotes the abstract's "34.1 mg/kg per dose", which equals 34,100 μg/kg. The right panel (blue) shows the doses used in the Results, Methods and Fig. 6 legend, with the NOAEL at 34.1 μg/kg. The connector marks the factor of 1,000 between the two. The bottom line of the right panel gives the apparent clearance recomputed from the paper's own day-1 AUC, which is plausible only for the μg/kg reading. All values come from the paper or from recompute.py.txt.

Apparent clearance from the paper's day-1 AUC under each unit reading.

Figure 2. Where the factor of 1,000 enters. The dose label "34.1" for the top group is read in the abstract's unit (orange, mg/kg) and in the body's unit (blue, μg/kg). Each reading is divided by the paper's own day-1 AUC for that group (1,020 ng·h/mL). The mg/kg reading gives an apparent clearance of 33,431 mL/h/kg, 2.6 times the total cardiac output of a monkey, which is not possible. The μg/kg reading gives 33.4 mL/h/kg, which is plausible. The cardiac output value is an external reference (Davies & Morris 1993); all other numbers come from the paper or from recompute.py.txt.

Why it matters

The abstract sentence is the paper's only summary of primate safety. It is also the evidence the abstract offers just before it says the drug is in a first-in-human trial (NCT04077021). The data in the body are reported consistently, and the qualitative finding holds: CCW702 was well tolerated at every dose tested, and the NOAEL was the top dose. The error is in magnitude. Read as printed, the abstract suggests that monkeys tolerated about 34 mg/kg per dose, roughly the highest mouse efficacy dose (35.2 mg/kg Q7D), which implies a wide tolerability margin over the efficacious range. In fact the highest dose tested in monkeys was 0.0341 mg/kg per dose, 33 to 1,000 times below the mouse Q7D efficacy doses. Mouse and monkey doses in mg/kg are not directly interchangeable across species, but that is the comparison a reader of the abstract would make. No dose above 34.1 μg/kg was tested in monkeys, so the study established a NOAEL but not a maximum tolerated dose. The Methods also report clinical pathology changes and microscopic findings, judged not adverse, at doses above 2 μg/kg. A reader who relies on the abstract, or a database or review that cites it, will carry a tolerated dose that is 1,000 times too high.

Explanations considered

Status

As of 2026-10-23. Version of record: the publisher's site (science.org, the target of the DOI redirect) returned HTTP 403, so I could not read its HTML or PDF directly. Four publisher-derived sources carry the same text: the live PMC page, the Europe PMC full-text XML (publisher JATS), the PubMed record (PMID 34380625), and the publisher's Fig. 6 image file, whose panel labels read "2 μg/kg/dose", "9.8 μg/kg/dose" and "34.1 μg/kg/dose". The quotations above were checked verbatim against the PMC-archived text. Supplement: the supplementary PDF (abi8193_sm.pdf, figs. S1 to S18, including fig. S17 with the monkey PK) is hosted only on science.org, which returned 403. Europe PMC's supplementary bundle holds only the main figures, so I could not check figs. S17 and S18. Corrections: I found none. Crossref has no update-to or updated-by relation, and the updates: filter returns 0 results. Crossref Labs (which includes Retraction Watch data) lists no updates. Europe PMC and PubMed show no errata or comment/correction links, and two web searches for an erratum or correction found none. PubPeer: the search page and the API returned 403 or not found, so I could not check PubPeer directly. A general web search found no PubPeer thread.

Suggested correction

In the abstract, replace "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose" with "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 μg/kg per dose (the highest dose tested)". The authors may also want to check the unit labels of the dose-normalized Cmax values in the first NHP Results paragraph and in fig. S17 ("(ng/ml)/(mg/kg)"), which the paper's own values suggest should be per μg/kg.

Recomputation script (python recompute.py.txt)
"""Recompute the cynomolgus dose-unit check for PMC8357232
(Sci Adv 2021, doi:10.1126/sciadv.abi8193).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file, five candidate figure designs (candidate1..5.svg/.png)
plus fig1 (= candidate 1) and fig2 (= candidate 2).

Inputs are copied from the article as archived in PubMed Central:
  Abstract: "well tolerated up to 34.1 mg/kg per dose"
  Results (NHP para 2), Methods (10-day cyno study), Fig. 6 legend: doses 2, 9.8, 34.1 ug/kg; NOAEL 34.1 ug/kg
  Results (NHP para 2): day-1 exposure 1.02 h*ug/mL, day-9 0.705 h*ug/mL, 34.1 ug/kg group
  Results (NHP para 1): SC F = 108%, SC Cmax 1.5 and IV Cmax 28.3 "(ng/ml)/(mg/kg)", SC t1/2 11.4 h
  Methods: monkeys weighed 2.5 to 3.2 kg
  Results (mouse efficacy): Q7D doses 1.13, 3.5, 11.1, 35.2 mg/kg
External reference values (not from the paper), cynomolgus/rhesus monkey, Davies & Morris,
Pharm Res 1993;10:1093-5: plasma volume 44.8 mL/kg, cardiac output 217 mL/min/kg,
hepatic blood flow 43.6 mL/min/kg.
"""
import math
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (paper)
ABSTRACT_TOP_MG_KG = 34.1
CYNO_DOSES_UG_KG = [2.0, 9.8, 34.1]
NOAEL_UG_KG = 34.1
N_DOSES = 5
AUC_DAY1_H_UG_ML = 1.02
AUC_DAY9_H_UG_ML = 0.705
F_SC = 1.08
CMAX_SC_PER_UNIT = 1.5      # printed as (ng/ml)/(mg/kg)
CMAX_IV_PER_UNIT = 28.3     # printed as (ng/ml)/(mg/kg)
T_HALF_SC_H = 11.4
WEIGHT_KG = (2.5, 3.2)
MOUSE_Q7D_MG_KG = [1.13, 3.5, 11.1, 35.2]
# ------------------------------------------------------------------ external reference
PLASMA_VOL_ML_KG = 44.8
CARDIAC_OUTPUT_ML_MIN_KG = 217.0
HEPATIC_FLOW_ML_MIN_KG = 43.6

ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
                     "savefig.facecolor": BG, "svg.fonttype": "none"})


def section(t):
    print("\n== " + t)


# ------------------------------------------------------------------ 1. unit conversion
section("1. Abstract dose vs. dose tested")
abstract_ug = ABSTRACT_TOP_MG_KG * 1000
top_tested = max(CYNO_DOSES_UG_KG)
ratio = abstract_ug / top_tested
print(f"abstract 34.1 mg/kg = {abstract_ug:,.0f} ug/kg")
print(f"highest dose tested (Results/Methods/Fig. 6) = {top_tested} ug/kg = {top_tested/1000:.4f} mg/kg; NOAEL = {NOAEL_UG_KG} ug/kg")
print(f"ratio abstract / tested = {ratio:,.0f}")
print(f"cumulative over {N_DOSES} doses at top dose = {N_DOSES*top_tested:.1f} ug/kg = {N_DOSES*top_tested/1000:.3f} mg/kg")
for w in WEIGHT_KG:
    print(f"  per-dose amount for a {w} kg monkey: {top_tested*w/1000:.3f} mg if ug/kg, {ABSTRACT_TOP_MG_KG*w:.1f} mg if mg/kg")

# ------------------------------------------------------------------ 2. clearance check
section("2. Apparent clearance from the paper's own day-1 AUC (34.1 group)")
auc_ng_h_ml = AUC_DAY1_H_UG_ML * 1000
cl = {}
for label, dose_ng_kg in [("34.1 ug/kg", 34.1e3), ("34.1 mg/kg", 34.1e6)]:
    cl[label] = dose_ng_kg / auc_ng_h_ml           # mL/h/kg = CL/F; F ~ 1 (108%)
    print(f"CL/F if dose = {label}: {cl[label]:,.1f} mL/h/kg = {cl[label]/1000:,.2f} L/h/kg")
co_ml_h_kg = CARDIAC_OUTPUT_ML_MIN_KG * 60
hep_ml_h_kg = HEPATIC_FLOW_ML_MIN_KG * 60
print(f"reference: cardiac output {co_ml_h_kg:,.0f} mL/h/kg, hepatic blood flow {hep_ml_h_kg:,.0f} mL/h/kg")
print(f"mg/kg reading: CL/F = {cl['34.1 mg/kg']/co_ml_h_kg:.1f} x total cardiac output")
print(f"ug/kg reading: CL/F = {cl['34.1 ug/kg']/co_ml_h_kg*100:.2f}% of cardiac output")
print(f"day-9 CL/F (ug/kg reading): {34.1e3/(AUC_DAY9_H_UG_ML*1000):.1f} mL/h/kg")

# ------------------------------------------------------------------ 3. side note: Cmax labels
section("3. Side note: the '(ng/ml)/(mg/kg)' Cmax labels")
c0_per_ug = 1000 / PLASMA_VOL_ML_KG   # ng/mL per ug/kg if dose stays in plasma volume
print(f"IV dose of 1 ug/kg in {PLASMA_VOL_ML_KG} mL/kg plasma -> C0 about {c0_per_ug:.1f} ng/mL "
      f"(1 mg/kg -> {c0_per_ug*1000:,.0f} ng/mL); reported IV Cmax {CMAX_IV_PER_UNIT} per unit dose")
print(f"implied initial volume if per mg/kg: {1e6/CMAX_IV_PER_UNIT/1000:,.1f} L/kg; if per ug/kg: {1e3/CMAX_IV_PER_UNIT:.1f} mL/kg")
cmax_sc = CMAX_SC_PER_UNIT * 34.1
k = math.log(2) / T_HALF_SC_H
print(f"SC Cmax at top dose = {cmax_sc:.1f} ng/mL; crude AUC ~ Cmax/k = {cmax_sc/k:.0f} ng*h/mL vs reported {auc_ng_h_ml:.0f}")
print("  (this pairing is consistent under either reading, so it does not by itself fix the unit)")

# ------------------------------------------------------------------ 4. context
section("4. Context: mouse Q7D efficacy doses vs. highest cyno dose tested")
for d in MOUSE_Q7D_MG_KG:
    print(f"  mouse {d} mg/kg = {d*1000:,.0f} ug/kg = {d*1000/top_tested:,.0f} x the top cyno dose")
print(f"abstract figure 34.1 mg/kg vs top mouse dose 35.2 mg/kg: ratio {ABSTRACT_TOP_MG_KG/35.2:.2f}")


# ================================================================== figures
def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)


def box(ax, x, y, w, h, edge, lw=2.0, fill=BG):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                lw=lw, ec=edge, fc=fill, transform=ax.transData, clip_on=False))


def cand1():
    """Two text panels: Abstract vs body, joined by an annotated connector."""
    fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
    ax.set_xlim(-0.01, 1.01); ax.set_ylim(0, 1); ax.set_position([0.01, 0.01, 0.98, 0.98])
    box(ax, 0.0, 0.04, 0.38, 0.90, ORANGE)
    ax.text(0.02, 0.89, "Abstract (as printed)", fontsize=11.5, weight="bold", va="top")
    ax.text(0.02, 0.77, "\u201cIn cynomolgus monkeys,\nCCW702 was well\ntolerated up to", va="top", fontsize=10.5, linespacing=1.35)
    ax.text(0.02, 0.48, "34.1 mg/kg", va="top", fontsize=15, weight="bold")
    ax.text(0.02, 0.36, "per dose\u201d", va="top", fontsize=10.5)
    ax.text(0.02, 0.08, "34.1 mg/kg = 34,100 \u03bcg/kg", va="bottom", fontsize=10, color=INK2)
    box(ax, 0.58, 0.04, 0.42, 0.90, BLUE)
    ax.text(0.60, 0.89, "Results, Methods, Fig. 6", fontsize=11.5, weight="bold", va="top")
    ax.text(0.60, 0.77, "Doses tested: 2, 9.8, 34.1 \u03bcg/kg\n(every other day, five doses)", va="top", fontsize=10, linespacing=1.35)
    ax.text(0.60, 0.59, "\u201c\u2026 NOAEL \u2026 the highest dose\nin the study of", va="top", fontsize=10, linespacing=1.35)
    ax.text(0.60, 0.43, "34.1 \u03bcg/kg\u201d", va="top", fontsize=15, weight="bold")
    ax.text(0.60, 0.08, "Day-1 AUC 1.02 h\u00b7\u03bcg/mL gives\nCL/F 33 mL/h/kg only for \u03bcg/kg", va="bottom", fontsize=9.8, color=INK2)
    ax.add_patch(FancyArrowPatch((0.385, 0.42), (0.575, 0.42), arrowstyle="<->", mutation_scale=14,
                                 lw=2, color=INK2, transform=ax.transData))
    ax.text(0.48, 0.47, "same dose,\nunit differs", ha="center", va="bottom", fontsize=10.5)
    ax.text(0.48, 0.37, f"\u00d7{ratio:,.0f}", ha="center", va="top", fontsize=15, weight="bold")
    save(fig, "candidate1"); save(fig, "fig1"); plt.close(fig)


def cand2():
    """Arithmetic flow: dose / AUC -> clearance; divergence at the unit step."""
    fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
    ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    box(ax, 0.00, 0.38, 0.18, 0.24, INK2)
    ax.text(0.09, 0.50, "Dose label\n\u201c34.1\u201d\n(top group)", ha="center", va="center", fontsize=10.5)
    rows = [(0.68, ORANGE, "Abstract: mg/kg", "34,100,000 ng/kg", f"{cl['34.1 mg/kg']:,.0f} mL/h/kg",
             f"{cl['34.1 mg/kg']/co_ml_h_kg:.1f}\u00d7 total cardiac\noutput: not possible"),
            (0.08, BLUE, "Body: \u03bcg/kg", "34,100 ng/kg", f"{cl['34.1 ug/kg']:,.1f} mL/h/kg",
             f"{cl['34.1 ug/kg']/co_ml_h_kg*100:.1f}% of cardiac\noutput: plausible")]
    for y, col, unit, ng, clv, verdict in rows:
        box(ax, 0.235, y, 0.25, 0.24, col, lw=3)
        ax.text(0.36, y + 0.12, f"{unit}\n= {ng}", ha="center", va="center", fontsize=9.8, linespacing=1.15)
        box(ax, 0.53, y, 0.18, 0.24, INK2)
        ax.text(0.62, y + 0.12, "\u00f7 day-1 AUC\n1,020\nng\u00b7h/mL", ha="center", va="center", fontsize=9.8)
        ax.text(0.74, y + 0.17, "CL/F = " + clv, fontsize=10.5, weight="bold", va="center")
        ax.text(0.74, y + 0.07, verdict, fontsize=9.8, color=INK2, va="center", linespacing=1.2)
        ax.add_patch(FancyArrowPatch((0.485, y + 0.12), (0.53, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=INK2, transform=ax.transData))
        ax.add_patch(FancyArrowPatch((0.18, 0.50), (0.235, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=col, transform=ax.transData))
    ax.text(0.36, 0.50, "the factor of 1,000\nenters at this step", ha="center", va="center", fontsize=10.5, weight="bold")
    ax.text(0.0, -0.04, "AUC: Results, 34.1 group, day 1. F \u2248 1 (108%), so CL/F \u2248 CL. "
            "Cardiac output 13,020 mL/h/kg (monkey reference value).", fontsize=9.5, color=INK2, va="top")
    save(fig, "candidate2"); save(fig, "fig2"); plt.close(fig)


def cand3():
    """Dose ladder on one log axis (ug/kg)."""
    fig, ax = plt.subplots(figsize=(7.2, 3.0))
    rows = ["Mouse efficacy, Q7D\n(Results)", "Cyno, as stated\nin Abstract", "Cyno, doses tested\n(Results, Methods, Fig. 6)"]
    ax.scatter([d * 1000 for d in MOUSE_Q7D_MG_KG], [0] * 4, s=70, marker="s", facecolor="white", edgecolor=INK2, lw=2, zorder=3, label="Mouse doses (context)")
    ax.scatter([abstract_ug], [1], s=90, color=ORANGE, zorder=3, label="Abstract: 34.1 mg/kg")
    ax.scatter(CYNO_DOSES_UG_KG, [2] * 3, s=90, color=BLUE, zorder=3, label="Body: 2, 9.8, 34.1 μg/kg")
    ax.annotate("", xy=(abstract_ug, 1.5), xytext=(34.1, 1.5), arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
    ax.text(math.sqrt(abstract_ug * 34.1), 1.62, "×1,000", ha="center", va="bottom", fontsize=11, weight="bold")
    ax.text(abstract_ug, 0.75, "34,100", ha="center", va="top", fontsize=10)
    for d in CYNO_DOSES_UG_KG:
        ax.text(d, 2.25, f"{d:g}", ha="center", va="bottom", fontsize=10)
    for d in MOUSE_Q7D_MG_KG:
        ax.text(d * 1000, -0.25, f"{d*1000:,.0f}", ha="center", va="top", fontsize=10)
    ax.set_xscale("log"); ax.set_xlim(1, 1e5); ax.set_ylim(-0.7, 2.7)
    ax.set_yticks([0, 1, 2]); ax.set_yticklabels(rows); ax.tick_params(axis="y", length=0, labelcolor=INK)
    ax.set_xlabel("Dose per administration (μg/kg, log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
    ax.legend(loc="upper left", frameon=False, fontsize=9.5, bbox_to_anchor=(1.0, 1.0))
    save(fig, "candidate3"); plt.close(fig)


def cand4():
    """Implied clearance under each reading vs physiological ceilings, one log axis."""
    fig, ax = plt.subplots(figsize=(7.2, 2.6))
    ax.axvline(hep_ml_h_kg, color=INK2, lw=1, ls=":")
    ax.axvline(co_ml_h_kg, color=INK2, lw=1, ls="--")
    ax.text(hep_ml_h_kg, 1.55, f"hepatic blood flow\n{hep_ml_h_kg:,.0f}", ha="right", va="top", fontsize=9.5, color=INK2)
    ax.text(co_ml_h_kg * 1.1, 1.55, f"total cardiac\noutput {co_ml_h_kg:,.0f}", ha="left", va="top", fontsize=9.5, color=INK2)
    ax.scatter([cl["34.1 mg/kg"]], [1], s=90, color=ORANGE, zorder=3)
    ax.scatter([cl["34.1 ug/kg"]], [0], s=90, color=BLUE, zorder=3)
    ax.text(cl["34.1 mg/kg"] * 1.25, 1, f"{cl['34.1 mg/kg']:,.0f}", ha="left", va="center", fontsize=10.5, weight="bold")
    ax.text(cl["34.1 ug/kg"] * 1.25, 0, f"{cl['34.1 ug/kg']:.1f}", ha="left", va="center", fontsize=10.5, weight="bold")
    ax.set_yticks([0, 1]); ax.set_yticklabels(["Dose read as 34.1 μg/kg\n(Results, Methods)", "Dose read as 34.1 mg/kg\n(Abstract)"])
    ax.tick_params(axis="y", length=0, labelcolor=INK)
    ax.set_xscale("log"); ax.set_xlim(1, 1e6); ax.set_ylim(-0.6, 1.7)
    ax.set_xlabel("Apparent clearance CL/F = dose / day-1 AUC (mL/h/kg, log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
    save(fig, "candidate4"); plt.close(fig)


def cand5():
    """Where the dose appears, and in which unit: a location-by-unit dot table."""
    locs = ["Abstract", "Results, NHP para 1 (detectable ≥ 9.8)", "Results, NHP para 2 (dose groups)",
            "Results, NHP para 2 (NOAEL)", "Results, NHP para 2 (cytokines, AUC)",
            "Fig. 6 legend, panels A and B", "Methods, 10-day cyno study (NOAEL)"]
    unit = ["mg", "ug", "ug", "ug", "ug", "ug", "ug"]
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    for i, (l, u) in enumerate(zip(locs, unit)):
        y = len(locs) - 1 - i
        x = 0 if u == "mg" else 1
        ax.scatter([x], [y], s=110, color=ORANGE if u == "mg" else BLUE, zorder=3)
        ax.scatter([1 - x], [y], s=60, facecolor="none", edgecolor=GRID, lw=1.5, zorder=2)
    ax.set_yticks(range(len(locs))); ax.set_yticklabels(locs[::-1]); ax.tick_params(axis="y", length=0, labelcolor=INK)
    ax.set_xticks([0, 1]); ax.set_xticklabels(["mg/kg", "μg/kg"], fontsize=11, color=INK)
    ax.xaxis.tick_top(); ax.tick_params(axis="x", length=0)
    ax.set_xlim(-0.6, 2.6); ax.set_ylim(-0.6, len(locs) - 0.4)
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ax.spines.values(): s.set_visible(False)
    ax.text(1.35, len(locs) - 1, "34.1 mg/kg (as printed)\n= 34,100 μg/kg", va="center", fontsize=10, color=INK2)
    ax.text(1.35, 2.5, "All six body locations:\n2, 9.8, 34.1 μg/kg\n(filled dot = unit used;\nopen = not used)", va="center", fontsize=10, color=INK2)
    save(fig, "candidate5"); plt.close(fig)


for f in (cand1, cand2, cand3, cand4, cand5):
    f()
print("\nfigures written to", HERE)

The abstract's anxiety odds ratio (5.80) lies outside its own 95% CI (1.19 to 4.22)

Ann Rheum Dis (2017) · doi:10.1136/annrheumdis-2017-212000 · PMC5867422

Summary

The abstract reports that high fear scores on the FAIR questionnaire were associated with high HADS anxiety, "OR 5.80, 95% CI (1.19 to 4.22)". The point estimate is above the upper limit of its own interval, which a Wald interval cannot produce. The geometric centre of the printed interval is 2.24, while the abstract's other four odds ratios sit at the centres of their intervals to within rounding. Figure 3, the paper's forest plot, splits HADS anxiety into two categories. Read by pixel, its 8–10 vs <8 row has an interval of about 1.20 to 4.18, which matches the abstract's interval, and its >10 vs <8 row has an OR of about 5.97, which matches the abstract's 5.80. The abstract therefore appears to pair the estimate of one category with the interval of the other, without naming either. Under either pairing the association remains positive and its interval excludes 1, but the size of the effect given in the abstract is not a valid estimate-interval pair.

What the paper reports

Abstract, Results (identical in the PMC archive and in the publisher's typeset PDF):

High fear scores were associated with high Arthritis Helplessness Index scores (OR 6.85, 95% CI (3.95 to 11.87)); high Hospital Anxiety and Depression Scale anxiety (OR 5.80, 95% CI (1.19 to 4.22)) and depression (OR 2.37, 95% CI (1.29 to 4.37)) scores; low education level (OR 3.48, 95% CI (1.37 to 8.83)); and high perceived disease activity (OR 2.36, 95% CI (1.10 to 5.04)).

Methods, statistical analysis:

Univariate, then multivariate logistic regression was used to identify patient variables (including demographic, social and economic characteristics; disease status, and anxiety/depression and helplessness levels) independently associated with the highest compared with the lowest fear score cluster.

Results, "Factors associated with fear scores" (the text gives no numbers and refers to Figure 3):

Cluster 1 (high fear scores) was associated with higher global rating of disease activity by the patient, high AHI helplessness scores and high HADS anxiety and depression scores.

Figure 3 caption:

High scores indicate those >75 (shown in red); moderate scores, 50–75 (shown in blue); and low scores, <50. Data are presented as ORs (95% CIs).

Figure 3 itself is an image. Its HADS anxiety block has two rows, labeled "8-10 vs <8" and ">10 vs <8". It shows markers on a log axis from 0.1 to 10 and prints no numbers.

Discussion, second and third paragraphs:

In contrast, a strong association was observed between FAIR scores and scores on the AHI (≥20) or HADS (≥10 for anxiety and ≥8 for depression), all of which are non-specific markers of psychological distress.

In agreement with this, we observed a robust association between fears and non-specific measures of psychological distress, such as the AHI, the HADS anxiety score and, to a lesser extent, the HADS depression score.

What the paper's own numbers show

Method. An odds ratio from logistic regression with a Wald interval satisfies CI = exp(b ∓ 1.96·SE) with OR = exp(b). So the OR must lie inside its interval and equal the interval's geometric centre, √(lower × upper), up to rounding. Each arm of the interval implies an SE(log OR): ln(OR/lower)/1.96 for the lower arm and ln(upper/OR)/1.96 for the upper arm. The two must agree and both must be positive. I applied this to all five odds ratios in the abstract.

Figure 3 gives the HADS anxiety association in two categories, so I also measured its red markers (high vs low fear cluster) by pixel position. I calibrated the log axis on its tick marks at 0.1, 1 and 10, which are exactly 194 px per decade apart, so 1 px is about 1.2% of an OR. As a check on the method, the four abstract ORs that correspond to a single Figure 3 row read 0.4% to 2.1% high, and their lower limits match to within 0.02. Upper ends beyond 10 are cut off at the edge of the plot, as the AHI row shows: its printed upper limit is 11.87, but the line stops at about 9.8. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
HADS anxiety OR and 95% CI Abstract 5.80 (1.19 to 4.22) OR lies outside its CI: 1.58 above the upper limit (a factor of 1.37)
Geometric centre of the HADS anxiety CI Abstract — √(1.19 × 4.22) = 2.24; reported OR / centre = 2.59
SE(log OR) implied by each arm, HADS anxiety derived — lower arm 0.808, upper arm −0.162 (a negative SE is impossible); symmetric SE 0.323
Interval of the printed width centred on 5.80 derived — 3.08 to 10.92
AHI OR and 95% CI Abstract 6.85 (3.95 to 11.87) centre 6.85; arms 0.281 / 0.280 (consistent)
HADS depression OR and 95% CI Abstract 2.37 (1.29 to 4.37) centre 2.37; arms 0.310 / 0.312 (consistent)
Low education OR and 95% CI Abstract 3.48 (1.37 to 8.83) centre 3.48; arms 0.476 / 0.475 (consistent)
High perceived activity OR and 95% CI Abstract 2.36 (1.10 to 5.04) centre 2.35; arms 0.389 / 0.387 (consistent)
HADS anxiety 8–10 vs <8 Figure 3 (image, no numbers) marker and line OR ≈ 2.28, line ≈ 1.20 to ≈ 4.18: matches the abstract's interval and its centre (2.24)
HADS anxiety >10 vs <8 Figure 3 (image, no numbers) marker and line OR ≈ 5.97, line ≈ 3.03 to ≥ 9.94 (cut off at 10): matches the abstract's point estimate (5.80)
Figure-reading check, AHI / depression / education / activity Figure 3 vs Abstract 6.85 / 2.37 / 3.48 / 2.36 read as 6.88 / 2.42 / 3.52 / 2.41 (ratio to printed 1.004 to 1.021)
Significance, if the OR is 2.24 with the printed CI derived — z = 2.50, p ≈ 0.013
Significance, if the OR is 5.80 with an SE like the printed CI's derived (illustration only) — 3.08 to 10.92; z = 5.44

The lower end of the Figure 3 >10 row (≈ 3.03) lies close to the lower limit of an interval of the printed width centred on 5.80 (3.08). Its lower arm implies an SE of 0.346, against 0.323 for the printed interval. That fits the reading that 5.80 is the >10 estimate and that its own interval was replaced by the 8–10 interval.

A separate, smaller inconsistency turned up in the same abstract. It does not affect the odds ratios:

Quantity Location Reported Recomputed
Mean FAIR score, RA Abstract vs Results Abstract: "51.2 (±25.4) in RA"; Results: "51.8±25.4" Overall mean 54.9 with axSpA 60.5 (n = 432 and 240) implies an RA mean of 51.79, so 51.8 fits. With 51.2, the overall mean would be 54.52, not 54.9

Figure

The printed HADS anxiety pair compared with the two internally consistent pairings and the Figure 3 rows

Figure 1. HADS anxiety odds ratios for the high vs low fear cluster, on one log axis with the null value (OR = 1) as a thin vertical line. Orange marks are numbers printed in the abstract. Blue marks are recomputed from them: the geometric centre of the printed interval (2.24) and an interval of the printed width centred on the printed OR (3.08 to 10.92). Gray hollow diamonds are approximate readings of the two HADS anxiety rows of the paper's Figure 3, measured by pixel position; the >10 row's line runs to the edge of that plot at 10, so its upper end is a lower bound. The printed pair (top row) is impossible because 5.80 lies above 4.22. Each of the two consistent pairings agrees with one Figure 3 row, and both lie wholly above 1.

The abstract's numbers traced to the two Figure 3 rows

Figure 2. The abstract's estimate and interval trace to different rows of Figure 3. The printed interval, 1.19 to 4.22, matches the 8–10 vs <8 row (≈ 1.20 to ≈ 4.18). The printed point estimate, 5.80, matches the >10 vs <8 row (≈ 5.97). The abstract names neither category. Figure 3 values are approximate readings, because the figure prints no numbers.

Why it matters

The HADS anxiety estimate is one of five headline associations in the abstract, and for most readers the abstract is the only place it appears as a number. The Results text gives no odds ratios, and Figure 3 shows them only graphically.

  1. The printed pair cannot be used as published. A reader or meta-analyst who takes the SE from the printed interval and the estimate from the printed OR gets a log OR that is about one SE above the interval's upper limit. A reader who back-calculates from either arm alone gets an SE of 0.808 or a negative one.
  2. The size of the anxiety effect depends on which number is right. With the >10 category the OR is about 5.8, close to the AHI estimate (6.85) and well above depression (2.37). With the 8–10 category it is about 2.2–2.3, about the same as depression. The Discussion's statement that the association is with "the HADS anxiety score and, to a lesser extent, the HADS depression score" holds for the >10 category but not for the 8–10 category. The abstract does not say which category its number refers to, and readers have no way to tell.
  3. Direction and significance are not affected. Both consistent pairings lie wholly above 1 (z = 2.50 for 2.24 with the printed interval). Figure 3, which appears to be drawn correctly, shows the same. The abstract's conclusion that "High fear scores were associated with psychological distress" is unaffected.

Explanations considered

Status

Suggested correction

  1. In the abstract, replace "OR 5.80, 95% CI (1.19 to 4.22)" with an estimate and interval from the same row of the regression output, and name the category. For example: "HADS anxiety >10 vs <8 (OR 5.80, 95% CI (x to y))", or the 8–10 vs <8 estimate with 1.19 to 4.22, whichever was intended. Reporting both categories would remove the ambiguity.
  2. Give the Figure 3 odds ratios and intervals as numbers, in the figure or in a table (online if necessary), so that readers do not have to read them off the plot. Extending the axis past 10 would show the full AHI and HADS >10 intervals.
  3. Make the HADS anxiety cut-off consistent between the Discussion ("≥10 for anxiety") and Figure 3 (">10").
  4. Minor: change the abstract's RA mean FAIR score from 51.2 to 51.8, which agrees with the Results and with the overall mean of 54.9.
Recomputation script (python recompute.py.txt)
"""Recompute the internal consistency of the odds ratios in the abstract of PMC5867422
(Ann Rheum Dis 2017, doi:10.1136/annrheumdis-2017-212000, the FAIR questionnaire paper).

Run:  python recompute.py.txt                     # prints all numbers, writes fig1/fig2 (.svg, .png)
      python recompute.py.txt --candidates        # also writes five candidate figure designs
                                                  # to candidates/ (A-E) for review
      python recompute.py.txt --figure f03.jpg    # re-measures Figure 3 from the publisher's image
Needs numpy and matplotlib.

Inputs are copied from the paper as archived in PMC (identical in the publisher's typeset PDF):
  Abstract (Results): the five odds ratios with 95% CIs, the RA and axSpA mean scores;
  Results ("Distribution of scores"): overall mean 54.9, RA 51.8, axSpA 60.5; n = 432 RA, 240 axSpA.
Method: an odds ratio from logistic regression with a Wald interval satisfies
  CI = exp(b -/+ 1.959964 * SE),  OR = exp(b),
so the OR must lie inside its CI and equal the geometric centre sqrt(lower * upper)
(up to rounding), and SE(log OR) = (ln upper - ln lower) / (2 * 1.959964).

Figure 3 (the paper's forest plot) gives the HADS anxiety OR in two categories (8-10 vs <8 and
>10 vs <8) but prints no numbers. Its red markers (high vs low fear cluster) were read by pixel
position on the log axis, calibrated on the tick marks at 0.1, 1 and 10 (x = 145.5, 339.5, 533.5 px;
194 px per decade, so 1 px is about 1.2% of an OR). The readings stored below were produced by
read_figure3() on the image file annrheumdis-2017-212000f03.jpg (703 x 1015 px, SHA-256
903d9a0e66f16d94a68154971c8494fc339ad26b9623da0327d18fab51888df0), downloaded from
https://europepmc.org/api/fulltextRepo?pprId=PMC5867422&type=FILE&fileName=annrheumdis-2017-212000f03.jpg&mimeType=image/jpeg
Pass --figure <path> to repeat the measurement. Interval ends drawn past OR = 10 are cut off at
the edge of the plot, so an upper end read near 10 is a lower bound for the true limit.
"""
import os
import sys
from math import exp, log, sqrt, erf

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.ticker import FixedLocator, NullLocator, FuncFormatter  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964

# ---------------------------------------------------------------- inputs from the paper
ABSTRACT = {  # label: (OR, lower, upper), Abstract (Results)
    "Helplessness (AHI)":         (6.85, 3.95, 11.87),
    "HADS anxiety":               (5.80, 1.19, 4.22),
    "HADS depression":            (2.37, 1.29, 4.37),
    "Low education":              (3.48, 1.37, 8.83),
    "High perceived activity":    (2.36, 1.10, 5.04),
}
# Figure 3 rows matching each abstract OR (red markers, cluster 1 vs cluster 3)
FIG_ROW_FOR = {
    "Helplessness (AHI)": "AHI >=20 vs <20",
    "HADS depression": "HADS depression >=8 vs <8",
    "Low education": "Primary vs tertiary education",
    "High perceived activity": "Patient-rated activity >60 vs <=30",
}
FIG_LABELS = ["AHI >=20 vs <20", "HADS anxiety 8-10 vs <8", "HADS anxiety >10 vs <8",
              "HADS depression >=8 vs <8", "Patient-rated activity 30-60 vs <=30",
              "Patient-rated activity >60 vs <=30", "Foreign vs France born",
              "Primary vs tertiary education", "Secondary vs tertiary education",
              "Employed vs retired", "Unemployed vs retired", "Living alone vs with others"]
# Stored readings (point, left end, right end), produced by read_figure3() on the file above
FIG3_STORED = {
    "AHI >=20 vs <20":                      (6.88, 3.94, 9.82),
    "HADS anxiety 8-10 vs <8":              (2.28, 1.20, 4.18),
    "HADS anxiety >10 vs <8":               (5.97, 3.03, 9.94),
    "HADS depression >=8 vs <8":            (2.42, 1.31, 4.23),
    "Patient-rated activity 30-60 vs <=30": (1.85, 1.02, 3.30),
    "Patient-rated activity >60 vs <=30":   (2.41, 1.11, 4.99),
    "Foreign vs France born":               (2.30, 1.09, 4.65),
    "Primary vs tertiary education":        (3.52, 1.37, 8.72),
    "Secondary vs tertiary education":      (1.67, 0.90, 2.89),
    "Employed vs retired":                  (3.54, 1.78, 6.72),
    "Unemployed vs retired":                (3.02, 1.31, 6.64),
    "Living alone vs with others":          (2.34, 1.20, 4.49),
}
N_RA, N_AXSPA = 432, 240
MEAN_ALL, MEAN_AXSPA = 54.9, 60.5
MEAN_RA_ABSTRACT, MEAN_RA_RESULTS = 51.2, 51.8


def read_figure3(path):
    """Measure the red markers of Figure 3 by pixel. Returns {label: (OR, left, right)}."""
    import matplotlib.image as mimg
    img = mimg.imread(path).astype(int)
    R, G, B = img[..., 0], img[..., 1], img[..., 2]
    red = (R > 170) & (G < 120) & (B < 120)
    red[:, 560:] = False                     # drop the legend
    x1, ppd = 339.5, 194.0                   # OR = 1 tick, pixels per decade (ticks 145.5/339.5/533.5)
    to_or = lambda x: 10 ** ((x - x1) / ppd)  # noqa: E731
    rows = np.where(red.sum(1) > 0)[0]
    groups, cur = [], [rows[0]]
    for r in rows[1:]:
        if r - cur[-1] <= 3:
            cur.append(r)
        else:
            groups.append(cur)
            cur = [r]
    groups.append(cur)
    out = {}
    for lab, g in zip(FIG_LABELS, groups):
        cc = red[g[0]:g[-1] + 1].sum(0)
        xs = np.where(cc > 0)[0]
        dot = np.where(cc >= 0.8 * cc.max())[0]
        out[lab] = (round(to_or(dot.mean()), 2), round(to_or(xs.min()), 2),
                    round(to_or(xs.max() + 1), 2))
    return out


def p_two_sided(z):
    return 2 * (1 - 0.5 * (1 + erf(abs(z) / sqrt(2))))


# ---------------------------------------------------------------- 1. abstract ORs
print("=== 1. Abstract ORs: does each OR sit at the geometric centre of its 95% CI? ===")
RES = {}
for k, (o, lo, hi) in ABSTRACT.items():
    gm = sqrt(lo * hi)
    se_lo = (log(o) - log(lo)) / Z
    se_hi = (log(hi) - log(o)) / Z
    se_sym = (log(hi) - log(lo)) / (2 * Z)
    RES[k] = dict(gm=gm, se_lo=se_lo, se_hi=se_hi, se=se_sym, inside=lo <= o <= hi)
    print(f"{k:25s} OR {o:5.2f} CI ({lo:.2f}, {hi:5.2f})  inside={str(lo <= o <= hi):5s} "
          f"centre sqrt(lo*hi) = {gm:5.2f}  ratio OR/centre = {o / gm:5.3f}  "
          f"SE from lower arm = {se_lo:6.3f}, upper arm = {se_hi:6.3f}, symmetric = {se_sym:.3f}")

o, lo, hi = ABSTRACT["HADS anxiety"]
a = RES["HADS anxiety"]
hw = (log(hi) - log(lo)) / 2
same_width = (exp(log(o) - hw), exp(log(o) + hw))
print("\n=== 2. HADS anxiety pair ===")
print(f"5.80 exceeds the upper limit 4.22 by {o - hi:.2f} (a factor of {o / hi:.3f}); "
      f"on the log scale by {(log(o) - log(hi)) / a['se']:.2f} SE (SE = {a['se']:.3f} from the printed CI)")
print(f"OR / centre of printed CI = {o / a['gm']:.2f}")
print(f"Interval of the printed width centred on 5.80: ({same_width[0]:.2f}, {same_width[1]:.2f})")
zc = log(a["gm"]) / a["se"]
zo = log(o) / a["se"]
print(f"Pairing 1, OR {a['gm']:.2f} (1.19 to 4.22): z = {zc:.2f}, p = {p_two_sided(zc):.4f}")
print(f"Pairing 2, OR 5.80 ({same_width[0]:.2f} to {same_width[1]:.2f}) if SE were the same: "
      f"z = {zo:.2f}, p = {p_two_sided(zo):.1e}")

print("Single-digit misprints of the OR, checked against the printed CI (arms must agree):")
for cand in (2.80, 3.80, 4.80, 5.30):
    print(f"  OR {cand:.2f}: SE from lower arm {(log(cand) - log(lo)) / Z:.3f}, "
          f"upper arm {(log(hi) - log(cand)) / Z:.3f}")

# ---------------------------------------------------------------- 3. Figure 3
fig_path = sys.argv[sys.argv.index("--figure") + 1] if "--figure" in sys.argv else None
FIG3 = read_figure3(fig_path) if fig_path else FIG3_STORED
print(f"\n=== 3. Figure 3 red markers ({'measured from ' + fig_path if fig_path else 'stored readings'}) ===")
for lab in FIG_LABELS:
    p, l_, r_ = FIG3[lab]
    print(f"{lab:38s} OR ~{p:5.2f}  line from ~{l_:4.2f} to ~{r_:5.2f}")
print("\nCalibration: abstract value vs Figure 3 reading for the four unambiguous ORs")
ratios = []
for k, lab in FIG_ROW_FOR.items():
    o_, l_, h_ = ABSTRACT[k]
    p, fl, fr = FIG3[lab]
    ratios.append(p / o_)
    clip = "  (true upper limit beyond the plot edge at 10)" if h_ > 10 else ""
    print(f"{k:25s} abstract {o_:5.2f} ({l_:.2f}, {h_:5.2f})  figure {p:5.2f} ({fl:.2f}, {fr:5.2f})  "
          f"point ratio {p / o_:.3f}{clip}")
print(f"mean point ratio figure/abstract = {np.mean(ratios):.3f} (range {min(ratios):.3f}-{max(ratios):.3f})")
f810, f10 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
print("\nHADS anxiety: which Figure 3 row does each abstract number match?")
print(f"abstract CI (1.19, 4.22) vs 8-10 row line ({f810[1]:.2f}, {f810[2]:.2f}); "
      f"vs >10 row line ({f10[1]:.2f}, {f10[2]:.2f})")
print(f"abstract OR 5.80 vs >10 row point {f10[0]:.2f} (ratio {f10[0] / 5.80:.3f}); "
      f"vs 8-10 row point {f810[0]:.2f}")
print(f"centre of abstract CI {a['gm']:.2f} vs 8-10 row point {f810[0]:.2f} (ratio {f810[0] / a['gm']:.3f})")
print(f"interval of printed width centred on 5.80 ({same_width[0]:.2f}, {same_width[1]:.2f}) vs "
      f">10 row ({f10[1]:.2f}, >= {f10[2]:.2f}, cut off at the plot edge)")
se_fig10 = (log(f10[0]) - log(f10[1])) / Z
print(f">10 row: SE(log OR) from the figure's lower arm = {se_fig10:.3f}; implied upper limit "
      f"{exp(log(f10[0]) + Z * se_fig10):.2f}")

# ---------------------------------------------------------------- 4. side check
print("\n=== 4. Side check: RA mean FAIR score (abstract 51.2 vs Results 51.8) ===")
ra_implied = (MEAN_ALL * (N_RA + N_AXSPA) - N_AXSPA * MEAN_AXSPA) / N_RA
print(f"RA mean implied by overall 54.9 and axSpA 60.5: {ra_implied:.2f}")
for m in (MEAN_RA_ABSTRACT, MEAN_RA_RESULTS):
    print(f"overall mean if RA = {m}: {(N_RA * m + N_AXSPA * MEAN_AXSPA) / (N_RA + N_AXSPA):.2f}")

# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#52514e"
plt.rcParams.update({
    "font.family": ["DejaVu Sans"], "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "path", "lines.linewidth": 2,
})
MS = 9  # marker size in points (>= 8 px)


def save(fig, stem):
    fig.savefig(stem + ".svg", bbox_inches="tight", pad_inches=0.12)
    fig.savefig(stem + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def log_axis(ax, lo=0.7, hi=16, ticks=(1, 2, 3, 5, 10)):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    ax.xaxis.set_major_locator(FixedLocator(list(ticks)))
    ax.xaxis.set_minor_locator(NullLocator())
    ax.xaxis.set_major_formatter(FuncFormatter(lambda v, _: f"{v:g}"))
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.axvline(1, color=INK2, lw=0.9, zorder=1)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel("Odds ratio, high vs low fear cluster (log scale; thin line = no association)")


def interval(ax, y, pt, l_, r_, ptc, linec, marker="o", hollow=False, arrow=False):
    ax.plot([l_, r_], [y, y], color=linec, lw=2, solid_capstyle="butt", zorder=2)
    for x in (l_, r_):
        ax.plot([x, x], [y - 0.12, y + 0.12], color=linec, lw=2, zorder=2)
    if arrow:
        ax.annotate("", xy=(r_ * 1.16, y), xytext=(r_, y),
                    arrowprops=dict(arrowstyle="-|>", color=linec, lw=2, shrinkA=0, shrinkB=0))
    ax.plot([pt], [y], marker=marker, ms=MS + (1 if marker == "D" else 2), zorder=3,
            mfc=BG if hollow else ptc, mec=ptc, mew=2, ls="none")


def legend_handles(items):
    hs = []
    for lab, kw in items:
        hs.append(plt.Line2D([], [], label=lab, **kw))
    return hs


# ---- Candidate A: the printed pair and the centre of its own interval
def fig_core(stem):
    fig, ax = plt.subplots(figsize=(7.2, 2.4))
    interval(ax, 1, 5.80, 1.19, 4.22, ORANGE, ORANGE)
    interval(ax, 0, a["gm"], 1.19, 4.22, BLUE, ORANGE)
    ax.set_yticks([1, 0])
    ax.set_yticklabels(["Abstract as printed:\nOR 5.80 (1.19 to 4.22)",
                        f"Centre of the printed CI:\nsqrt(1.19 x 4.22) = {a['gm']:.2f}"], color=INK)
    ax.set_ylim(-0.6, 1.7)
    log_axis(ax)
    ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 1.12), xytext=(7.4, 1.45),
                fontsize=10, color=INK, va="center",
                arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
    ax.legend(handles=legend_handles([
        ("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
        ("recomputed from the printed CI", dict(color=BLUE, marker="o", ms=MS, ls="none"))]),
        loc="upper center", bbox_to_anchor=(0.45, -0.32), ncol=2, frameon=False, fontsize=10)
    save(fig, stem)


# ---- Candidate B: all five abstract ORs, reported point vs geometric centre
def fig_all_ors(stem):
    keys = list(ABSTRACT)
    fig, ax = plt.subplots(figsize=(7.2, 3.3))
    for i, k in enumerate(keys):
        y = len(keys) - 1 - i
        o_, l_, h_ = ABSTRACT[k]
        ax.plot([l_, h_], [y, y], color=ORANGE, lw=2, zorder=2)
        for x in (l_, h_):
            ax.plot([x, x], [y - 0.12, y + 0.12], color=ORANGE, lw=2)
        ax.plot([RES[k]["gm"]], [y], marker="o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=4)
        ax.plot([o_], [y], marker="o", ms=MS, color=ORANGE, zorder=3)
        txt = (f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}" if abs(o_ / RES[k]["gm"] - 1) < 0.01
               else f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}: OR outside its CI")
        ax.text(17.5, y, txt, va="center", fontsize=10,
                color=INK if abs(o_ / RES[k]["gm"] - 1) < 0.01 else INK, weight="normal"
                if abs(o_ / RES[k]["gm"] - 1) < 0.01 else "bold")
    ax.set_yticks(range(len(keys)))
    ax.set_yticklabels(keys[::-1], color=INK)
    ax.set_ylim(-0.7, len(keys) - 0.3)
    log_axis(ax, 0.7, 16)
    ax.legend(handles=legend_handles([
        ("OR and 95% CI as printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
        ("geometric centre of the printed CI (recomputed)",
         dict(color=BLUE, marker="o", ms=MS + 5, mfc="none", mew=2, ls="none"))]),
        loc="upper center", bbox_to_anchor=(0.5, -0.28), ncol=1, frameon=False, fontsize=10)
    save(fig, stem)


# ---- Candidate C: the two arms of each interval, expressed as an implied SE
def fig_arms(stem):
    keys = list(ABSTRACT)
    fig, ax = plt.subplots(figsize=(7.2, 3.2))
    for i, k in enumerate(keys):
        y = len(keys) - 1 - i
        r = RES[k]
        ax.plot([r["se_lo"], r["se_hi"]], [y + 0.13, y - 0.13], color=GRID, lw=2, zorder=1)
        ax.plot([r["se_lo"]], [y + 0.13], marker="o", ms=MS, color=BLUE, ls="none", zorder=3)
        ax.plot([r["se_hi"]], [y - 0.13], marker="s", ms=MS, mfc=BG, mec=BLUE, mew=2, ls="none", zorder=3)
        ax.text(0.95, y, f"{r['se_lo']:.3f} / {r['se_hi']:.3f}", va="center", fontsize=10, color=INK)
    ax.axvline(0, color=INK2, lw=0.9)
    ax.set_xlim(-0.3, 1.2)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_yticks(range(len(keys)))
    ax.set_yticklabels(keys[::-1], color=INK)
    ax.set_ylim(-0.7, len(keys) - 0.3)
    ax.set_xlabel("SE(log OR) implied by each arm of the printed CI\n"
                  "(lower arm = ln(OR/lower)/1.96; upper arm = ln(upper/OR)/1.96; must agree and be > 0)")
    ax.legend(handles=legend_handles([
        ("from the lower arm", dict(color=BLUE, marker="o", ms=MS, ls="none")),
        ("from the upper arm", dict(color=BLUE, marker="s", ms=MS, mfc=BG, mew=2, ls="none"))]),
        loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
    save(fig, stem)


# ---- Candidate D: text panels, abstract excerpt vs the two Figure 3 rows
def fig_text_panels(stem):
    from matplotlib.patches import FancyBboxPatch
    fig = plt.figure(figsize=(7.2, 3.0))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1)
    ax.axis("off")
    for x0, w, title in ((0.01, 0.38, "Abstract (Results), as printed"),
                         (0.60, 0.39, "Figure 3, high vs low fear\n(red markers), read by pixel")):
        ax.add_patch(FancyBboxPatch((x0, 0.13), w, 0.84, boxstyle="round,pad=0.0,rounding_size=0.015",
                                    fc="#f3f2ee", ec=GRID, lw=1))
        ax.text(x0 + 0.02, 0.88, title, fontsize=10.5, weight="bold", va="center", color=INK)
    ax.text(0.03, 0.70, "\u201chigh Hospital Anxiety and\nDepression Scale anxiety", fontsize=10.5,
            va="center", color=INK)
    ax.text(0.03, 0.53, "(OR 5.80,", fontsize=12, va="center", color=ORANGE, weight="bold")
    ax.text(0.03, 0.38, "95% CI (1.19 to 4.22))\u201d", fontsize=12, va="center", color=ORANGE,
            weight="bold")
    ax.text(0.03, 0.22, "no HADS category is named", fontsize=10, va="center", color=INK2, style="italic")
    p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
    ax.text(0.62, 0.66, "HADS anxiety >10 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
    ax.text(0.62, 0.58, f"OR \u2248{p2[0]:.2f}  (\u2248{p2[1]:.2f} to \u226510;", fontsize=10.5,
            va="center", color=GRAY)
    ax.text(0.62, 0.50, "line cut off at the plot edge)", fontsize=10.5, va="center", color=GRAY)
    ax.text(0.62, 0.34, "HADS anxiety 8\u201310 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
    ax.text(0.62, 0.26, f"OR \u2248{p1[0]:.2f}  (\u2248{p1[1]:.2f} to \u2248{p1[2]:.2f})", fontsize=10.5,
            va="center", color=GRAY)
    kw = dict(arrowstyle="-|>", color=INK2, lw=1.4, shrinkA=4, shrinkB=4)
    ax.annotate("", xy=(0.605, 0.62), xytext=(0.175, 0.53), arrowprops=kw)
    ax.annotate("", xy=(0.605, 0.30), xytext=(0.355, 0.38), arrowprops=kw)
    ax.text(0.495, 0.60, "point estimate\nmatches", fontsize=10, ha="center", va="center", color=INK,
            bbox=dict(fc=BG, ec="none", pad=1.5))
    ax.text(0.495, 0.27, "interval\nmatches", fontsize=10, ha="center", va="center", color=INK,
            bbox=dict(fc=BG, ec="none", pad=1.5))
    ax.text(0.01, 0.05, "Orange: numbers printed in the abstract.  Gray: approximate values read from "
            "Figure 3, which prints no numbers.", fontsize=10, color=INK2, va="center")
    save(fig, stem)


# ---- Candidate E: printed pair, the two internally consistent pairings, and Figure 3
def fig_pairings(stem):
    fig, ax = plt.subplots(figsize=(7.2, 4.1))
    p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
    rows = [
        (6.0, "Abstract as printed", lambda y: interval(ax, y, 5.80, 1.19, 4.22, ORANGE, ORANGE)),
        (4.3, f"Centre of printed CI ({a['gm']:.2f})\nwith the printed CI",
         lambda y: interval(ax, y, a["gm"], 1.19, 4.22, BLUE, ORANGE)),
        (3.3, "Figure 3: 8–10 vs <8",
         lambda y: interval(ax, y, p1[0], p1[1], p1[2], GRAY, GRAY, marker="D", hollow=True)),
        (1.6, "Printed OR 5.80 with\nthe printed CI’s width",
         lambda y: interval(ax, y, 5.80, *same_width, ORANGE, BLUE)),
        (0.6, "Figure 3: >10 vs <8",
         lambda y: interval(ax, y, p2[0], p2[1], p2[2], GRAY, GRAY, marker="D", hollow=True, arrow=True)),
    ]
    for y, _, draw in rows:
        draw(y)
    ax.set_yticks([r[0] for r in rows])
    ax.set_yticklabels([r[1] for r in rows], color=INK)
    ax.text(0.72, 5.15, "If the interval is right: HADS anxiety 8–10 vs <8", fontsize=10,
            weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
    ax.text(0.72, 2.45, "If the point estimate is right: HADS anxiety >10 vs <8", fontsize=10,
            weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
    ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 6.15), xytext=(7.3, 6.45),
                fontsize=10, color=INK, va="center", arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
    ax.text(11.6, 1.6, f"{same_width[0]:.2f} to\n{same_width[1]:.2f}", fontsize=10, va="center", color=INK)
    ax.text(12.2, 0.6, "cut off\nat 10", fontsize=10, va="center", color=INK2)
    ax.text(4.6, 4.3, "1.19 to 4.22", fontsize=10, va="center", color=INK)
    ax.set_ylim(0.0, 6.9)
    log_axis(ax, 0.7, 20, ticks=(1, 2, 3, 5, 10, 20))
    ax.legend(handles=legend_handles([
        ("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
        ("recomputed from the printed numbers", dict(color=BLUE, marker="o", ms=MS, lw=2)),
        ("read from Figure 3 (approximate)",
         dict(color=GRAY, marker="D", ms=MS, mfc=BG, mew=2, lw=2))]),
        loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=3, frameon=False, fontsize=10,
        handlelength=2.2, columnspacing=1.2)
    save(fig, stem)


fig_pairings(os.path.join(HERE, "fig1"))
fig_text_panels(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (printed pair vs consistent pairings) and fig2.svg/png (abstract vs Figure 3 rows)")

if "--candidates" in sys.argv:
    cdir = os.path.join(HERE, "candidates")
    os.makedirs(cdir, exist_ok=True)
    for tag, fn in (("A_core_interval", fig_core), ("B_all_abstract_ors", fig_all_ors),
                    ("C_interval_arms", fig_arms), ("D_text_panels", fig_text_panels),
                    ("E_two_pairings", fig_pairings)):
        fn(os.path.join(cdir, tag))
    print("wrote candidates/A-E (.svg, .png)")

Printed 95% CI for a mortality predictor implies P ≈ 0.98, not the reported 0.006

BMC Med (2024) · doi:10.1186/s12916-024-03812-2 · PMC11656579

Summary

The paper fits a multivariable logistic model of mortality in patients who developed immune checkpoint inhibitor-related endocrinopathies (IREs; N = 173). It reports the pre-treatment platelet-to-lymphocyte ratio (PLR) as OR 1.004, 95% CI 0.7–1.4, P = 0.006, and names PLR as one of two significant predictors. The same three figures appear in the Abstract, in the Results text and in Table 5, and they cannot all be correct. A 95% CI of 0.7–1.4 contains 1 and implies a two-sided P of 0.98 (no lower than 0.73 under any rounding of the printed values). An OR of 1.004 with P = 0.006 would need a 95% CI of about 1.0011–1.0069. A reader therefore cannot tell from the paper whether PLR independently predicts mortality in this subgroup.

What the paper reports

After MVR, factors influencing mortality in the subgroup with IREs included a higher age (OR 1.1, 95% CI 1.04–1.2, P = 0.001) and platelet-to-lymphocyte ratio (OR 1.004, 95% CI 0.7–1.4, P = 0.006).

— Abstract, Results

According to the MVR analysis, only age at ICI initiation (OR 1.1, 95% CI: 1.04–1.2; P = 0.001) and PLR (OR 1.004, 95% CI: 0.7–1.4; P = 0.006) were found to be statistically significant factors for predicting mortality.

— Results, "Factors influencing mortality among patients who developed endocrinopathy post-ICI"

PLR | 1.001 (1.0001-1.003) | 0.03 | 1.004 (0.7-1.4) | 0.006

— Table 5, "Univariate and multivariate logistic regression analysis of factors influencing mortality following endocrinopathies post immune checkpoint inhibitor therapy", PLR row. The columns are univariate OR (95% CI), univariate P, multivariate OR (95% CI) and multivariate P. In the version of record the multivariate P of 0.006 is set in bold. The table footer says "P values in bold indicate statistical significance".

Similarly, mortality was predicted by age at ICI initiation and a higher pre-ICI PLR.

— Discussion, first paragraph

A P value of < 0.05 indicated statistical significance.

— Methods, "Statistical analysis"

Two related passages bear on how PLR is scaled. The first gives the definition (the archived text renders the superscript in "× 10³" as "103"):

Platelet-to-lymphocyte ratio (PLR) was calculated for the included patients using the formula PLR = Lymphocyte Count (× 103/μL) ÷ Platelet Count (× 103/μL).

— Methods, "Data sources and measures"

PLR | 145.3 (64.2-229.2) | 143.9 (27-259.6) | 0.7

— Table 3, PLR row (median and IQR, IRE vs no IRE)

What the paper's own numbers show

Method. A Wald 95% CI for a logistic-regression odds ratio is exp(ln OR ± 1.96 × SE), so it is symmetric on the log scale. The standard error can therefore be recovered from the printed bounds as SE = (ln hi − ln lo)/3.92, and the two-sided P follows from z = ln(OR)/SE. Run backwards, the 95% CI that must go with a given OR and P is exp(ln OR ± 1.96 × ln OR / z_P). To allow for rounding, each printed number was varied across its whole rounding interval (0.7 means 0.65–0.75, 1.004 means 1.0035–1.0045), and the estimate was also allowed to sit at the geometric centre of the CI. As a check on the method, the same calculation was run on nine other rows from Tables 4 and 5. Every number below is printed by recompute.py.txt (Python, numpy/scipy).

Quantity Location Reported Recomputed from the paper's numbers
Does the 95% CI contain 1? Abstract; Results; Table 5 0.7–1.4 Yes
SE of ln(OR) implied by the CI Table 5 0.7–1.4 (ln 1.4 − ln 0.7)/3.92 = 0.177
Geometric centre of the CI Table 5 OR 1.004 0.990
Two-sided P, multivariable PLR Abstract; Results; Table 5 0.006 0.98 with OR 1.004 (z = 0.023); 0.95 with the CI centre; one-sided 0.49
Smallest P under any rounding of the printed values Table 5 0.006 0.976 with the estimate rounding to 1.004; 0.73 if the estimate is taken as the CI centre
P if 0.7–1.4 were a 90%, 99% or 99.9% CI Table 5 0.006 0.985, 0.976, 0.970
95% CI required by OR 1.004 with P = 0.006 Abstract; Results; Table 5 0.7–1.4 1.0011–1.0069 (SE 0.00145, 122 times smaller than the SE implied by the printed CI)
Required CI over rounding of OR and P derived 0.7–1.4 lower bound 1.0010–1.0013, upper bound 1.0060–1.0078
OR per 100 PLR units, if 1.004 is per unit derived 1.004 1.004^100 = 1.49, which lies outside 0.7–1.4
Printed CI read as per 100 units, converted to per unit derived 0.7–1.4 0.9964–1.0034, which still contains 1
Control: univariable PLR, 1.001 (1.0001–1.003) Table 5 P = 0.03 0.036 (0.027–0.044 over rounding of the bounds)
Control: multivariable age, 1.1 (1.04–1.2) Abstract; Results; Table 5 P = 0.001 0.0024 (0.0002–0.0075)
Control: univariable HbA1c, 1.3 (1.01–1.6) Table 5 P = 0.03 0.041 (0.036–0.046)
Control: IRE model, pembrolizumab, 2.6 (1.05–6.3) Abstract; Table 4 P = 0.04 0.039 (0.038–0.040)
PLR scale: Methods formula, lymphocytes ÷ platelets Methods; Table 3 medians 145.3 and 143.9 Using the reference ranges in the Table 3 footer, lymphocytes ÷ platelets falls between 0.0024 and 0.020, and platelets ÷ lymphocytes between 50 and 410. The Table 3 values fit platelets ÷ lymphocytes

Figure 2 and the script's output list the other control rows: univariable age, univariable haemoglobin, and IRE-model age, WBC and nivolumab. In those nine control rows the recomputed P agrees with the printed P to within rounding, or within about a factor of two where the printed estimate is coarsely rounded. In the multivariable PLR row, the printed P and the P implied by the printed CI differ by more than two orders of magnitude and fall on opposite sides of the paper's 0.05 threshold.

Figure

Printed odds ratio and 95% CI for PLR in the multivariable mortality model, compared with the 95% CI that the printed OR and P value would require

Figure 1. Odds ratio for mortality per unit PLR in the multivariable model of the IRE subgroup, on one log-scale axis. The thin vertical line marks OR = 1. The orange interval and dot show the paper's printed result: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006. The interval straddles OR = 1. Its geometric centre is 0.990 (open blue diamond), and recomputed from its width it gives a two-sided P of 0.98 for an OR of 1.004. The filled blue diamond marks the 95% CI that an OR of 1.004 with P = 0.006 would actually require, 1.0011–1.0069. At this scale that interval is narrower than the marker. The printed interval and the printed P value cannot describe the same estimate.

Printed versus recomputed two-sided P values for ten rows of Tables 4 and 5

Figure 2. Two-sided P values printed in Tables 4 and 5 (orange circles; the arrow marks a value printed as "<0.001") compared with P values recomputed from each row's printed 95% CI (blue diamonds, with the estimate at the CI's geometric centre). The light-blue bars show how far the recomputed P moves when each printed CI bound varies across its rounding interval. The dashed line marks P = 0.05. In nine rows the printed and recomputed values are close, which shows that the method recovers the paper's own reporting. The multivariable PLR row (top) is the exception. Its printed P is 0.006, while its printed CI gives 0.73–1.0.

Why it matters

Mortality in the IRE subgroup is one of the paper's stated secondary outcomes. PLR is one of only two variables the Abstract lists as multivariable predictors of it. The Results text and Table 5 repeat the result, and the Discussion's opening summary restates it ("mortality was predicted by age at ICI initiation and a higher pre-ICI PLR"). The printed figures support two incompatible readings:

Nothing in the paper shows which reading is correct. The supplement contains no regression output that could settle it (see Status).

How PLR is defined adds a second difficulty with this variable. As written in Methods, the formula (lymphocytes ÷ platelets) would give values of about 0.002–0.02, not the medians near 145 shown in Table 3. If the values in Table 3 are platelets ÷ lymphocytes, as their size suggests, the formula in Methods is inverted. Either way, a reader cannot be sure what unit the per-unit OR refers to.

The paper's main conclusions do not depend on this row. These are the IRE incidence of 26.7%, the commonest IRE types, the predictors of IRE development, and the statement that "The development of IREs did not seem to influence mortality" (Abstract, Conclusions). The error affects a secondary, prognostic result, but that result is printed in the Abstract.

Explanations considered

From the paper's numbers alone, it is not possible to tell which of the three printed values is in error.

Status

As of 2026-10-23:

Sources: doi.org/10.1186/s12916-024-03812-2 · link.springer.com article · Table 5 page · PMC11656579 · Europe PMC record · Crossref works record

Suggested correction

The authors could re-check the multivariable logistic model of mortality in the IRE subgroup against the original software output. They could then report the PLR odds ratio, its 95% CI and its P value exactly as produced, so that the three values agree, and state the PLR unit that the OR refers to.

Separately, the PLR formula in Methods ("Lymphocyte Count ÷ Platelet Count") should be checked against the values in Table 3, which fit platelets ÷ lymphocytes.

Recomputation script (python recompute.py.txt)
"""Recompute the multivariable PLR mortality result of PMC11656579
(BMC Med 2024, doi:10.1186/s12916-024-03812-2) and regenerate the figures.

Run:   python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file.

All inputs are copied from the paper (Abstract, Results, Methods, Tables 3, 4, 5).
Method: Wald test on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964) and
z = ln(OR) / SE, two-sided P = 2 * (1 - Phi(|z|)).
"""
import itertools
import os

import numpy as np
from scipy.stats import norm

HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)  # 1.959964


def se_from_ci(lo, hi, z=Z):
    return (np.log(hi) - np.log(lo)) / (2 * z)


def p_two_sided(est, lo, hi, z=Z):
    se = se_from_ci(lo, hi, z)
    zz = np.log(est) / se
    return se, zz, 2 * norm.sf(abs(zz))


def half_unit(s):
    """Half of the last printed digit ('0.7' -> 0.05; '1.0001' -> 0.00005).
    A bound printed without decimals ('1') is treated as one decimal (1.0 +/- 0.05)."""
    d = len(s.split(".")[1]) if "." in s else 1
    return 0.5 * 10 ** (-d)


def interval(s, n=41):
    v, h = float(s), half_unit(s)
    return np.linspace(v - h, v + h - 1e-12, n)


def p_range(est_s, lo_s, hi_s, use_printed_est=True):
    """Min and max two-sided P over every value that rounds to the printed numbers.
    The estimate is taken as the geometric midpoint of the (unrounded) CI and,
    if use_printed_est, also anywhere inside its own rounding interval."""
    ps = []
    for lo, hi in itertools.product(interval(lo_s), interval(hi_s)):
        if not (0 < lo < hi):
            continue
        mid = np.sqrt(lo * hi)
        ps.append(p_two_sided(mid, lo, hi)[2])
        if use_printed_est:
            for est in interval(est_s, 11):
                ps.append(p_two_sided(est, lo, hi)[2])
    return min(ps), max(ps)


line = "-" * 78

# ---------------------------------------------------------------- 1. the PLR row
print(line)
print("1. Multivariable PLR, mortality model in the IRE subgroup (Abstract; Results; Table 5)")
print("   printed: OR 1.004, 95% CI 0.7-1.4, P = 0.006")
OR, LO, HI, P_REP = 1.004, 0.7, 1.4, 0.006
se, zz, p = p_two_sided(OR, LO, HI)
mid = np.sqrt(LO * HI)
print(f"   CI contains 1: {LO < 1 < HI}")
print(f"   SE(ln OR) from CI          = {se:.4f}")
print(f"   z = ln(1.004)/SE           = {zz:.4f}")
print(f"   two-sided P                = {p:.4f}   one-sided P = {p / 2:.4f}")
print(f"   geometric midpoint of CI   = {mid:.4f}")
_, zm, pm = p_two_sided(mid, LO, HI)
print(f"   P using the CI midpoint    = {pm:.4f} (z = {zm:.4f})")

# rounding: estimate held to values that round to 1.004
pmin_est = min(p_two_sided(e, lo, hi)[2]
               for lo in interval("0.7") for hi in interval("1.4") for e in interval("1.004", 11))
pmin_all, pmax_all = p_range("1.004", "0.7", "1.4")
print(f"   min P, estimate rounding to 1.004, bounds rounding to 0.7 / 1.4 = {pmin_est:.4f}")
print(f"   min P, also allowing the estimate to be the CI midpoint        = {pmin_all:.4f}")

for level in (0.90, 0.99, 0.999):
    zl = norm.ppf(0.5 + level / 2)
    print(f"   if 0.7-1.4 were a {level * 100:g}% CI: P = {p_two_sided(OR, LO, HI, zl)[2]:.4f}")

# ---------------------------------------------------------------- 2. CI required by OR and P
print(line)
print("2. 95% CI that OR 1.004 with P = 0.006 would require")
z_req = norm.isf(P_REP / 2)
se_req = np.log(OR) / z_req
req_lo, req_hi = np.exp(np.log(OR) - Z * se_req), np.exp(np.log(OR) + Z * se_req)
print(f"   z = {z_req:.4f}; SE = {se_req:.6f}; CI = {req_lo:.4f}-{req_hi:.4f}")
print(f"   ratio of SE implied by printed CI to SE required: {se / se_req:.0f}")
los, his = [], []
for e in interval("1.004", 11):
    for pp in np.linspace(0.0055, 0.0065, 11):
        s = np.log(e) / norm.isf(pp / 2)
        los.append(np.exp(np.log(e) - Z * s)); his.append(np.exp(np.log(e) + Z * s))
print(f"   over OR 1.0035-1.0045 and P 0.0055-0.0065: lower {min(los):.4f}-{max(los):.4f}, "
      f"upper {min(his):.4f}-{max(his):.4f}")
print(f"   required CI at 1 decimal: {req_lo:.1f}-{req_hi:.1f}")

# ---------------------------------------------------------------- 3. scaling checks
print(line)
print("3. Rescaling checks")
print(f"   1.004^10 = {OR ** 10:.4f}; 1.004^100 = {OR ** 100:.4f}")
print(f"   1.004^100 inside 0.7-1.4: {LO < OR ** 100 < HI}")
print(f"   0.7-1.4 read as per 100 units -> per unit {LO ** 0.01:.5f}-{HI ** 0.01:.5f} (contains 1)")
print(f"   per-100-unit OR implied by the required CI: {req_lo ** 100:.3f}-{req_hi ** 100:.3f}")

# ---------------------------------------------------------------- 4. PLR definition vs Table 3
print(line)
print("4. PLR definition (Methods) vs values (Table 3)")
print("   Methods: PLR = lymphocyte count / platelet count")
print("   Table 3 median PLR: 145.3 (IRE), 143.9 (no IRE)")
print("   Table 3 footer reference ranges: platelets 150-410, lymphocytes 1-3 (x10^3/uL)")
print(f"   lymphocytes/platelets at the reference limits: {1 / 410:.4f}-{3 / 150:.4f}")
print(f"   platelets/lymphocytes at the reference limits: {150 / 3:.0f}-{410 / 1:.0f}")

# ---------------------------------------------------------------- 5. control rows
print(line)
print("5. Same calculation on other printed rows. P from the CI midpoint; 'range' lets the CI bounds")
print("   vary over their rounding intervals (estimate = CI midpoint); 'wide' also lets the")
print("   estimate vary over its own rounding interval")
CONTROLS = [
    # label, source, est, lo, hi, printed P (string), printed P value for plotting
    ("Mortality MVR: PLR", "Table 5", "1.004", "0.7", "1.4", "0.006", 0.006),
    ("Mortality UV: PLR", "Table 5", "1.001", "1.0001", "1.003", "0.03", 0.03),
    ("Mortality MVR: age", "Table 5", "1.1", "1.04", "1.2", "0.001", 0.001),
    ("Mortality UV: age", "Table 5", "1.04", "1.02", "1.07", "<0.001", 0.001),
    ("Mortality UV: HbA1c", "Table 5", "1.3", "1.01", "1.6", "0.03", 0.03),
    ("Mortality UV: hemoglobin", "Table 5", "0.8", "0.7", "1", "0.03", 0.03),
    ("IRE MVR: age", "Table 4", "1.02", "1.003", "1.03", "0.02", 0.02),
    ("IRE MVR: WBC", "Table 4", "0.94", "0.89", "0.99", "0.04", 0.04),
    ("IRE MVR: pembrolizumab", "Table 4", "2.6", "1.05", "6.3", "0.04", 0.04),
    ("IRE MVR: nivolumab", "Table 4", "2.6", "1.04", "6.6", "0.04", 0.04),
]
results = []
for lab, src, e, lo, hi, prs, prv in CONTROLS:
    lo_f, hi_f = float(lo), float(hi)
    _, zc, pc = p_two_sided(np.sqrt(lo_f * hi_f), lo_f, hi_f)
    a, b = p_range(e, lo, hi, use_printed_est=False)
    wa, wb = p_range(e, lo, hi)
    results.append((lab, src, prs, prv, pc, a, b))
    print(f"   {lab:24s} {src}: {e} ({lo}-{hi}) printed P {prs:>6s} | "
          f"recomputed {pc:.4f}; range {a:.4f}-{b:.4f}; wide {wa:.4f}-{wb:.4f}")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
    "axes.edgecolor": FG2, "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})


def style(ax):
    ax.grid(axis="x", which="both", color=GRID, lw=0.8, zorder=0)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


# Figure 1: printed interval vs required interval, one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.7))
style(ax)
ax.set_xscale("log")
ax.axvline(1, color=FG2, lw=0.9, zorder=1)
ax.text(1, 2.62, "OR = 1 (no association)", ha="center", va="bottom", color=FG2, fontsize=10)
y_p, y_r = 1.75, 0.75
ax.plot([LO, HI], [y_p, y_p], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
for x in (LO, HI):
    ax.plot([x, x], [y_p - 0.08, y_p + 0.08], color=ORANGE, lw=2, zorder=3)
ax.plot(OR, y_p, "o", ms=9, color=ORANGE, zorder=4)
ax.plot(mid, y_p, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(LO, y_p + 0.17, "Printed: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006",
        ha="left", va="bottom", color=ORANGE, fontsize=10.5, fontweight="bold")
ax.text(HI, y_p - 0.17, f"Recomputed from this CI: centre {mid:.3f}; P = {p:.2f} for OR 1.004",
        ha="right", va="top", color=BLUE, fontsize=10)
ax.plot([req_lo, req_hi], [y_r, y_r], color=BLUE, lw=2, zorder=3)
ax.plot(OR, y_r, "D", ms=8, color=BLUE, zorder=4)
ax.text(LO, y_r + 0.17,
        f"Recomputed: 95% CI needed for OR 1.004 with P = 0.006: {req_lo:.4f}–{req_hi:.4f}",
        ha="left", va="bottom", color=BLUE, fontsize=10.5, fontweight="bold")
ax.text(1.03, y_r, "(narrower than the marker at this scale)",
        ha="left", va="center", color=FG2, fontsize=10)
ax.set_xlim(0.62, 1.6)
ax.set_ylim(0.25, 2.9)
ticks = [0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.4]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.set_xlabel("Odds ratio for mortality per unit PLR (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
           Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper's numbers")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=2,
          frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)

# Figure 2: printed vs recomputed P values across rows, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0))
style(ax)
ax.set_xscale("log")
n = len(results)
ys = np.arange(n)[::-1]
ax.axvline(0.05, color=FG2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(0.05, n - 0.35, "P = 0.05", ha="center", va="bottom", color=FG2, fontsize=10)
for y, (lab, src, prs, prv, pc, a, b) in zip(ys, results):
    ax.plot([a, b], [y, y], color=BLUE, alpha=0.35, lw=7, solid_capstyle="butt", zorder=2)
    ax.plot(pc, y, "D", ms=8, color=BLUE, zorder=4)
    ax.plot(prv, y, "o", ms=9, mfc=ORANGE, mec=ORANGE, zorder=3)
    if prs.startswith("<"):
        ax.annotate("", xy=(prv * 0.55, y), xytext=(prv, y),
                    arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.set_yticks(ys)
ax.set_yticklabels([f"{r[0]}  ({r[1]})" for r in results])
for t, r in zip(ax.get_yticklabels(), results):
    if r[0] == "Mortality MVR: PLR":
        t.set_fontweight("bold"); t.set_color(FG)
ax.set_xlim(1e-4, 1.5)
ax.set_ylim(-0.7, n - 0.1)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "1"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_xlabel("Two-sided P value (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=0, label="P printed in the paper"),
           Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="P recomputed from the printed 95% CI"),
           Line2D([], [], color=BLUE, alpha=0.35, lw=7, label="Range over rounding of the printed CI bounds")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.14), ncol=1,
          frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print(line)
print("Figures written to", HERE)

Abstract reverses the direction of both age effects on cortical thickness reported in Table 3

Brain (2023) · doi:10.1093/brain/awad100 · PMC10473572

Summary

In its brain-resilience results, the abstract says that "older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau". The paper's own Table 3 reports a negative age coefficient on AD-signature cortical thickness (standardized β = −0.49, 95% CI −0.613 to −0.366, P < 0.001) and a negative Time × Age coefficient (standardized β = −0.051, 95% CI −0.083 to −0.020, P = 0.002). Both mean that older age goes with thinner cortex and faster thinning. The Results, both relevant Discussion passages and Supplementary Table 5 all give that direction, and the printed intervals reproduce the printed t values, so the signs in the table are not typesetting slips. Only the abstract sentence states the reverse. Because the abstract is the part most often read and indexed, readers who rely on it get the direction of a secondary finding backwards; the estimates themselves and the paper's main conclusions are unaffected.

What the paper reports

Abstract, results paragraph:

Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau.

For contrast, the same abstract paragraph gives the cognitive-resilience result with the favourable direction attached to younger age:

Younger age, higher education and greater cortical thickness were associated with better cognitive performance at baseline.

Results, "Brain resilience", first paragraph:

In models that estimated main effects (i.e. after removing the three-way interaction term), older age was related to thinner cross-sectional AD-signature cortex [stβ (95%CI) = -0.49 (-0.613,-0.366), P < 0.001] and to accelerated cortical thinning [stβ (95%CI) = -0.051 (-0.083,-0.02), P < 0.01] independent of temporal meta-ROI tau.

Results of analyses with global tau burden were consistent with these findings (Supplementary Table 5).

Table 3 ("Results of linear mixed-effect models investigating determinants of brain resilience to tau burden in the temporal meta-ROI"), Age row. Columns: Time × Tau × Variable β (unstandardized | standardized | t | P), Time × Variable β (same four), Variable β (same four):

Agea | 0.001 (-0.001,0.002) | 0.009 (-0.021,0.038) | 0.566 | 0.572 | -0.001 (-0.002,0) | -0.051 (-0.083,-0.02) | -3.182 | 0.002 | -0.012 (-0.015,-0.009) | -0.49 (-0.613,-0.366) | -7.774 | <0.001

Table 3 footer, on coding of the outcome and predictors:

Time (years), temporal meta-ROI tau (SUVR), age at baseline (years), educational level (years of education) and ICV (dm3) were centred and used in continuous form

The ‘Time × Variable β’ and ‘Variable β’ coefficients are main effects from models where the non-significant interaction term was removed.

Methods, "Determinants":

For the current study, age (in years) was defined as the age at the time of the tau-PET scan

Discussion, first paragraph:

While there was no evidence for age as a moderator in brain resilience models, older age was associated with lower cortical thickness at the time of the tau-PET scan, and with faster cortical thinning over time.

Discussion, "Age and cortical thickness":

we found that younger age was associated with both higher baseline cortical thickness and slower rate of cortical thinning at similar levels of tau burden.

Conclusion:

we found that age, education, ICV and cortical thickness play a role in cognitive resilience, while age and education contribute to brain resilience.

What the paper's own numbers show

Method. The outcome of the Table 3 models is AD-signature cortical thickness in mm (a higher value is a thicker cortex), and age is continuous, centred, in years. A negative Age coefficient therefore means a thinner cortex at the tau-PET scan for older participants, and a negative Time × Age coefficient means a more negative thickness slope, i.e. faster thinning, for older participants. The abstract's sentence requires both coefficients to be positive. To check that the printed signs belong to the printed statistics, I recomputed each standard error from its 95% CI (SE = (upper − lower)/(2 × 1.96)), the t value and P value that follow, and conversely the CI implied by the printed β and t. I also checked that the unstandardized and standardized rows agree, and repeated the sign check on Supplementary Table 5 (global tau, global thickness). Raw data are not available, so the models themselves cannot be refitted. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Age → baseline thickness, standardized β Table 3; Results −0.49 (−0.613, −0.366), t = −7.774, P < 0.001 CI midpoint −0.4895; SE 0.0630; t −7.78; P ≈ 7.5 × 10⁻¹⁵; negative, CI excludes 0
Time × Age → rate of thinning, standardized β Table 3; Results −0.051 (−0.083, −0.02), t = −3.182, P = 0.002 (Results: P < 0.01) CI midpoint −0.0515; SE 0.0161; t −3.17; P ≈ 0.0015; negative, CI excludes 0
CI rebuilt from printed β and t Table 3 as above Age −0.614 to −0.366; Time × Age −0.082 to −0.020 (match the printed CIs)
Age, unstandardized (mm per year of age) Table 3 −0.012 (−0.015, −0.009) t −7.84 from the CI; negative; 10 years older ≈ 0.12 mm thinner at the tau-PET scan
Time × Age, unstandardized Table 3 −0.001 (−0.002, 0) negative; t from the CI is only −1.96 because the values are rounded to 3 decimals
Agreement of standardized and unstandardized rows Table 3 — −0.49 / −0.012 = 40.8; a printed −0.001 (−0.0015 to −0.0005) × 40.8 = −0.061 to −0.020, which contains the printed −0.051
Expected thickness gap, 1 SD older vs average age, average tau Table 3 model without the three-way term — −0.49 SD at the tau-PET scan; −0.567 SD at the median MRI follow-up (18 months); −0.758 SD at the longest (63 months): the gap widens
Age as moderator of the tau effect (Time × Tau × Age) Table 3 0.009 (−0.021, 0.038), P = 0.572 t 0.60 from the CI; not significant, consistent with "independent of tau"
Age → baseline thickness, global analysis Supplementary Table 5 −0.374 (−0.522 – −0.227), t = −5.001, P < 0.001 t −4.97 from the CI; negative
Time × Age → rate of thinning, global analysis Supplementary Table 5 −0.044 (−0.084 – −0.003), t = −2.112, P = 0.036 t −2.13 from the CI; negative
Direction stated in the abstract Abstract older → higher baseline thickness, slower thinning requires Age β > 0 and Time × Age β > 0; both printed coefficients are < 0

Figure

The abstract's statement of the age effects beside the Results text and Table 3 values, with t values recomputed from the printed intervals

Figure 1. The left panel holds the abstract's sentence on age and brain resilience and the direction it states. The right panel holds the Results sentence and the Table 3 Age row (standardized β with 95% CI and t), all as printed (orange). The connector marks the conflict: the two panels give opposite directions for both the baseline (cross-sectional) effect and the longitudinal (Time × Age) effect. The blue lines give the t and P values recomputed from the printed confidence intervals; they match the printed t values, so the negative signs are internally consistent within the table.

Standardized age coefficients from Table 3 with 95% CIs, as printed and as rebuilt from the printed β and t, against the direction the abstract states

Figure 2. Each panel shows one Table 3 coefficient on its own symmetric axis, with zero as a thin reference line; the shaded half is the direction the abstract states (older age with thicker cortex, or with slower thinning). Orange circles and bars are the printed estimates and 95% CIs; blue squares and bars are the intervals rebuilt from the printed β and t (SE = β/t). Both intervals lie entirely on the negative side, away from the direction stated in the abstract.

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the abstract, change the sentence so that it states the direction reported in Table 3, the Results and the Discussion, for example:

Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with lower baseline cortical thickness and faster rate of cortical thinning independent of tau.

Equivalently, "younger age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau", which matches the wording of the Discussion. No change to the estimates, tables or conclusions appears to be needed.

Recomputation script (python recompute.py.txt)
"""Recompute the age effects on brain resilience in PMC10473572
(Brain 2023, doi:10.1093/brain/awad100).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure
designs (cand1..cand5, .svg and .png) in the candidates/ subfolder.

All inputs are copied from the article as archived in PubMed Central:
  Table 3, Age row (brain resilience, outcome = AD-signature cortical thickness, mm):
      Time x Tau x Age, Time x Age and Age coefficients, unstandardized and standardized,
      with 95% CIs, t and P.
  Results > Brain resilience (the same standardized values, quoted in the text).
  Methods / Results: median MRI follow-up 18 months (range 9-63 months).
  Supplementary Table 5, Age row (global tau, global cortical thickness as outcome).
No raw data are available, so the model itself cannot be refitted. The checks below test
(i) that the printed CIs, t and P belong together, so the printed signs are not typos, and
(ii) what direction the printed coefficients imply.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from scipy.stats import norm  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (from the paper)
# Table 3, Age row: (beta, lo, hi, t, P as printed)
T3 = {
    "age_std": (-0.49, -0.613, -0.366, -7.774, "<0.001"),
    "timexage_std": (-0.051, -0.083, -0.020, -3.182, "0.002"),
    "age_unstd": (-0.012, -0.015, -0.009, -7.774, "<0.001"),
    "timexage_unstd": (-0.001, -0.002, 0.0, -3.182, "0.002"),
    "3way_std": (0.009, -0.021, 0.038, 0.566, "0.572"),
}
# Supplementary Table 5, Age row (global tau, global cortical thickness)
S5 = {
    "age_std": (-0.374, -0.522, -0.227, -5.001, "<0.001"),
    "timexage_std": (-0.044, -0.084, -0.003, -2.112, "0.036"),
}
FU_MEDIAN_Y, FU_MAX_Y = 18 / 12, 63 / 12  # brain resilience subsample, MRI follow-up

LABELS = {
    "age_std": "Age, standardized (baseline thickness)",
    "timexage_std": "Time x Age, standardized (rate of thinning)",
    "age_unstd": "Age, unstandardized (mm per year of age)",
    "timexage_unstd": "Time x Age, unstandardized (mm/yr per year of age)",
    "3way_std": "Time x Tau x Age, standardized (moderation)",
}


def from_ci(b, lo, hi):
    se = (hi - lo) / (2 * Z)
    t = b / se
    return (lo + hi) / 2, se, t, 2 * norm.sf(abs(t))


def main():
    R = {}
    print("== 1. Table 3, Age row: SE, t and P recomputed from the printed 95% CIs")
    print("   (normal approximation; SE = (upper - lower) / (2 x 1.96))")
    for k, (b, lo, hi, t, p) in T3.items():
        mid, se, tr, pr = from_ci(b, lo, hi)
        R[k] = (mid, se, tr, pr)
        print(f"  {LABELS[k]}")
        print(f"     printed beta {b} ({lo}, {hi}), t = {t}, P = {p}")
        print(f"     CI midpoint {mid:.4f}; SE {se:.4f}; t from CI {tr:.2f}; P from CI {pr:.2g}; "
              f"sign {'negative' if b < 0 else 'positive'}; CI excludes 0: {hi < 0 or lo > 0}")

    print("\n== 2. 95% CI rebuilt from the printed beta and t (SE = beta / t)")
    CI_T = {}
    for k in ("age_std", "timexage_std"):
        b, lo, hi, t, p = T3[k]
        se = b / t
        CI_T[k] = (b - Z * se, b + Z * se)
        print(f"  {LABELS[k]}: SE = {se:.4f}; CI {b - Z * se:.3f} to {b + Z * se:.3f} "
              f"(printed {lo} to {hi})")

    print("\n== 3. Do the unstandardized and standardized rows agree in sign and scale?")
    ratio = T3["age_std"][0] / T3["age_unstd"][0]
    print(f"  Age: standardized / unstandardized = {T3['age_std'][0]} / {T3['age_unstd'][0]} "
          f"= {ratio:.1f} (= SD of age / SD of thickness, in years per mm)")
    lo_u, hi_u = -0.0015, -0.0005  # values that round to -0.001
    print(f"  Time x Age unstandardized printed as -0.001, i.e. between {lo_u} and {hi_u};")
    print(f"     times {ratio:.1f} gives a standardized value between {lo_u * ratio:.3f} and "
          f"{hi_u * ratio:.3f}; printed standardized value {T3['timexage_std'][0]} lies inside")
    print("     (time is in years in both, so the same factor applies)")
    print(f"  Unstandardized Age: -0.012 mm per year of age -> 10 years older = "
          f"{10 * T3['age_unstd'][0]:.2f} mm thinner AD-signature cortex at the tau-PET scan")

    print("\n== 4. What the Table 3 coefficients imply (model without the 3-way term, average tau)")
    b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
    for yrs, lab in ((0, "baseline (tau-PET)"), (FU_MEDIAN_Y, "median MRI follow-up, 18 months"),
                     (FU_MAX_Y, "longest MRI follow-up, 63 months")):
        print(f"  1 SD older, at {lab}: expected thickness difference = {b0} + ({b1} x {yrs:.2f}) "
              f"= {b0 + b1 * yrs:.3f} SD")
    print("  Both terms are negative: the older participant starts thinner and the gap widens,")
    print("  i.e. older age -> lower baseline thickness and faster thinning.")
    print("  The abstract ('older age ... higher baseline cortical thickness and slower rate of")
    print("  cortical thinning') would need Age beta > 0 and Time x Age beta > 0.")
    b, lo, hi, t, p = T3["3way_std"]
    print(f"  Age as moderator of the tau effect: {b} ({lo}, {hi}), P = {p}: not significant,")
    print("  consistent with the abstract's 'independent of tau'.")

    print("\n== 5. Supplementary Table 5 (global tau, global cortical thickness), Age row")
    for k, (b, lo, hi, t, p) in S5.items():
        mid, se, tr, pr = from_ci(b, lo, hi)
        print(f"  {LABELS[k]}: printed {b} ({lo}, {hi}), t = {t}, P = {p}; "
              f"t from CI {tr:.2f}; P from CI {pr:.2g}; sign {'negative' if b < 0 else 'positive'}")

    os.makedirs(CAND, exist_ok=True)
    designs = [cand1_text_panels, cand2_intervals, cand3_grid, cand4_trajectory, cand5_combined]
    for i, f in enumerate(designs, 1):
        fig = f(R, CI_T)
        save(fig, os.path.join(CAND, f"cand{i}"))
        plt.close(fig)
    # chosen designs for the write-up
    save(cand1_text_panels(R, CI_T), os.path.join(HERE, "fig1"))
    save(cand2_intervals(R, CI_T), os.path.join(HERE, "fig2"))
    print("\nFigures written: fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")


# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
PANEL = "#f3f2ef"
plt.rcParams.update({
    "font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
    "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
W = 7.2


def save(fig, stem):
    fig.savefig(stem + ".svg")
    fig.savefig(stem + ".png", dpi=200)


def box(ax, x, y, w, h, edge=SEC, face=PANEL, lw=1.0):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc=face, ec=edge, lw=lw, transform=ax.transAxes))


def lines(ax, x, y, items, dy=0.058):
    """items: list of (text, color, weight, size)."""
    for txt, col, wt, sz in items:
        ax.text(x, y, txt, color=col, fontweight=wt, fontsize=sz, va="top", ha="left",
                transform=ax.transAxes)
        y -= dy * sz / 10.5
    return y


def fmt(v, nd=3):
    return f"{v:.{nd}f}".replace("-", "−")


# 1 ---- two text panels joined by an annotated connector
def cand1_text_panels(R, CI_T):
    fig = plt.figure(figsize=(W, 4.9))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    ax.text(0.03, 0.965, "The abstract and the body give opposite directions for both age effects",
            fontsize=12, fontweight="bold", va="top")
    ax.text(0.03, 0.91, "Brain resilience models; outcome = AD-signature cortical thickness (mm)",
            fontsize=10, color=SEC, va="top")
    # left panel
    box(ax, 0.03, 0.255, 0.385, 0.605)
    lines(ax, 0.05, 0.835, [
        ("ABSTRACT", SEC, "bold", 10),
        ("\u201c\u2026older age was associated", TXT, "normal", 10.5),
        ("with higher baseline cortical", TXT, "bold", 10.5),
        ("thickness and slower rate of", TXT, "bold", 10.5),
        ("cortical thinning independent", TXT, "bold", 10.5),
        ("of tau.\u201d", TXT, "normal", 10.5),
    ], dy=0.052)
    lines(ax, 0.05, 0.43, [
        ("Direction stated:", SEC, "normal", 10),
        ("older \u2192 thicker at baseline", ORANGE, "bold", 10.5),
        ("older \u2192 slower thinning", ORANGE, "bold", 10.5),
    ], dy=0.052)
    # right panel
    box(ax, 0.55, 0.255, 0.425, 0.605)
    lines(ax, 0.57, 0.835, [
        ("RESULTS AND TABLE 3", SEC, "bold", 10),
        ("\u201colder age was related to", TXT, "normal", 10.5),
        ("thinner cross-sectional", TXT, "bold", 10.5),
        ("AD-signature cortex [\u2026] and", TXT, "normal", 10.5),
        ("to accelerated cortical", TXT, "bold", 10.5),
        ("thinning\u201d", TXT, "bold", 10.5),
    ], dy=0.052)
    lines(ax, 0.57, 0.50, [
        ("Table 3, Age row, std. \u03b2 (95% CI):", SEC, "normal", 10),
        ("Age  \u22120.49 (\u22120.613, \u22120.366)", ORANGE, "bold", 10),
        ("Time\u00d7Age  \u22120.051 (\u22120.083, \u22120.02)", ORANGE, "bold", 10),
        ("t = \u22127.774 and \u22123.182", ORANGE, "bold", 10),
        ("older \u2192 thinner, faster thinning", TXT, "bold", 10.5),
    ], dy=0.052)
    # connector
    ar = FancyArrowPatch((0.42, 0.56), (0.545, 0.56), arrowstyle="<|-|>", mutation_scale=14,
                         lw=2, color=TXT, transform=ax.transAxes)
    ax.add_patch(ar)
    ax.text(0.4825, 0.605, "opposite", ha="center", fontsize=10, fontweight="bold")
    ax.text(0.4825, 0.535, "sign for\nboth", ha="center", va="top", fontsize=10)
    # recomputed strip
    _, se0, t0, p0 = R["age_std"]; _, se1, t1, p1 = R["timexage_std"]
    ax.text(0.03, 0.215, "Recomputed from the printed 95% CIs:", fontsize=10, color=SEC, va="top")
    ax.text(0.03, 0.165, f"Age t \u2248 {fmt(t0, 2)} (P \u2248 {p0:.0e});  Time\u00d7Age t \u2248 "
            f"{fmt(t1, 2)} (P \u2248 {p1:.4f})", fontsize=10, color=BLUE, fontweight="bold", va="top")
    ax.text(0.03, 0.115, "Both intervals exclude 0, and the t values match the printed ones,",
            fontsize=10, color=BLUE, va="top")
    ax.text(0.03, 0.078, "so the negative signs in Table 3 are not typesetting slips.",
            fontsize=10, color=BLUE, va="top")
    ax.text(0.97, 0.008, "orange = printed in the paper    blue = recomputed from its numbers",
            fontsize=10, color=SEC, ha="right", va="bottom")
    return fig


# 2 ---- dot-and-interval plots, one panel per coefficient, zero line, abstract direction shaded
def cand2_intervals(R, CI_T):
    fig, axs = plt.subplots(2, 1, figsize=(W, 4.2), gridspec_kw=dict(hspace=0.75))
    spec = [("age_std", "Age → baseline thickness (standardized β)", 0.7,
             "older → thinner at baseline", "abstract: older → thicker at baseline"),
            ("timexage_std", "Time × Age → rate of thinning (standardized β per year)", 0.1,
             "older → faster thinning", "abstract: older → slower thinning")]
    for ax, (k, title, lim, lneg, lpos) in zip(axs, spec):
        b, lo, hi, t, p = T3[k]
        ax.axvspan(0, lim, color=ORANGE, alpha=0.08, lw=0)
        ax.axvline(0, color=SEC, lw=0.8)
        ax.errorbar([b], [1], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2,
                    capsize=0)
        cl, ch = CI_T[k]
        ax.errorbar([b], [0], xerr=[[b - cl], [ch - b]], fmt="s", color=BLUE, ms=8, lw=2,
                    capsize=0)
        ax.set_yticks([1, 0])
        ax.set_yticklabels(["Table 3, printed", "rebuilt from β and t"], fontsize=10)
        ax.set_ylim(-0.7, 2.3)
        ax.set_xlim(-lim, lim)
        ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
        for s in ("top", "right", "left"):
            ax.spines[s].set_visible(False)
        ax.tick_params(axis="y", length=0)
        ax.set_title(title, loc="left", fontsize=10.5, fontweight="bold", pad=6)
        ax.text(lim * 0.97, 2.15, lpos, ha="right", va="top", fontsize=10, color=SEC)
        ax.text(-lim * 0.97, 2.15, lneg, ha="left", va="top", fontsize=10, color=SEC)
        ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)} ({fmt(lo)}, {fmt(hi)})", (hi, 1),
                    xytext=(8, 0), textcoords="offset points", va="center", fontsize=10,
                    color=ORANGE, fontweight="bold")
        ax.annotate(f"({fmt(cl)}, {fmt(ch)})", (ch, 0), xytext=(8, 0),
                    textcoords="offset points", va="center", fontsize=10, color=BLUE,
                    fontweight="bold")
    fig.subplots_adjust(left=0.22, right=0.97, top=0.9, bottom=0.08)
    return fig


# 3 ---- consistency grid: every place the paper states the direction
def cand3_grid(R, CI_T):
    rows = [
        ("Abstract", "thicker", "slower", True),
        ("Results › Brain resilience", "thinner", "faster (accelerated)", False),
        ("Table 3, Age row", "−0.49 (thinner)", "−0.051 (faster)", False),
        ("Discussion, 1st paragraph", "lower", "faster", False),
        ("Discussion › Age and cortical thickness*", "lower", "faster", False),
        ("Supplementary Table 5 (global)", "−0.374 (thinner)", "−0.044 (faster)", False),
    ]
    fig = plt.figure(figsize=(W, 3.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
    ax.text(0.03, 0.95, "Where the paper states the direction of the age effects on thickness",
            fontsize=11.5, fontweight="bold", va="top")
    x0, x1, x2 = 0.03, 0.46, 0.73
    yh = 0.83
    ax.text(x1, yh, "Older age → baseline\nthickness", fontsize=10, color=SEC, va="center",
            fontweight="bold")
    ax.text(x2, yh, "Older age → rate\nof thinning", fontsize=10, color=SEC, va="center",
            fontweight="bold")
    dy = 0.095
    for i, (src, a, b, bad) in enumerate(rows):
        y = yh - 0.12 - i * dy
        if bad:
            ax.add_patch(FancyBboxPatch((0.02, y - dy / 2 + 0.008), 0.96, dy - 0.016,
                                        boxstyle="round,pad=0,rounding_size=0.01",
                                        fc="#fbe6dd", ec=ORANGE, lw=1.5, transform=ax.transAxes))
        else:
            ax.plot([0.02, 0.98], [y - dy / 2, y - dy / 2], color=GRID, lw=0.8,
                    transform=ax.transAxes)
        col = ORANGE if bad else TXT
        ax.text(x0 + 0.01, y, src, fontsize=10.5, va="center", fontweight="bold" if bad else "normal")
        ax.text(x1, y, a, fontsize=10.5, va="center", color=col, fontweight="bold")
        ax.text(x2, y, b, fontsize=10.5, va="center", color=col, fontweight="bold")
    ax.text(0.03, 0.025, "Highlighted: the only statement with the opposite direction.\n"
            "*Phrased as 'younger age → higher baseline thickness, slower thinning'.",
            fontsize=10, color=SEC, va="bottom")
    return fig


# 4 ---- implied expected difference over follow-up, from the Table 3 coefficients
def cand4_trajectory(R, CI_T):
    import numpy as np
    b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
    t = np.linspace(0, FU_MAX_Y, 50)
    fig, ax = plt.subplots(figsize=(W, 3.6))
    ax.axhspan(0, 0.9, color=ORANGE, alpha=0.08, lw=0)
    ax.axhline(0, color=SEC, lw=0.8)
    ax.plot(t, b0 + b1 * t, color=BLUE, lw=2)
    ax.plot([0], [b0], "o", color=ORANGE, ms=8)
    ax.annotate(f"Table 3 Age β = {fmt(b0, 2)}\n(95% CI {fmt(T3['age_std'][1])}, "
                f"{fmt(T3['age_std'][2])})", (0, b0), xytext=(8, -20), textcoords="offset points",
                va="top", fontsize=10, color=ORANGE, fontweight="bold")
    for yrs, lab in ((FU_MEDIAN_Y, "median follow-up"), (FU_MAX_Y, "longest follow-up")):
        v = b0 + b1 * yrs
        ax.plot([yrs], [v], "s", color=BLUE, ms=8)
        ax.annotate(f"{lab}: {fmt(v, 2)}", (yrs, v), xytext=(0, 10), textcoords="offset points",
                    ha="center" if yrs < 4 else "right", fontsize=10, color=BLUE,
                    fontweight="bold")
    ax.text(0.1, 0.8, "region the abstract implies (older participant thicker, gap closing or "
            "reversing)", fontsize=10, color=SEC, va="top")
    ax.text(FU_MAX_Y * 0.98, -0.2, f"slope = Time×Age β = {fmt(b1)} SD/year\n(gap widens: "
            "faster thinning)", fontsize=10, color=BLUE, ha="right", va="top")
    ax.set_xlim(-0.1, FU_MAX_Y + 0.1); ax.set_ylim(-0.9, 0.9)
    ax.set_xlabel("Years from tau-PET")
    ax.set_ylabel("Thickness difference, 1 SD older\nvs average age (SD units)")
    ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.set_title("Expected AD-signature thickness gap implied by Table 3 (average tau)",
                 loc="left", fontsize=11, fontweight="bold")
    fig.tight_layout()
    return fig


# 5 ---- compact: abstract claim on the left, the two printed intervals on the right
def cand5_combined(R, CI_T):
    fig = plt.figure(figsize=(W, 3.4))
    axt = fig.add_axes([0, 0, 0.36, 1]); axt.axis("off")
    box(axt, 0.06, 0.34, 0.9, 0.5)
    lines(axt, 0.11, 0.80, [
        ("ABSTRACT", SEC, "bold", 10),
        ("“older age was associated", TXT, "normal", 10.5),
        ("with higher baseline", TXT, "bold", 10.5),
        ("cortical thickness and", TXT, "bold", 10.5),
        ("slower rate of cortical", TXT, "bold", 10.5),
        ("thinning”", TXT, "bold", 10.5),
        ("implies both β > 0", ORANGE, "bold", 10.5),
    ])
    axt.text(0.06, 0.95, "Claim vs Table 3", fontsize=11.5, fontweight="bold", va="top")
    ax = fig.add_axes([0.47, 0.2, 0.5, 0.62])
    ks = [("age_std", "Age\n(baseline)"), ("timexage_std", "Time×Age\n(slope)")]
    ax.axvspan(0, 0.7, color=ORANGE, alpha=0.08, lw=0)
    ax.axvline(0, color=SEC, lw=0.8)
    for y, (k, lab) in zip((1, 0), ks):
        b, lo, hi, t, p = T3[k]
        ax.errorbar([b], [y], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2)
        ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)}", (b, y), xytext=(0, 9),
                    textcoords="offset points", ha="center", fontsize=10, color=ORANGE,
                    fontweight="bold")
    ax.set_yticks([1, 0]); ax.set_yticklabels([k[1] for k in ks], fontsize=10)
    ax.set_ylim(-0.6, 1.6); ax.set_xlim(-0.7, 0.7)
    ax.set_xlabel("Standardized β, Table 3 (95% CI)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.text(0.68, 1.5, "abstract's\ndirection", ha="right", va="top", fontsize=10, color=SEC)
    ax.set_title("Table 3 (orange, printed): both < 0", loc="left", fontsize=10.5,
                 fontweight="bold")
    _, _, t0, _ = R["age_std"]; _, _, t1, _ = R["timexage_std"]
    fig.text(0.47, 0.03, f"Recomputed from CIs: t ≈ {fmt(t0, 2)} and {fmt(t1, 2)}",
             fontsize=10, color=BLUE, fontweight="bold")
    return fig


if __name__ == "__main__":
    main()

A Discussion passage reverses the lesion–donation directions stated in the Abstract and Results

Brain (2018) · doi:10.1093/brain/awy064 · PMC7341482

Summary

The paper maps penetrating brain injuries in 94 veterans against how much money each gave away on an altruistic decision task, and its Abstract, Results, Discussion summary and Figure 3 legend agree that lesions of the dorsomedial parietal cortex (precuneus, paracentral lobule) increased donations while lesions of the right posterior middle temporal gyrus (MTG) and superior temporal sulcus decreased them. The Discussion subsection that interprets this result says the opposite for both regions ("Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations"), and calls left-sided a parietal effect that the Abstract calls "bilateral" and Table 3 lists for both hemispheres (12 right, 11 left subjects). The paragraph's closing interpretation follows the reversed direction, so the paper's functional reading of one of its two main findings conflicts with that finding as stated, and the figures' colour bars do not independently settle which statement is correct. No statistic depends on the wording, but no correction has been published.

What the paper reports

The direction given in the Abstract (Abstract, main finding (ii)):

altruistic donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri decreased donations

The same direction in the Results (Results > Donation, paragraph 1, sentences 1–2):

Lesions that led to increased donations were restricted to the paracentral lobule and precuneus, as well as to small areas in the right dorsomedial prefrontal cortex (Fig. 3).

Injuries leading to a decrease in donations were restricted to small sectors of the right ventrolateral occipital cortex and posterior middle temporal gyrus (Fig. 3).

The same direction in the Discussion's summary of main findings (Discussion, paragraph 1):

altruistic (costly) donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri had the opposite effect

The colour code of the donation map (Figure 3 legend):

Statistical β-maps of lesions that decreased (red) or increased (blue) donations on the Altruistic Decision Task.

The reversed statement (Discussion > Brain regions implicated in donations, paragraph 1, sentence 3):

Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations.

The interpretation that closes the same paragraph:

Overall, our findings suggest that the superomedial parietal cortex is somehow involved in the facilitation of donations, whereas the ventrolateral occipitotemporal cortex is necessary for decisions that reduce the probability of donating.

The definition of the sign of the lesion maps (Materials and methods, lesion analysis paragraph):

each with a positive or a negative sign (i.e. the variable of interest is positively or negatively correlated with the presence of a brain lesion at a specific location

What the paper's own numbers show

Method. The error concerns the direction of an effect, not a computed value. For each passage that describes the lesion–donation result, I recorded which way it says a lesion moves donations in the two regions involved: medial parietal (precuneus/paracentral) and right posterior temporal (posterior MTG, pSTS, ventrolateral occipital). I then compared each passage with Results > Donation. The closing sentence of the Discussion paragraph speaks of the function of intact regions, so I translated it with the usual lesion-deficit reading: if a region "facilitates" donations, damage to it should lower them. I checked the laterality claim against Table 3, and checked that the Table 2 subgroups add up and that their means fit their labels. To see whether the figures follow the colour code their legends give, I counted strongly red and strongly blue pixels in the relevant panels of the paper's Figs 2 and 3 as archived in PMC (the figures are not reproduced here). As a side check, I tested whether each Table 2 column sums to $60, which the task design requires (see below). Every quote and number is checked or computed in recompute.py.txt.

Quantity Location Reported Recomputed / checked
Effect of medial parietal (precuneus/paracentral) lesions on donations Abstract (ii); Results > Donation; Discussion summary (ii) increased Same direction in all three passages
Same effect Discussion > Brain regions implicated in donations, sentence 3 "decreased" Opposite to the three passages above
Effect of right posterior MTG/pSTS lesions on donations Abstract (ii); Results > Donation; Discussion summary (ii) decreased Same direction in all three passages
Same effect Discussion > Brain regions implicated in donations, sentence 3 "increased" Opposite to the three passages above
Lesion effects implied by the closing interpretation (parietal "facilitation"; occipitotemporal cortex needed to "reduce" donating) Same paragraph, last sentence — Parietal lesion → fewer donations, occipitotemporal lesion → more: the same as the reversed sentence
Passages agreeing with Results / reversing both regions All six passages above, Fig. 3 legend included — 4 agree (Results included) / 2 reverse
Laterality of the medial parietal effect Abstract and Discussion summary: "bilateral"; Discussion subsection: "left" left (subsection) Table 3, "Dorsomedial parietal cortex (precuneus)": side R/L, 12 right / 11 left subjects (23 in all)
Fig. 3, medial parietal panels A, B, G: red / blue pixels Fig. 3 blue = increased (legend) 0 / 562, 0 / 664, 0 / 148: blue only, so "increased" under the legend
Fig. 3, lateral occipitotemporal panels E, F: red / blue pixels Fig. 3 red = decreased (legend) 313 / 50 and 419 / 40: mostly red, so "decreased" under the legend
Fig. 2 legend check: right perisylvian (A) vs medial dorsomedial prefrontal (E–G), red / blue pixels Fig. 2; Results > Punishment (perisylvian decreased, dorsomedial prefrontal increased punishment) red = decreased, blue = increased 3817 / 312 and 241 / 3092: the legend code matches the Results text for punishment as well
Fig. 3 colour bar Fig. 3 (labelled "Beta") — The end labelled 10 is dark red, RGB (140, 20, 29); the end labelled −10 is dark blue, RGB (0, 49, 96)
Lesion subgroup sizes (P / D / S) Table 2 18+34+22+20; 43+23+28(+0); 32+25+32+5 94 / 94 / 94, equal to the 94 patients
Mean donations, D+ (n = 23) / D− (n = 28) / controls (n = 28) Table 2 $20.0 / $9.7 / $12.9 D+ is 7.1 above controls and D− 3.2 below: the means fit the subgroup labels. Table 2 does not say which regions define D+ and D−
Side check: mean donations + punishments + savings Table 2, P− column 13.5 + 2.6 + 39.4 55.5 (55.35–55.65 under rounding), not 60
Side check: same sum Table 2, D− column 9.7 + 4.5 + 43.8 58.0 (57.85–58.15), not 60
Side check: same sum Table 2, D0 and S+ columns (savings printed without decimals) 14.7 + 4.6 + 40; 11.4 + 2.6 + 45 59.3 (58.70–59.90) and 59.0 (58.40–59.60): just short of 60 even allowing ±0.5 for the whole-dollar savings
Side check: same sum Table 2, the other 8 columns, controls included — 59.6–60.0, all consistent with 60 under rounding

Four sources agree on the direction: the Abstract, the Results, the Discussion's summary, and the Figure 3 legend read against the map colours. The Discussion subsection reverses both regions, and its closing interpretation follows that reversal.

The Table 2 sums are a separate matter and do not bear on which direction is correct. The task gives each participant $60, with 30 opportunities to donate and 30 to punish at $1 each, and pays out "$60 minus the amount you spent on donations and punishments" (supplementary instructions). Donations + punishments + savings therefore equal 60 for every participant, and so do subgroup means taken over the same participants. The P− and D− columns are well outside what rounding allows.

Figure

The conflicting statements: Abstract and Results versus the Discussion subsection on donations

Figure 1. Short excerpts quoted from the paper. The blue panel holds the direction stated in the Abstract (finding ii) and in Results > Donation; the Discussion's opening summary and the Figure 3 legend give the same direction. The orange panel holds the Discussion subsection "Brain regions implicated in donations": its third sentence, and the closing sentence that interprets the finding. The connector names the conflict. In the subsection, lesions in both regions have the opposite effect on donations, and the parietal effect, "bilateral" in the Abstract, becomes "left".

Direction of the lesion effect on donations, by passage and region

Figure 2. The direction each passage gives for the effect of a lesion on donations, by region. Blue marks the direction stated in the Abstract and Results; orange marks the reverse. The first four rows agree. The Discussion sentence reverses both regions, and so does the paragraph's closing interpretation when read in the usual lesion-deficit way (italic, because the direction is inferred rather than stated). The grey bottom row is conditional: it shows what Figure 3 would say if its colour bar (red = +10, blue = −10, labelled "Beta") were read with the Methods' definition, under which a positive sign means that a lesion goes with more donations. That reading would agree with the Discussion sentence and not with the legend. The paper does not say how β was signed for display (see Explanations considered).

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

Ask the authors to state the sign convention used to display the β-maps in Figs 2 and 3, check it against the underlying maps, and correct whichever passages disagree with it.

If the Abstract, Results and figure legends are correct, the Discussion subsection "Brain regions implicated in donations" could read, for example: "Damage to the medial parietal lobe (bilaterally) increased donations, whereas damage to the right posterior middle temporal gyrus decreased donations." The closing interpretation would then need to be revised to match: on the usual lesion-deficit reading, the intact superomedial parietal cortex would be associated with restraining donations, and the right ventrolateral occipitotemporal cortex with supporting them.

If instead the colour bars are correct and the legends reversed, the Abstract (findings i and ii), the Results, the Discussion summary and the figure legends would all need correcting.

Separately, the authors could check the P−, D−, D0 and S+ columns of Table 2, whose mean donations, punishments and savings should sum to $60.

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7341482 and regenerate the figures.

Run:  python recompute.py.txt
      (writes fig1.svg/png and fig2.svg/png next to this file)

Paper: "Altruistic decisions following penetrating traumatic brain injury",
Brain 2018;141(5):1558-1569, doi:10.1093/brain/awy064, PMC7341482.

All inputs are transcribed from the article (Abstract, Results > Donation,
Discussion, Tables 2 and 3, legends of Figs 2 and 3) and from its single
supplementary file (task instructions, Supplementary Table 2 and figure legends).

Optional parts, skipped with a message if unavailable:
  part 0 checks every quote used in the write-up verbatim against the PMC archive
         (needs the papers repository and its src/show.py);
  part 4 downloads the paper's Fig. 2 and Fig. 3 images from the PMC CDN into a
         temporary directory, checks their MD5, and counts strongly red and strongly
         blue pixels in a few panels. Nothing from the images is saved or redrawn.
"""
import hashlib
import os
import subprocess
import tempfile
import textwrap
import urllib.request

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------------------
# 0. Quotes used in writeup.md, checked verbatim against the PMC archive
# ---------------------------------------------------------------------------
QUOTES = {
    "Abstract (ii)":
        "altruistic donations were increased by bilateral lesions of the dorsomedial "
        "parietal cortex, whereas lesions of the right posterior superior temporal "
        "sulcus and middle temporal gyri decreased donations",
    "Results > Donation (1)":
        "Lesions that led to increased donations were restricted to the paracentral "
        "lobule and precuneus, as well as to small areas in the right dorsomedial "
        "prefrontal cortex (Fig. 3).",
    "Results > Donation (2)":
        "Injuries leading to a decrease in donations were restricted to small sectors "
        "of the right ventrolateral occipital cortex and posterior middle temporal "
        "gyrus (Fig. 3).",
    "Discussion summary (ii)":
        "altruistic (costly) donations were increased by bilateral lesions of the "
        "dorsomedial parietal cortex, whereas lesions of the right posterior superior "
        "temporal sulcus and middle temporal gyri had the opposite effect",
    "Fig. 3 legend":
        "Statistical β-maps of lesions that decreased (red) or increased (blue) "
        "donations on the Altruistic Decision Task.",
    "Discussion > donations, sentence 3":
        "Damage to the left medial parietal lobe decreased donations, whereas damage "
        "to the right middle temporal gyrus increased donations.",
    "Discussion > donations, closing sentence":
        "Overall, our findings suggest that the superomedial parietal cortex is "
        "somehow involved in the facilitation of donations, whereas the ventrolateral "
        "occipitotemporal cortex is necessary for decisions that reduce the "
        "probability of donating.",
    "Methods, sign of maps":
        "each with a positive or a negative sign (i.e. the variable of interest is "
        "positively or negatively correlated with the presence of a brain lesion at a "
        "specific location",
    "Methods, task":
        "each participant had the opportunity to keep a maximum of $60, donate a "
        "maximum of $30, or spend a maximum of $30 in punishments",
    "Results > Punishment":
        "In the right hemisphere, the lesions leading to decreased punishment followed "
        "a perisylvian distribution",
    "Results > Punishment (increase)":
        "Lesions that increased punishments were found (i) bilaterally in the "
        "dorsomedial prefrontal cortex",
}
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
print("== 0. Verbatim check of quotes against the PMC archive ==")
try:
    full = subprocess.run([PY, "src/show.py", "PMC7341482", "--full"], cwd=REPO,
                          capture_output=True, text=True, timeout=120).stdout
    if not full:
        raise RuntimeError("show.py returned nothing")
    for k, q in QUOTES.items():
        print(f"  {k:42s} {'found verbatim' if q in full else 'NOT FOUND'}")
except Exception as e:  # repository not available
    print(f"  skipped ({e})")

# ---------------------------------------------------------------------------
# 1. Direction of the lesion effect on donations, passage by passage
#    "+" = lesion increased donations, "-" = lesion decreased donations
# ---------------------------------------------------------------------------
PAR = "medial parietal"
TMP = "right posterior temporal"
passages = [
    # (location, kind, {region: sign}, parietal laterality)
    ("Abstract, finding (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
    ("Results > Donation", "stated", {PAR: "+", TMP: "-"}, "not stated"),
    ("Discussion, opening summary (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
    ("Fig. 3 legend read with map colours", "stated", {PAR: "+", TMP: "-"}, "n/a"),
    ("Discussion > donations, sentence 3", "stated", {PAR: "-", TMP: "+"}, "left"),
    ("Discussion > donations, closing sentence", "implied (lesion-deficit reading)",
     {PAR: "-", TMP: "+"}, "n/a"),
]
print("\n== 1. Direction of lesion effect on donations, by passage ==")
ref = passages[1][2]  # Results > Donation
n_agree = n_rev = 0
for loc, kind, d, lat in passages:
    flips = [r for r in d if d[r] != ref[r]]
    tag = ("REVERSES Results for both regions" if len(flips) == 2 else
           "agrees with Results" if not flips else f"reverses {flips}")
    n_agree += not flips
    n_rev += len(flips) == 2
    print(f"  {loc:42s} parietal {d[PAR]}  temporal {d[TMP]}  "
          f"parietal side: {lat:10s} [{kind}] -> {tag}")
print(f"  passages agreeing with Results: {n_agree} (including Results itself); "
      f"reversing both regions: {n_rev}")

# ---------------------------------------------------------------------------
# 2. Table 2 (lesion subgroups)
#    Methods and supplementary instructions: $60 to start, 30 donate and 30 punish
#    opportunities at $1 each; payout = $60 minus donations minus punishments.
#    So for each participant D + P + S = 60, and the same holds for subgroup means
#    computed over the same participants.
# ---------------------------------------------------------------------------
T2 = {  # (donations, punishments, savings, n) as printed
    "P0": ("13.2", "3.8", "42.9", 18), "P+": ("16.1", "6.6", "37.3", 34),
    "P-": ("13.5", "2.6", "39.4", 22), "P±": ("14.1", "4.0", "41.8", 20),
    "D0": ("14.7", "4.6", "40", 43), "D+": ("20.0", "4.7", "35.2", 23),
    "D-": ("9.7", "4.5", "43.8", 28),
    "S0": ("14.3", "4.4", "41", 32), "S+": ("11.4", "2.6", "45", 25),
    "S-": ("17.9", "6.7", "35", 32), "S±": ("14.2", "2.4", "43.4", 5),
    "Controls": ("12.9", "5.8", "41.2", 28),
}
N_PATIENTS = 94


def halfwidth(s):
    """Largest rounding error of a printed value (0.05 for one decimal, 0.5 for none)."""
    return 0.5 * 10 ** (-(len(s.split(".")[1]) if "." in s else 0))


print("\n== 2. Table 2 ==")
for g in "PDS":
    ks = [k for k in T2 if k[0] == g]
    print(f"  {g} subgroups: n = " + " + ".join(str(T2[k][3]) for k in ks)
          + f" = {sum(T2[k][3] for k in ks)} (patients: {N_PATIENTS})")
d_plus, d_minus, ctl = (float(T2[k][0]) for k in ("D+", "D-", "Controls"))
print(f"  mean donations  D+ {d_plus:.1f}  D- {d_minus:.1f}  controls {ctl:.1f};  "
      f"D+ minus controls = {d_plus - ctl:+.1f},  D- minus controls = {d_minus - ctl:+.1f}")
print("  mean donations + punishments + savings (each participant's total is 60):")
offenders = []
for k, (d, p, s, n) in T2.items():
    tot = float(d) + float(p) + float(s)
    hw = halfwidth(d) + halfwidth(p) + halfwidth(s)
    ok = tot - hw <= 60 + 1e-9 <= tot + hw + 2e-9
    if not ok:
        offenders.append((k, tot, tot - hw, tot + hw))
    print(f"    {k:9s} n={n:3d}  {d:>5s} + {p:>4s} + {s:>5s} = {tot:5.1f}  "
          f"(range under rounding {tot - hw:.2f} to {tot + hw:.2f})  "
          f"{'consistent with 60' if ok else 'NOT consistent with 60'}")
print("  columns whose sum cannot be 60 under rounding: "
      + "; ".join(f"{k} {t:.1f} ({lo:.2f}-{hi:.2f})" for k, t, lo, hi in offenders))

# ---------------------------------------------------------------------------
# 3. Table 3 (donation rows), as printed; laterality of the parietal effect
# ---------------------------------------------------------------------------
T3_DON = [("Middle temporal sulcus (posterior half)", "R", "37", "6"),
          ("Dorsomedial parietal cortex (precuneus)", "R/L", "5/7m", "12/11"),
          ("Medial orbitofrontal/gyrus rectus", "R/L", "11", "6/7"),
          ("Inferior frontal gyrus", "L", "45/46/47", "17")]
print("\n== 3. Table 3, donation rows ==")
for r in T3_DON:
    print(f"  {r[0]:42s} side {r[1]:4s} BA {r[2]:9s} subjects (R/L) {r[3]}")
prec_r, prec_l = map(int, T3_DON[1][3].split("/"))
print(f"  precuneus: {prec_r} right + {prec_l} left subjects = {prec_r + prec_l}; "
      "listed for both hemispheres, not left only")
print("  Table 3 gives no direction (increase/decrease) for any donation region")

# ---------------------------------------------------------------------------
# 4. Optional: colour check of the paper's Fig. 2 and Fig. 3 (not redrawn)
# ---------------------------------------------------------------------------
FIGS = {
    "f2": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/bd9cea49707e/awy064f2.jpg",
           "bd9cea49707e56866080625dd2ce2009"),
    "f3": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/af8bc4065be1/awy064f3.jpg",
           "af8bc4065be1d710719d4406426873f8"),
}
# panel boxes (x0, x1, y0, y1) in pixels of the 764 x 512 PMC images
BOX3 = {"A medial parietal (precuneus/SPL)": (40, 260, 60, 270),
        "B medial (precuneus/dmPFC)": (255, 480, 0, 160),
        "G medial": (490, 690, 300, 512),
        "E lateral (occipital/pMTG)": (0, 255, 300, 512),
        "F lateral (occipital/pMTG)": (255, 490, 300, 512)}
BOX2 = {"A right lateral perisylvian": (0, 240, 20, 215),
        "E/F/G medial dmPFC": (250, 700, 215, 512)}


def counts(img, box):
    x0, x1, y0, y1 = box
    a = img[y0:y1, x0:x1, :3].astype(float)
    if a.max() <= 1:
        a = a * 255
    R, G, B = a[..., 0], a[..., 1], a[..., 2]
    red = ((R - B) > 60) & ((R - G) > 50)
    blue = ((B - R) > 60) & (B > 110)
    return int(red.sum()), int(blue.sum())


print("\n== 4. Colour check of the paper's figures "
      "(legends: red = decreased, blue = increased) ==")
try:
    from matplotlib.image import imread
    imgs = {}
    with tempfile.TemporaryDirectory() as td:
        for k, (url, md5) in FIGS.items():
            req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
            data = urllib.request.urlopen(req, timeout=30).read()
            if hashlib.md5(data).hexdigest() != md5:
                raise RuntimeError(f"{k}: image differs from the one checked")
            path = os.path.join(td, k + ".jpg")
            with open(path, "wb") as fh:
                fh.write(data)
            imgs[k] = imread(path)
    print("  Fig. 3 (donations):")
    for k, b in BOX3.items():
        r, bl = counts(imgs["f3"], b)
        print(f"    {k:36s} red {r:5d}  blue {bl:5d}")
    print("  Fig. 2 (punishment), legend convention checked against Results > Punishment:")
    for k, b in BOX2.items():
        r, bl = counts(imgs["f2"], b)
        print(f"    {k:36s} red {r:5d}  blue {bl:5d}")
    col = imgs["f3"][:, 722, :3].astype(int)  # colour bar column of Fig. 3
    top, bot = tuple(int(v) for v in col[60]), tuple(int(v) for v in col[460])
    print(f"  Fig. 3 colour bar (labelled 'Beta'): end labelled 10 has RGB {top} (red); "
          f"end labelled -10 has RGB {bot} (blue)")
except Exception as e:  # network or image problems: skip this optional part
    print(f"  skipped ({e})")

# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "text.color": TXT, "svg.fonttype": "none"})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


# ---- fig1: two text panels joined by an annotated connector ---------------
W, H = 7.2, 5.0
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")

PX, PW, PY0, PH = (0.12, 3.78), 3.3, 1.55, 3.33


def panel(x, color, title, sub, blocks):
    ax.add_patch(FancyBboxPatch((x, PY0), PW, PH,
                                boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=color, lw=2))
    ax.text(x + 0.15, PY0 + PH - 0.14, title, fontsize=11, fontweight="bold",
            color=color, va="top")
    ax.text(x + 0.15, PY0 + PH - 0.42, sub, fontsize=10, color=SEC, va="top")
    yy = PY0 + PH - 0.80
    for head, body in blocks:
        ax.text(x + 0.15, yy, head, fontsize=10, color=SEC, va="top", style="italic")
        yy -= 0.23
        wrapped = textwrap.fill(body, 40)
        ax.text(x + 0.15, yy, wrapped, fontsize=10, color=TXT, va="top",
                linespacing=1.3)
        yy -= 0.205 * (wrapped.count("\n") + 1) + 0.2


left = [("Abstract, finding (ii)",
         "“...donations were increased by bilateral lesions of the dorsomedial "
         "parietal cortex, whereas lesions of the right posterior superior temporal "
         "sulcus and middle temporal gyri decreased donations”"),
        ("Results > Donation",
         "“Lesions that led to increased donations were restricted to the "
         "paracentral lobule and precuneus ...”")]
right = [("Paragraph 1, third sentence",
          "“Damage to the left medial parietal lobe decreased donations, whereas "
          "damage to the right middle temporal gyrus increased donations.”"),
         ("Same paragraph, closing sentence",
          "“...the superomedial parietal cortex is somehow involved in the "
          "facilitation of donations ...”")]
panel(PX[0], BLUE, "Abstract and Results",
      "(also Discussion summary, Fig. 3 legend)", left)
panel(PX[1], ORANGE, "Discussion subsection",
      "“Brain regions implicated in donations”", right)

# connector: a bracket from the bottom of each panel to a labelled box
lx, rx = PX[0] + PW / 2, PX[1] + PW / 2
ybot, ymid = PY0 - 0.03, 1.12
ax.plot([lx, lx, rx, rx], [ybot, ymid, ymid, ybot], color=TXT, lw=2,
        solid_joinstyle="miter", zorder=1)
ax.add_patch(FancyBboxPatch((W / 2 - 1.85, 0.86), 3.7, 0.52,
                            boxstyle="round,pad=0.02,rounding_size=0.06",
                            fc=BG, ec=TXT, lw=2, zorder=2))
ax.text(W / 2, 1.12, "Conflict: opposite direction for both regions",
        ha="center", va="center", fontsize=10.5, fontweight="bold", color=TXT, zorder=3)
rows = [("medial parietal lesions:", "increased", "decreased"),
        ("right MTG lesions:", "decreased", "increased"),
        ("parietal laterality:", "“bilateral”", "“left”")]
for i, (lab, a, b) in enumerate(rows):
    y = 0.55 - 0.21 * i
    ax.text(2.85, y, lab, ha="right", va="center", fontsize=10, color=SEC)
    ax.text(2.95, y, a, ha="left", va="center", fontsize=10, color=BLUE,
            fontweight="bold")
    ax.text(4.0, y, "→", ha="center", va="center", fontsize=10, color=SEC)
    ax.text(4.15, y, b, ha="left", va="center", fontsize=10, color=ORANGE,
            fontweight="bold")
save(fig, "fig1")

# ---- fig2: direction grid, passage x region --------------------------------
rows = [
    ("Abstract, finding (ii)", "+", "-", "stated"),
    ("Results > Donation", "+", "-", "stated"),
    ("Discussion, opening summary (ii)", "+", "-", "stated"),
    ("Fig. 3 legend + map colours", "+", "-", "stated"),
    ("Discussion > donations, sentence 3", "-", "+", "stated"),
    ("Same paragraph, closing sentence*", "-", "+", "implied"),
    ("Fig. 3 colour bar, if +β = more†", "-", "+", "conditional"),
]
nr = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 5.4), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 7.2)
ax.set_ylim(-3.95, nr + 0.85)
ax.axis("off")
cols = {1: (3.85, "Medial parietal\n(precuneus/paracentral)"),
        2: (5.9, "Right posterior MTG/\npSTS/occipital")}
ax.text(0.08, nr + 0.35, "Passage", va="center", fontsize=10, fontweight="bold")
for _, (cx, lab) in cols.items():
    ax.text(cx, nr + 0.35, lab, ha="center", va="center", fontsize=10,
            fontweight="bold")
ax.plot([0.05, 7.15], [nr - 0.5] * 2, color=SEC, lw=1)
for i, (lab, a, b, kind) in enumerate(rows):
    y = nr - 1 - i
    ax.plot([0.05, 7.15], [y - 0.5] * 2, color=GRID, lw=0.8)
    ax.text(0.08, y, lab, va="center", fontsize=10,
            color=TXT if kind == "stated" else SEC,
            style="normal" if kind == "stated" else "italic")
    for j, s in ((1, a), (2, b)):
        cx = cols[j][0]
        color = BLUE if s == rows[1][j] else ORANGE
        if kind == "conditional":
            color = SEC
        word = "▲ increased" if s == "+" else "▼ decreased"
        ax.text(cx, y, word, ha="center", va="center", fontsize=10.5,
                color=color, fontweight="bold" if kind == "stated" else "normal")
leg = [(BLUE, "same direction as Abstract and Results"),
       (ORANGE, "reverse direction"),
       (SEC, "depends on a sign convention the paper does not state")]
for k, (c, t) in enumerate(leg):
    yy = -0.95 - 0.42 * k
    ax.plot([0.15], [yy], marker="s", ms=9, color=c, ls="none")
    ax.text(0.32, yy, t, va="center", fontsize=10, color=TXT)
foot = ("* Lesion-deficit reading: a region that “facilitates” donations "
        "should lower them when damaged.\n"
        "† The colour bar, labelled “Beta”, runs from red (+10) to blue "
        "(−10). The Methods\n   define the sign as a positive or negative "
        "correlation with lesion presence;\n   the legend says red = decreased.")
ax.text(0.08, -2.25, foot, fontsize=10, color=SEC, va="top", linespacing=1.35)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.01)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)

Abstract reports 18 PDX models for the co-clinical trial; the paper's data contain 15

Cancer Res (2022) · doi:10.1158/0008-5472.CAN-22-0198 · PMC9478530

Summary

The abstract says the combination of MEK and CDK4/6 inhibition was tested "across 18 PDX models" and that 60% of tumors regressed, meeting the co-clinical trial's primary endpoint. Every other count in the paper gives 15 models. The Results subgroups add up to 6 + 2 + 3 + 4 = 15, the Results report "9 of 15 models (60%)", the Fig. 1 legends say 15 models, and Supplementary Tables S1 and S3 each list the same 15 model IDs. Every model used in the later experiments is one of those 15. Of the two counts, only 15 gives exactly 60% (9/15); with 18 the nearest values are 55.6% (10/18) and 61.1% (11/18), 60% of 18 is 10.8 models, and the 9 responders reported in the Results give 50%. The response rate and the "endpoint met" conclusion hold at 9 of 15. What the abstract gets wrong is the size of the evidence base: it overstates it by 3 models (20%), and it sits in the place readers and meta-analysts most often take sample sizes from.

What the paper reports

Abstract:

Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed, meeting the co-clinical trial primary endpoint.

Results, "Combined inhibition of MEK and CDK4/6 in colorectal cancer PDX models", first paragraph:

we evaluated typical RAS models that include a mixture of exon 2 KRASG12/13 (n = 6) P-loop domain mutated PDX models, extended RAS models that include NRASQ61 switch II domain mutated tumors (n = 2) along with KRASA146T nucleotide exchange promoting domain mutated tumors (n = 3; refs. 23, 24), and wild-type RAS tumors (n = 4).

Results, same section, second paragraph:

The PDX co-clinical trial of MEK (trametinib) and CDK4/6 (palbociclib) inhibition met its prespecified primary endpoint of tumor regression of ≥30% in more than 30% of the models. As shown in Fig. 1B and C, the combination of MEK and CDK4/6 inhibition produced ≥30% regression in 9 of 15 models (60%) with significantly greater tumor growth inhibition when compared with single agent–treated PDXs.

Figure 1 legend, panels A to D:

Six PDX models harboring codon 12 or 13 (“typical” RAS), 5 with codons 61 or 149 mutations (“extended” RAS), and 4 wild-type RAS PDX models were analyzed by RPPA

Trametinib (0.25 mpk, daily; green), palbociclib (75 mpk, daily; blue), and their combination (red) were evaluated in 15 mCRC PDX models.

C, Waterfall representation of 15 PDX models’ responses, per PDX model results.

D, Effect of treatments on mice body weight. Right, Tukey plot, after 21 days of treatment. Summarized data for 15 PDX models.

Results, "Mechanisms of acquired resistance":

Intriguingly, elevated phospho-SHP2 level was detected in 11 of 15 PDX models treated with the combination

Materials and Methods, "In vivo studies and PDXs":

PDX model details are provided in Supplementary Table S1.

Discussion, first paragraph:

The co-clinical trial met its primary endpoint of demonstrating tumor regressions in greater than 30% of models

What the paper's own numbers show

Method. I added up the RAS subgroup counts given in the Results text and in the Fig. 1A legend. I then built the list of PDX model IDs from Supplementary Table S1 (sheet "PDX details"), Supplementary Table S3 (the ranking of combination responses, grouped by RAS status), and the Fig. 1B panel titles. I added every model named for the follow-on experiments: the dose study (Fig. 2, Supp. Table S6), the durability cohort (Fig. 4, Supp. Table S7), the SHP2 triple combination (Fig. 5) and Supp. Fig. S2. Last, I checked which denominators can produce the abstract's 60%. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
PDX models given MEK + CDK4/6 inhibition Abstract 18 15; no set of 18 models appears anywhere in the paper or supplement
RAS subgroups, text Results, paragraph 1 n = 6, 2, 3, 4 6 + 2 + 3 + 4 = 15
RAS subgroups, figure legend Fig. 1A legend 6 typical, 5 extended, 4 wild type 6 + 5 + 4 = 15; the text's 2 + 3 = 5 extended models match
Models in the co-clinical trial Results, paragraph 2; Fig. 1B–D legends "9 of 15 models"; "15 mCRC PDX models" 15
Model panels Fig. 1B – 15
Model rows Supp. Table S1 ("PDX details") – 15 unique IDs
Model rows by RAS group Supp. Table S3 – 15 (6 typical, 5 extended, 4 wild type); same IDs as S1 and Fig. 1B
Models in follow-on experiments Figs. 2, 4, 5; Supp. Fig. S2; Supp. Tables S6, S7 3, 8, 2 and 2 models 9 distinct models, 0 outside the 15
All distinct models named anywhere Paper and supplement – 15, so 18 − 15 = 3 models are unaccounted for
Durability cohort Results; Fig. 4 "eight PDX models (4 of each RASmut subtype)" 8, all among the 15 (4 typical, 4 extended)
Regression rate, 15 models Abstract; Results 60%; "9 of 15 models (60%)" 9/15 = 60.0%
Same 9 responders over 18 models – – 9/18 = 50.0%
Responders implied by 60% of 18 Abstract 60% 0.6 × 18 = 10.8, not a whole number; 10/18 = 55.6%, 11/18 = 61.1%
Values of k for which k/n rounds to 60% – – n = 15: k = 9 only; n = 18: none
Primary endpoint (more than 30% of models with ≥30% regression) Results met 9/15 = 60% > 30%. Exact one-sided binomial test against 30% (my calculation; the paper reports no test): P = 0.015. Clopper–Pearson 95% CI 32.3% to 83.7%, one-sided 95% lower bound 36.0%

Figure

The abstract's "18 PDX models" set against the counts of 15 in the Results, the Fig. 1 legends and Supplementary Tables S1 and S3

Figure 1. Two text panels hold short excerpts from the article as archived in PMC. The left panel (orange, the value reported in the abstract) gives the abstract's sentence with its count of 18 models and its 60% response rate. The right panel (blue, counts found in or recomputed from the paper's own body and supplement) gives the Results' "9 of 15 models (60%)", the Results' subgroup counts, which add up to 15, the Fig. 1B legend's "15 mCRC PDX models", and the 15 rows in each of Supplementary Tables S1 and S3. The connector names the conflict: no set of 18 models is described anywhere in the paper, and of the two counts only 15 gives exactly 60% (9/15). The arithmetic is in recompute.py.txt.

Every attainable response rate k/n for n = 15 and n = 18, with the reported 60% marked

Figure 2. Each gray dot is a response rate that can occur with n models (k/n, k = 0 to n), on a full 0–100% axis. The orange vertical line marks the 60% reported in both the abstract and the Results. The dotted line marks the primary endpoint's 30% threshold. With n = 15, 60% falls exactly on 9/15 (filled blue). With n = 18, no attainable value equals 60%: the same 9 responders give 9/18 = 50.0% (filled blue), and the two nearest values are 10/18 = 55.6% and 11/18 = 61.1% (open blue). Of the two counts in the paper, the reported 60% therefore matches the 15-model denominator exactly, and not the 18 stated in the abstract.

Why it matters

The abstract is the only place in the paper that says 18, and the abstract is where readers, indexers and meta-analysts usually take the number of models. Anyone counting PDX models across preclinical studies of MEK plus CDK4/6 inhibition, or citing this study as an example of a co-clinical trial design, would record 3 models (20%) that the paper does not document.

The efficacy conclusion is unaffected. The correct figure is 9 of 15 models with at least 30% regression (60%), well above the prespecified 30% threshold. The exact 95% interval for 9/15 runs from 32.3% to 83.7%. A reader who combined the abstract's 18 with the 9 responders given in the Results would get 50%, which still exceeds the 30% threshold but does not match the abstract's own 60%. The subgroup comparisons (typical vs. extended RAS, TP53, CMS) already rest on small groups, and the 15-model count is the right basis for judging them.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the abstract, replace "Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed" with "Across 15 PDX models undergoing dual inhibition of MEK and CDK4/6, 9 (60%) regressed". This brings the abstract in line with the Results, the Fig. 1 legends and Supplementary Tables S1 and S3. The journal could also check the Fig. 1A legend's "codons 61 or 149" and the Results' "KRASA146T (n = 3)" against the Fig. 1B panel labels, which show two A146T models and one A146V model.

Recomputation script (python recompute.py.txt)
"""Recompute the PDX model count behind the abstract of PMC9478530
(Cancer Res 2022;82(18):3335, doi:10.1158/0008-5472.CAN-22-0198).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs (A-E) to ./candidates/.

Inputs are copied from the article as archived in PubMed Central and from its
supplementary files (Europe PMC supplementaryFiles package):
  Abstract:                 "Across 18 PDX models ... 60% of tumors regressed"
  Results, first section:   subgroup counts n = 6 (KRAS G12/13), 2 (NRAS Q61),
                            3 (KRAS A146T), 4 (RAS wild type); "9 of 15 models (60%)"
  Fig. 1 legend:            6 typical + 5 extended + 4 wild type; "15 mCRC PDX models"
  Supplementary Table S1:   sheet "PDX details", one row per model (model IDs only used here)
  Supplementary Table S3:   combination-response ranking, one row per model, by RAS group
  Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6 and S7: models named for the follow-on experiments
Model IDs are laboratory PDX codes, not patient identifiers.
"""
import os

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402
from scipy import stats  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_N = 18            # Abstract: "Across 18 PDX models"
ABSTRACT_PCT = 60          # Abstract: "60% of tumors regressed"
RESULTS_RESPONDERS = 9     # Results: "≥30% regression in 9 of 15 models (60%)"
RESULTS_N = 15
RESULTS_PCT = 60
ENDPOINT_THRESHOLD = 0.30  # Results: primary endpoint "≥30% in more than 30% of the models"
SUBGROUPS_TEXT = {"KRAS G12/13 (typical)": 6, "NRAS Q61 (extended)": 2,
                  "KRAS A146T (extended)": 3, "RAS wild type": 4}       # Results paragraph 1
SUBGROUPS_FIG1A = {"typical": 6, "extended": 5, "wild type": 4}          # Fig. 1A legend
FIG1_LEGEND_N = 15         # Fig. 1B, 1C and 1D legends: "15 mCRC PDX models" / "15 PDX models"
SHP2_OF = 15               # Results: "elevated phospho-SHP2 ... in 11 of 15 PDX models treated with the combination"

S3 = {  # Supplementary Table S3, model -> RAS group
    "B1008": "typical", "C1035": "typical", "C1047": "typical",
    "C1138": "typical", "C1177": "typical", "F3008": "typical",
    "B1011": "extended", "B8093": "extended", "C0997": "extended",
    "C1012": "extended", "C1151": "extended",
    "B1025": "wild type", "B8178": "wild type", "C1023": "wild type", "C1174": "wild type",
}
S1 = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
      "C1151", "C1012", "C1023", "B1025", "C1174", "B8178", "B1011"]   # Supp. Table S1, "PDX details"
FIG1B = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
         "C1151", "B1011", "C1012", "C1023", "B1025", "C1174", "B8178"]  # Fig. 1B panel titles
FIG2 = ["C1138", "B1011", "C1174"]                                     # Fig. 2 / Supp. Table S6 (dose study)
FIG4 = ["C1138", "C1047", "B8093", "B1011", "B1008", "F3008", "C1012", "C0997"]  # Fig. 4 / Supp. Table S7 (TP53 status, durability cohort)
FIG5 = ["B1008", "F3008"]                                              # Fig. 5 (SHP2 triple)
SUPP_S2 = ["F3008", "B1011"]                                           # Supp. Fig. S2

# ---------------------------------------------------------------- recomputation
def line(label, value):
    print(f"{label:<62} {value}")


print("== Model counts")
sum_text = sum(SUBGROUPS_TEXT.values())
sum_fig1a = sum(SUBGROUPS_FIG1A.values())
line("Results subgroups 6 + 2 + 3 + 4", sum_text)
line("Fig. 1A legend 6 + 5 + 4", sum_fig1a)
line("Extended group in text (NRAS Q61 + KRAS A146T) 2 + 3", 2 + 3)
s3_groups = {g: sum(v == g for v in S3.values()) for g in ("typical", "extended", "wild type")}
line("Supp. Table S3 models", f"{len(S3)}  {s3_groups}")
line("Supp. Table S1 models (unique)", len(set(S1)))
line("Fig. 1B model panels (unique)", len(set(FIG1B)))
line("S1, S3 and Fig. 1B list the same models", set(S1) == set(S3) == set(FIG1B))
followon = set(FIG2) | set(FIG4) | set(FIG5) | set(SUPP_S2)
line("Models in Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6, S7 (unique)", len(followon))
line("... of which outside the 15", len(followon - set(S3)))
union = set(S3) | set(S1) | set(FIG1B) | followon
line("Union of all models named in paper and supplement", len(union))
line("Abstract count minus union", ABSTRACT_N - len(union))
line("Abstract count relative to 15", f"{ABSTRACT_N / len(union):.2f} x  (+{ABSTRACT_N / len(union) - 1:.0%})")
fig4_groups = {g: sum(S3[m] == g for m in FIG4) for g in ("typical", "extended", "wild type")}
line("Fig. 4 durability cohort by group", fig4_groups)

print("\n== Response rate")
r15 = RESULTS_RESPONDERS / RESULTS_N
r18 = RESULTS_RESPONDERS / ABSTRACT_N
line("9/15", f"{r15:.1%}")
line("9/18", f"{r18:.1%}")
line("60% of 15", f"{0.60 * RESULTS_N:.1f}")
line("60% of 18", f"{0.60 * ABSTRACT_N:.1f}  (not a whole number)")
for k in range(10, 13):
    line(f"{k}/18", f"{k / 18:.1%}")
hits15 = [k for k in range(16) if round(100 * k / 15) == 60]
hits18 = [k for k in range(19) if round(100 * k / 18) == 60]
line("k with k/15 rounding to 60%", hits15)
line("k with k/18 rounding to 60%", hits18 if hits18 else "none")
line("(for contrast) n <= 20 with some k/n exactly 60%",
     [n for n in range(1, 21) if any(100 * k == 60 * n for k in range(n + 1))])

print("\n== Primary endpoint (>30% of models with >=30% regression)")
p15 = stats.binom.sf(RESULTS_RESPONDERS - 1, RESULTS_N, ENDPOINT_THRESHOLD)
lb15 = stats.beta.ppf(0.05, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
lo15 = stats.beta.ppf(0.025, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
hi15 = stats.beta.ppf(0.975, RESULTS_RESPONDERS + 1, RESULTS_N - RESULTS_RESPONDERS)
line("Exact one-sided binomial P(X >= 9 | n=15, p=0.30)", f"{p15:.4f}")
line("Clopper-Pearson one-sided 95% lower bound, 9/15", f"{lb15:.1%}")
line("Clopper-Pearson two-sided 95% CI, 9/15", f"{lo15:.1%} to {hi15:.1%}")
p18 = stats.binom.sf(RESULTS_RESPONDERS - 1, ABSTRACT_N, ENDPOINT_THRESHOLD)
line("(for contrast) P(X >= 9 | n=18, p=0.30)", f"{p18:.4f}")
for k in (10, 11):
    lo = stats.beta.ppf(0.025, k, ABSTRACT_N - k + 1)
    hi = stats.beta.ppf(0.975, k + 1, ABSTRACT_N - k)
    line(f"(for contrast) Clopper-Pearson two-sided 95% CI, {k}/18", f"{lo:.1%} to {hi:.1%}")

# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e6e5e1"
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "path", "lines.linewidth": 1.5,
    "figure.dpi": 200, "text.hinting": "none", "text.hinting_factor": 1,
})


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def runs(ax, x, y, parts, size=10.5, ha_box=None):
    """Draw text runs [(text, color, weight)] left to right starting at axes coords (x, y)."""
    fig = ax.figure
    r = fig.canvas.get_renderer()
    axw = ax.get_window_extent(renderer=r).width

    def width(s, weight):
        t = ax.text(0, 0, s, fontsize=size, fontweight=weight, transform=ax.transAxes)
        w = t.get_window_extent(renderer=r).width
        t.remove()
        return w / axw

    for text, color, weight in parts:
        core = text.lstrip(" ")
        x += width("|" + text[:len(text) - len(core)] + "|", "normal") - width("||", "normal")
        ax.text(x, y, core, color=color, fontsize=size, fontweight=weight,
                transform=ax.transAxes, va="baseline", ha="left")
        # advance by the run's width, with the bounding-box padding cancelled out
        x += width("|" + core + "|", weight) - width("||", weight)
    return x


def box(ax, x, y, w, h, edge, lw=1.5, ls="-", fill=BG):
    p = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                       transform=ax.transAxes, fc=fill, ec=edge, lw=lw, ls=ls)
    ax.add_patch(p)


def blank(h):
    fig = plt.figure(figsize=(W, h))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_axis_off()
    return fig, ax


# ---------------------------------------------------------------- A: two text panels
def cand_A(path):
    fig, ax = blank(3.9)
    ax.text(0.03, 0.93, "The abstract's model count disagrees with every count in the body",
            fontsize=12, fontweight="bold", transform=ax.transAxes)
    # left panel: abstract
    box(ax, 0.03, 0.12, 0.38, 0.70, ORANGE)
    ax.text(0.05, 0.755, "ABSTRACT", fontsize=10, color=ORANGE, fontweight="bold",
            transform=ax.transAxes)
    ax.text(0.05, 0.695, "(reported)", fontsize=10, color=ORANGE, transform=ax.transAxes)
    runs(ax, 0.05, 0.62, [("“Across ", INK, "normal"), ("18", ORANGE, "bold"),
                          (" PDX models", INK, "normal")])
    runs(ax, 0.05, 0.54, [("undergoing dual inhibition of", INK, "normal")])
    runs(ax, 0.05, 0.46, [("MEK and CDK4/6, ", INK, "normal"), ("60%", ORANGE, "bold"),
                          (" of tumors", INK, "normal")])
    runs(ax, 0.05, 0.38, [("regressed…”", INK, "normal")])
    ax.text(0.05, 0.20, f"With 18 models: 9/18 = {r18:.0%};\n60% of 18 = {0.6 * ABSTRACT_N:.1f} models",
            fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.4)
    # right panel: body
    box(ax, 0.59, 0.12, 0.38, 0.70, BLUE)
    ax.text(0.61, 0.755, "BODY & SUPPLEMENT", fontsize=10, color=BLUE, fontweight="bold",
            transform=ax.transAxes)
    ax.text(0.61, 0.695, "(reported / recomputed)", fontsize=10, color=BLUE, transform=ax.transAxes)
    runs(ax, 0.61, 0.62, [("“≥30% regression in ", INK, "normal"), ("9 of 15", BLUE, "bold")])
    runs(ax, 0.61, 0.54, [("models (60%)”", INK, "normal"), ("  Results", INK2, "normal")])
    runs(ax, 0.61, 0.45, [("n = 6 + 2 + 3 + 4 = ", INK, "normal"), (f"{sum_text}", BLUE, "bold"),
                          ("  Results", INK2, "normal")])
    runs(ax, 0.61, 0.36, [("“evaluated in ", INK, "normal"), ("15", BLUE, "bold"),
                          (" mCRC PDX", INK, "normal")])
    runs(ax, 0.61, 0.28, [("models”", INK, "normal"), ("  Fig. 1B legend", INK2, "normal")])
    runs(ax, 0.61, 0.19, [("Supp. Tables S1, S3: ", INK, "normal"), (f"{len(S3)}", BLUE, "bold"),
                          (" rows each", INK, "normal")])
    # connector
    ax.annotate("", xy=(0.585, 0.47), xytext=(0.415, 0.47), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
    ax.text(0.50, 0.55, "18 ≠ 15", ha="center", fontsize=12, fontweight="bold",
            transform=ax.transAxes)
    ax.text(0.50, 0.33, "no set of 18\nmodels is\ndescribed; of\nthe two counts,\nonly 9/15 = 60%", ha="center",
            va="center", fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.3)
    ax.text(0.03, 0.04, "Excerpts from the PMC-archived article and its supplement; "
            "arithmetic from recompute.py.txt.", fontsize=10, color=INK2, transform=ax.transAxes)
    save(fig, path)


# ---------------------------------------------------------------- B: count by location dot plot
def cand_B(path):
    rows = [
        ("Abstract: “Across 18 PDX models”", ABSTRACT_N, "abs"),
        ("Results: subgroups 6 + 2 + 3 + 4", sum_text, "body"),
        ("Results: “9 of 15 models (60%)”", RESULTS_N, "body"),
        ("Fig. 1A legend: 6 + 5 + 4", sum_fig1a, "body"),
        ("Fig. 1B–D legends: “15 … PDX models”", FIG1_LEGEND_N, "body"),
        ("Fig. 1B: model panels", len(set(FIG1B)), "body"),
        ("Results: phospho-SHP2 “in 11 of 15”", SHP2_OF, "body"),
        ("Supp. Table S1: model rows", len(set(S1)), "body"),
        ("Supp. Table S3: model rows", len(S3), "body"),
        ("All models named anywhere", len(union), "body"),
    ]
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0.42, 0.12, 0.54, 0.70])
    ys = list(range(len(rows)))[::-1]
    for (lab, v, kind), y in zip(rows, ys):
        c = ORANGE if kind == "abs" else BLUE
        ax.plot([0, v], [y, y], color=GRID, lw=1.5, zorder=1)
        ax.plot(v, y, "o", ms=9, color=c, mec=BG, mew=1.5, zorder=3)
        ax.text(v + 0.5, y, f"{v}", va="center", fontsize=10.5, color=INK)
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=INK)
    ax.set_xlim(0, 20.5)
    ax.set_xticks(range(0, 21, 5))
    ax.set_xlabel("Number of PDX models")
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.axvspan(15, 18, color=ORANGE, alpha=0.08, lw=0)
    ax.set_ylim(-0.6, len(rows) - 1 + 1.3)
    ax.text(16.5, ys[0] + 0.75, "3 models\nnot found", ha="center", va="center", fontsize=10,
            color=INK2, linespacing=1.1)
    h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Abstract (reported)")
    h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Body and supplement (stated or counted)")
    fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.005, 0.93), ncol=2,
               frameon=False, fontsize=10)
    fig.text(0.01, 0.945, "Model count for the dual MEK + CDK4/6 co-clinical trial, by location",
             fontsize=11.5, fontweight="bold")
    save(fig, path)


# ---------------------------------------------------------------- C: percentage lattice
def cand_C(path):
    fig = plt.figure(figsize=(W, 3.3))
    ax = fig.add_axes([0.12, 0.2, 0.84, 0.56])
    for n, y in ((15, 1), (18, 0)):
        xs = [100 * k / n for k in range(n + 1)]
        ax.plot(xs, [y] * len(xs), "o", ms=5, color="#b9b8b3", zorder=2)
        ax.plot([0, 100], [y, y], color=GRID, lw=1.5, zorder=1)
    ax.axvline(100 * ENDPOINT_THRESHOLD, color=INK2, lw=0.8, ls=(0, (2, 2)), zorder=0)
    ax.text(100 * ENDPOINT_THRESHOLD + 0.8, 1.55, "endpoint threshold 30%", fontsize=10, color=INK2)
    ax.axvline(ABSTRACT_PCT, color=ORANGE, lw=1.5, zorder=1)
    ax.text(ABSTRACT_PCT + 0.8, 1.55, "Abstract and Results: 60%", fontsize=10, color=ORANGE,
            fontweight="bold")
    ax.plot(100 * r15, 1, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
    ax.annotate("9/15 = 60.0%", (100 * r15, 1), xytext=(100 * r15 + 4, 1.28), fontsize=10.5,
                color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    ax.plot(100 * r18, 0, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
    ax.annotate(f"9/18 = {r18:.1%}\n(same 9 responders)", (100 * r18, 0), xytext=(100 * r18 - 24, -0.62),
                fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    for k in (10, 11):
        ax.plot(100 * k / 18, 0, "o", ms=9, mfc=BG, mec=BLUE, mew=1.5, zorder=4)
    ax.annotate(f"10/18 = {10 / 18:.1%}", (100 * 10 / 18, 0), xytext=(38, 0.42), fontsize=10,
                color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    ax.annotate(f"11/18 = {11 / 18:.1%}\n60% of 18 = 10.8 models",
                (100 * 11 / 18, 0), xytext=(66, -0.62), fontsize=10, color=INK,
                arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    ax.set_yticks([1, 0])
    ax.set_yticklabels(["n = 15", "n = 18"], fontsize=10.5, color=INK)
    ax.set_ylim(-0.9, 1.85)
    ax.set_xlim(0, 100)
    ax.set_xticks(range(0, 101, 10))
    ax.set_xticklabels([f"{t}%" for t in range(0, 101, 10)])
    ax.set_xlabel("Share of models regressing ≥30% (every attainable value k/n shown in gray)")
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    fig.text(0.01, 0.92, "Exactly 60% is attainable with 15 models (9/15), not with 18",
             fontsize=11.5, fontweight="bold")
    save(fig, path)


# ---------------------------------------------------------------- D: unit chart of models
def cand_D(path):
    fig, ax = blank(4.0)
    ax.text(0.02, 0.93, "The abstract says 18 models; the paper names 15", fontsize=12,
            fontweight="bold", transform=ax.transAxes)
    groups = [("Typical RAS\n(KRAS G12/13)", [m for m in S3 if S3[m] == "typical"]),
              ("Extended RAS\n(NRAS Q61, KRAS A146T)", [m for m in S3 if S3[m] == "extended"]),
              ("RAS wild type", [m for m in S3 if S3[m] == "wild type"]),
              ("Abstract's extra\nmodels", [None, None, None])]
    bw, bh, gap, x0 = 0.092, 0.13, 0.011, 0.29
    for i, (lab, models) in enumerate(groups):
        y = 0.70 - i * 0.175
        ax.text(0.02, y + bh / 2, lab, va="center", fontsize=10, color=INK if models[0] else ORANGE,
                transform=ax.transAxes, linespacing=1.2)
        for j, m in enumerate(models):
            x = x0 + j * (bw + gap)
            if m:
                box(ax, x, y, bw, bh, BLUE, fill="#e9f1fb")
                ax.text(x + bw / 2, y + bh / 2, m, ha="center", va="center", fontsize=10,
                        transform=ax.transAxes)
            else:
                box(ax, x, y, bw, bh, ORANGE, ls=(0, (3, 2)))
                ax.text(x + bw / 2, y + bh / 2, "?", ha="center", va="center", fontsize=11,
                        color=ORANGE, transform=ax.transAxes)
        ax.text(x0 + len(models) * (bw + gap) + 0.005, y + bh / 2,
                f"{len(models)}" if models[0] else "3 (none named)", va="center", fontsize=10.5,
                fontweight="bold", color=BLUE if models[0] else ORANGE, transform=ax.transAxes)
    ax.text(0.02, 0.025, f"Blue: models listed in Supp. Table S3 (6 + 5 + 4 = {len(S3)}); "
            "the same 15 appear in Supp. Table S1\nand Fig. 1B. Orange dashed: the 3 further models "
            "the abstract's “18” implies;\nnone is described in the paper or supplement.",
            fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.35)
    save(fig, path)


# ---------------------------------------------------------------- E: calculation flow
def cand_E(path):
    fig, ax = blank(3.6)
    ax.text(0.02, 0.93, "Following the paper's own counts to the abstract's 60%", fontsize=12,
            fontweight="bold", transform=ax.transAxes)

    def node(x, y, w, h, txt, edge, bold=False, fill=BG, ls="-"):
        box(ax, x, y, w, h, edge, fill=fill, ls=ls)
        ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10,
                fontweight="bold" if bold else "normal", transform=ax.transAxes, linespacing=1.3)

    def arrow(x1, y1, x2, y2):
        ax.annotate("", xy=(x2, y2), xytext=(x1, y1), xycoords="axes fraction",
                    arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.5, mutation_scale=12))

    ax.text(0.02, 0.80, "From the Results and Fig. 1 (recomputed)", fontsize=10, color=BLUE,
            fontweight="bold", transform=ax.transAxes)
    box(ax, 0.02, 0.48, 0.25, 0.26, BLUE)
    sub = "\n".join(f"{k:<12}{v:>2}" for k, v in
                    zip(("KRAS G12/13", "NRAS Q61", "KRAS A146T", "RAS WT"), SUBGROUPS_TEXT.values()))
    ax.text(0.145, 0.61, sub, ha="center", va="center", fontsize=10, family="DejaVu Sans Mono",
            transform=ax.transAxes, linespacing=1.3)
    arrow(0.27, 0.61, 0.32, 0.61)
    node(0.32, 0.52, 0.15, 0.18, f"{sum_text} models", BLUE, bold=True)
    arrow(0.47, 0.61, 0.52, 0.61)
    node(0.52, 0.52, 0.17, 0.18, f"{RESULTS_RESPONDERS} regressed\n≥30%", BLUE)
    arrow(0.69, 0.61, 0.74, 0.61)
    node(0.74, 0.52, 0.23, 0.18, f"9/15 = {r15:.1%}\nmatches 60%", BLUE, bold=True)

    ax.text(0.02, 0.38, "From the abstract (reported)", fontsize=10, color=ORANGE, fontweight="bold",
            transform=ax.transAxes)
    node(0.32, 0.10, 0.15, 0.18, f"{ABSTRACT_N} models", ORANGE, bold=True)
    arrow(0.47, 0.22, 0.52, 0.255)
    arrow(0.47, 0.15, 0.52, 0.10)
    node(0.52, 0.20, 0.45, 0.11, f"same 9 responders: 9/18 = {r18:.1%}, not 60%", ORANGE)
    node(0.52, 0.02, 0.45, 0.15, f"keep 60%: 0.6 × 18 = {0.6 * ABSTRACT_N:.1f} models\n(not a whole number)",
         ORANGE)
    save(fig, path)


cand_A(os.path.join(CAND, "A_text_panels"))
cand_B(os.path.join(CAND, "B_count_by_location"))
cand_C(os.path.join(CAND, "C_percentage_lattice"))
cand_D(os.path.join(CAND, "D_model_grid"))
cand_E(os.path.join(CAND, "E_calculation_flow"))
# Provisional choice for the write-up, pending review of the candidates:
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)

Fisher exact P reported as 0.0005 for 1/6 versus 7/18 remissions; the counts give 0.62

Clin Cancer Res (2012) · doi:10.1158/1078-0432.CCR-12-2442 · PMC3596113

Summary

The Discussion of this phase I trial compares the CR/CRi rate (complete remission plus complete remission with incomplete recovery) between dose groups of SCH 900776: 1 of 6 patients (17%) below 40 mg/m² and 7 of 18 (39%) at 40 mg/m² or above, reported as "P = 0.0005, Fisher exact test". The counts agree with Table 3, Table 4, the Results text and the abstract. A Fisher exact test on them gives a two-sided P of 0.62 (one-sided 0.32), and with 6 and 18 patients and 8 responders no table in the reported direction can reach a P below 0.066. This is the only P value in the paper, and the Discussion uses it as statistical support for a dose-response that would point to a contribution of SCH 900776 itself; the trial's toxicity-based dose-finding conclusions do not depend on it.

What the paper reports

The comparison and its P value (Discussion, paragraph 1):

Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.0005, Fisher exact test).

The sentence immediately before it, which states the claim the P value supports (Discussion, paragraph 1; the PMC text prints "doseresponse", the publisher's HTML "dose-response"):

The notion that part of the responses might relate to adding SCH 900776 to the timed sequential cytarabine infusions is supported by the presence of a doseresponse curve, with clinical responses to SCH 900776 beginning at 40 mg/m2, in conjunction with Cmax exceeding 1,000 ng/mL.

The same counts, without a P value (Results, "Clinical outcome"):

Similarly, the overall response rate (CR plus CRi) was 8 of 24 (33%), with 1 CRi (17%) of 6 patients treated with SCH 900776 doses of ≤ 20 mg/m2 and 7 (39%; 5 CR, 2 CRi) of 18 patients treated at dose level 3 (40 mg/m2) or higher, with at least 2 of 6 (33%) in each of dose levels 3 through 5.

Abstract, Results:

Complete remissions occurred in 8 of 24 (33%) patients, with 7 of 8 at 40 mg/m2 or higher.

Table 3 ("Dose escalation and clinical outcome for 24 adults with relapsed or refractory acute leukemia"), dose-level labels and the "Response (CR/CRi)" column, condensed to one line with the archived typography kept:

  1. 10 mg/m2 (n = 3): 1 · 2. 20 mg/m2 (n = 3): 0 · 3. 40 (mg/m2 (n = 6): 3 · 4. 56 (mg/m2 (n = 6): 2 · 5. 140 mg (n = 6): 2 · Total: 8

Table 4 lists the 8 responders individually, at dose levels 1, 3, 3, 3, 4, 4, 5 and 5.

The authors' caveats, later in the same paragraph (Discussion, paragraph 1):

we cannot exclude the possibility that some or all of the responders might have responded to timed sequential cytarabine without SCH 900776

Moreover, the small sample size and heterogeneous nature of this patient population preclude any ability to draw accurate conclusions about additive or synergistic clinical effects of adding SCH 900776 to cytarabine

The statistics subsection of the Methods names no hypothesis test (Methods, "Statistical considerations"):

The primary objective was to determine the combination-recommended phase II dose (RP2D) by assessing the safety, tolerability, DLT, and antileukemic activity (CR rate) of the combination of cytarabine and SCH 900776 in adult patients with relapsed and/or refractory acute leukemias.

What the paper's own numbers show

Method. I rebuilt the 2×2 table from Table 3. Dose levels 1–2 (10 and 20 mg/m², 3 patients each) form the group below 40 mg/m²: 1 + 0 = 1 responder of 6. Dose levels 3–5 (40 mg/m², 56 mg/m² and a flat 140 mg that the paper states is equivalent to 80 mg/m²; 6 patients each) form the group at or above 40 mg/m²: 3 + 2 + 2 = 7 responders of 18. Table 4's responder dose levels and the abstract's "7 of 8" give the same split. The table is therefore [[1 responder, 5 non-responders], [7, 11]]. I ran Fisher's exact test with scipy and cross-checked it by enumerating the hypergeometric distribution directly. I also ran the usual alternatives for a 2×2 table, two tests across all five dose levels, and enumerated every table with the same margins (6 vs 18 patients, 8 responders). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
CR/CRi below 40 mg/m² Discussion ¶1; Results; Table 3 1 of 6 (17%) 1/6 = 16.7% (Table 3: 1 + 0 of 3 + 3)
CR/CRi at or above 40 mg/m² Discussion ¶1; Results; Table 3; abstract 7 of 18 (39%); "7 of 8" 7/18 = 38.9% (Table 3: 3 + 2 + 2 of 6 + 6 + 6); Table 4: 7 of 8 responders at level ≥ 3
Fisher exact test, below vs at or above 40 mg/m² Discussion ¶1 P = 0.0005 two-sided P = 0.6214, 1,243 times larger
Fisher exact, one-sided (fewer responses below 40 mg/m²) – – P = 0.3191
Odds ratio, ≥ 40 vs < 40 mg/m² – – sample 3.18; conditional MLE 3.05, exact 95% CI 0.25 to 171.6
Exact 95% CIs for the two response rates – – 16.7% (0.4% to 64.1%) and 38.9% (17.3% to 64.3%)
Pearson chi-square / Yates-corrected chi-square – – P = 0.3173 / 0.6171
Barnard / Boschloo exact unconditional tests – – P = 0.4090 / 0.3761
Cochran-Armitage trend over the 5 dose levels – – P = 0.7423 (scores 1–5); 0.7159 (mg/m²); 0.6541 (log dose)
Fisher-Freeman-Halton exact test, 2×5 dose levels – – P = 0.8265
Smallest two-sided Fisher P in the reported direction, same margins – – P = 0.0664, at 0/6 vs 8/18
Tables with these margins giving P ≤ 0.0005 – – only 6/6 vs 2/18 (P = 0.00021), the reverse of the reported pattern
Responders needed at ≥ 40 mg/m² for P ≤ 0.0005 if the low-dose group stays at 1/6 – – 18 of 18 (P = 0.00014)

The arithmetic of the counts and percentages is correct; only the P value is not. It is 1,243 times smaller than the two-sided Fisher P for the stated counts and lies below the smallest value that any table with these margins can produce in the reported direction (0.066). With 8 responders among 24 patients, 2.0 responders would be expected in the low-dose group if response did not depend on dose; 1 was observed.

Figure

Reported P value compared with P values recomputed from the Table 3 counts, on one log axis

Figure 1. P values for CR/CRi below vs at or above 40 mg/m² (1/6 vs 7/18, Table 3) on one logarithmic axis. The orange point is the value printed in the Discussion (P = 0.0005, Fisher exact test). The blue points are recomputed from the paper's own counts: the stated test (Fisher exact, two-sided, and one-sided in the reported direction), common alternatives for a 2×2 table (Pearson and Yates chi-square, Barnard and Boschloo exact tests), and two tests across the five dose levels (Cochran-Armitage trend, Fisher-Freeman-Halton). The dashed line marks P = 0.05 for reference. The shaded region (P < 0.066) contains P values that no 2×2 table with the paper's margins can produce in the reported direction. All recomputed values lie between 0.32 and 0.83.

Two-sided Fisher P for every possible split of the 8 responders between the 6 low-dose and 18 high-dose patients

Figure 2. With the margins fixed by the paper (6 patients below 40 mg/m², 18 at or above, 8 responders), only seven 2×2 tables are possible. Each is labelled on the horizontal axis by its responder counts, below 40 mg/m² on the top line and at or above 40 mg/m² on the bottom line. Blue markers give the two-sided Fisher P for each table; the filled marker is the observed table (P = 0.62). The orange line is the reported P = 0.0005. Only the table in which all 6 low-dose patients respond, the opposite of the reported pattern, falls below it. The most extreme table in the reported direction (0/6 vs 8/18) gives P = 0.066.

Why it matters

The P value is the only formal statistical result in the paper. The Discussion presents it as evidence for a dose-response, and uses the dose-response to support "the notion that part of the responses might relate to adding SCH 900776" to cytarabine. As printed, it reads as strong evidence that response rates differ between the dose groups. Computed from the paper's own counts, the two-group comparison gives P = 0.62, the trend test across all five dose levels gives P = 0.65 to 0.74 depending on scoring, and the exact 95% CI for the odds ratio runs from 0.25 to 172. With 6 patients in the lower-dose group, the data are consistent both with no dose effect and with a large one; they do not establish a dose-response either way.

The error is limited in scope. It is not in the title, the abstract or any table. The abstract gives only correct counts ("8 of 24 (33%)", "7 of 8 at 40 mg/m2 or higher"), although its emphasis on the 7 of 8 carries the same dose-response reading, now without statistical support. The phase I conclusions (feasibility and tolerability, the dose-limiting toxicities at 140 mg, the recommended phase II dose of 100 mg, about 56 mg/m², the pharmacokinetics and the H2Ax observations) rest on other data and are unaffected. The raw counts are printed next to the P value, and the authors state in the same paragraph that the small, heterogeneous sample precludes firm conclusions about added effect. Even so, a reader who takes the P value at face value, for example when judging whether the dose-response signal justifies a randomized trial, or when citing the paper as evidence of activity attributable to the Chk1 inhibitor, would overestimate that signal considerably.

Three smaller inconsistencies, separate from the P value and not affecting any conclusion, turned up during the check:

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The sentence in Discussion paragraph 1 could read: "Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.62, two-sided Fisher exact test)." Because neither this comparison nor a trend test across the five dose levels (P ≈ 0.7) supports a dose-response at this sample size, the preceding sentence's statement that the responses are "supported by the presence of a dose-response curve" could be qualified as a descriptive observation. If the reported value came from a different comparison, the correction should say which one. Separately, "2 of 6 (17%)" for day-14 tumor clearance at ≤ 20 mg/m² should read "2 of 6 (33%)", and the CR/CRi breakdown at dose level 3 and the karyotype counts could be reconciled between Table 3, Table 4 and the Discussion.

Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC3596113 write-up and regenerate its figures.

Paper: "Phase I and Pharmacologic Trial of Cytosine Arabinoside with the Selective
Checkpoint 1 Inhibitor Sch 900776 in Refractory Acute Leukemias",
Clin Cancer Res 2012;18(24):6723-31, doi:10.1158/1078-0432.CCR-12-2442 (PMC3596113).

All inputs are copied from Table 3, Table 4 and the Results/Discussion text.
Run:  python recompute.py.txt      (needs numpy, scipy >= 1.7, matplotlib)
Writes fig1.svg/png and fig2.svg/png next to this file.
"""
import os
from math import comb
from itertools import product
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
# Table 3, per dose level: (level, dose in mg/m2, n, day-14 CTC, CR/CRi, CR, CRi).
# Level 5 was a flat 140 mg, which the paper states is "equivalent to 80 mg/m2".
TABLE3 = [
    (1, 10, 3, 1, 1, 0, 1),
    (2, 20, 3, 1, 0, 0, 0),
    (3, 40, 6, 4, 3, 2, 1),
    (4, 56, 6, 3, 2, 2, 0),
    (5, 80, 6, 3, 2, 1, 1),
]
# Table 4, one row per responder: (dose level, response)
TABLE4 = [(1, "CRi"), (3, "CR"), (3, "CR"), (3, "CR"), (4, "CR"), (4, "CR"), (5, "CRi"), (5, "CR")]
REPORTED_P = 0.0005   # Discussion, paragraph 1: "(P = 0.0005, Fisher exact test)"

lev = np.array([r[0] for r in TABLE3]); dose = np.array([r[1] for r in TABLE3], float)
ns = np.array([r[2] for r in TABLE3]); ctc = np.array([r[3] for r in TABLE3])
rs = np.array([r[4] for r in TABLE3])
cr3 = sum(r[5] for r in TABLE3); cri3 = sum(r[6] for r in TABLE3)


def fmt(p):
    return f"{p:.4f}" if p >= 1e-3 else f"{p:.5f}"


def section(t):
    print("\n" + "=" * 72 + "\n" + t + "\n" + "=" * 72)


# ------------------------------------------------- 1. counts
section("1. Counts from Table 3 (checked against Table 4 and the text)")
N, R = int(ns.sum()), int(rs.sum())
lo = dose < 40
lo_n, lo_r = int(ns[lo].sum()), int(rs[lo].sum())
hi_n, hi_r = int(ns[~lo].sum()), int(rs[~lo].sum())
print(f"Total CR/CRi: {R}/{N} = {R/N:.1%}")
print(f"<40 mg/m2 (levels 1-2):  {lo_r}/{lo_n} = {lo_r/lo_n:.1%}   (paper: '1 of 6 (17%)')")
print(f">=40 mg/m2 (levels 3-5): {hi_r}/{hi_n} = {hi_r/hi_n:.1%}  (paper: '7 of 18 (39%)')")
t4_levels = [lv for lv, _ in TABLE4]
print(f"Table 4 responder dose levels: {t4_levels}; below level 3: {sum(l < 3 for l in t4_levels)}, "
      f"level 3 or above: {sum(l >= 3 for l in t4_levels)}  (abstract: '7 of 8 at 40 mg/m2 or higher')")
tab = np.array([[lo_r, lo_n - lo_r], [hi_r, hi_n - hi_r]])
print("2x2 table [responders, non-responders], rows <40 / >=40:", tab.tolist())

# ------------------------------------------------- 2. the stated test
section("2. Fisher exact test of <40 vs >=40 mg/m2 (the test the paper names)")
or_lo_vs_hi, p2 = stats.fisher_exact(tab, alternative="two-sided")
p_less = stats.fisher_exact(tab, alternative="less")[1]
p_greater = stats.fisher_exact(tab, alternative="greater")[1]
print(f"Sample odds ratio (<40 vs >=40) = {or_lo_vs_hi:.3f}; (>=40 vs <40) = {1/or_lo_vs_hi:.2f}")
print(f"Two-sided P = {fmt(p2)}")
print(f"One-sided P, fewer responses below 40 mg/m2 (the reported direction) = {fmt(p_less)}")
print(f"One-sided P, opposite direction = {fmt(p_greater)}")
print(f"Reported P / recomputed two-sided P: {REPORTED_P} vs {p2:.4f} -> recomputed is {p2/REPORTED_P:,.0f} times larger")

# manual hypergeometric enumeration as a cross-check of scipy
pmf = {x: comb(R, x) * comb(N - R, lo_n - x) / comb(N, lo_n) for x in range(0, min(R, lo_n) + 1)}
p2_manual = sum(v for v in pmf.values() if v <= pmf[lo_r] * (1 + 1e-7))
print("Hypergeometric P(X = x responders below 40 mg/m2):", {k: round(v, 4) for k, v in pmf.items()})
print(f"Manual two-sided P = {fmt(p2_manual)}; manual P(X <= {lo_r}) = {fmt(sum(pmf[x] for x in range(lo_r + 1)))}")

ci_or = stats.contingency.odds_ratio(tab[::-1], kind="conditional")
lo_ci, hi_ci = ci_or.confidence_interval(0.95)
print(f"Conditional MLE odds ratio (>=40 vs <40) = {ci_or.statistic:.2f}, exact 95% CI {lo_ci:.2f} to {hi_ci:.1f}")
for lab, r, n in (("<40", lo_r, lo_n), (">=40", hi_r, hi_n)):
    ci = stats.binomtest(r, n).proportion_ci(0.95, method="exact")
    print(f"Clopper-Pearson 95% CI for CR/CRi rate {lab}: {r}/{n} = {r/n:.1%} ({ci.low:.1%} to {ci.high:.1%})")

# ------------------------------------------------- 3. other tests
section("3. Other common tests on the same 2x2 table")
results = {
    "Fisher exact, two-sided\n(the stated test)": p2,
    "Fisher exact, one-sided\n(reported direction)": p_less,
    "Pearson chi-square": stats.chi2_contingency(tab, correction=False)[1],
    "Chi-square, Yates": stats.chi2_contingency(tab, correction=True)[1],
    "Barnard exact": stats.barnard_exact(tab).pvalue,
    "Boschloo exact": stats.boschloo_exact(tab).pvalue,
}
for k in ("Pearson chi-square", "Chi-square, Yates", "Barnard exact", "Boschloo exact"):
    print(f"{k}: P = {fmt(results[k])}")

# ------------------------------------------------- 4. trend / all levels
section("4. Tests across the five dose levels")
def ca_trend(ns, rs, s):
    s = np.asarray(s, float); n_ = ns.sum(); pbar = rs.sum() / n_
    T = np.sum(s * (rs - ns * pbar))
    V = pbar * (1 - pbar) * (np.sum(ns * s**2) - np.sum(ns * s) ** 2 / n_)
    z = T / np.sqrt(V)
    return z, 2 * stats.norm.sf(abs(z))
ca = {}
for name, sc in (("level 1-5", lev), ("dose mg/m2", dose), ("log dose", np.log(dose))):
    z, p = ca_trend(ns, rs, sc); ca[name] = p
    print(f"Cochran-Armitage, scores = {name:10s}: z = {z:.3f}, two-sided P = {fmt(p)}")
results["Cochran-Armitage trend,\n5 dose levels"] = ca["level 1-5"]

def prob(rr):
    return np.prod([comb(int(n_), int(r_)) for r_, n_ in zip(rr, ns)]) / comb(N, R)
p_obs = prob(rs)
ffh = sum(prob(rr) for rr in product(*[range(int(n_) + 1) for n_ in ns])
          if sum(rr) == R and prob(rr) <= p_obs * (1 + 1e-9))
print(f"Fisher-Freeman-Halton exact test, 2x5 table: P = {fmt(ffh)}")
results["Freeman-Halton exact,\n2x5 dose levels"] = ffh

# ------------------------------------------------- 5. attainable P values
section("5. Every 2x2 table with the paper's margins (6 vs 18 patients, 8 responders)")
fisher_by_x = {}
for x in range(0, lo_n + 1):
    t = [[x, lo_n - x], [R - x, hi_n - (R - x)]]
    fisher_by_x[x] = stats.fisher_exact(t)[1]
    print(f"  {x}/6 below 40 vs {R-x}/18 at or above: two-sided Fisher P = {fmt(fisher_by_x[x])}")
expected_lo = R * lo_n / N
print(f"Expected responders below 40 mg/m2 under no difference: {expected_lo:.1f}")
p_min_dir = min(fisher_by_x[x] for x in range(0, lo_r + 1))
print(f"Smallest P in the reported direction (fewer responders below 40 mg/m2 than expected): {fmt(p_min_dir)} (at 0/6 vs 8/18)")
print(f"Smallest P for any table: {fmt(min(fisher_by_x.values()))} at 6/6 vs 2/18 (the reverse pattern)")
print(f"Tables with P <= 0.0005: {[f'{x}/6 vs {R-x}/18' for x, p in fisher_by_x.items() if p <= REPORTED_P]}")

print("\nKeeping 1/6 below 40 mg/m2 and 18 patients at or above (margins free):")
for k in range(hi_r, hi_n + 1):
    p = stats.fisher_exact([[1, 5], [k, hi_n - k]])[1]
    if p <= REPORTED_P:
        print(f"  first k with P <= 0.0005: {k}/18 ({k/18:.0%}), P = {fmt(p)}")
        break
print("Keeping 7/18 at or above 40 mg/m2: P for 0/6 below 40 =", fmt(stats.fisher_exact([[0, 6], [7, 11]])[1]))

# ------------------------------------------------- 6. alternative cut-points
section("6. Other dose cut-points (two-sided Fisher)")
for cut in (20, 56, 80):
    m = dose < cut
    t = [[int(rs[m].sum()), int(ns[m].sum() - rs[m].sum())], [int(rs[~m].sum()), int(ns[~m].sum() - rs[~m].sum())]]
    print(f"  <{cut} vs >={cut} mg/m2: {t[0][0]}/{sum(t[0])} vs {t[1][0]}/{sum(t[1])}: P = {fmt(stats.fisher_exact(t)[1])}")

# ------------------------------------------------- 7. other comparisons in the paper
section("7. Other comparisons in the paper (could 0.0005 belong to one of them?)")
other = {
    "CR/CRi with day-14 CTC (8/12) vs residual leukemia (0/12)": [[8, 4], [0, 12]],
    "Day-14 CTC <40 (2/6) vs >=40 (10/18)": [[2, 4], [10, 8]],
    "CR/CRi AML (8/21) vs other leukemias (0/3)": [[8, 13], [0, 3]],
    "CR/CRi complex karyotype (2/10) vs not (5/14)": [[2, 8], [5, 9]],
}
for k, t in other.items():
    p_two = stats.fisher_exact(t)[1]
    p_one = min(stats.fisher_exact(t, alternative=a)[1] for a in ("less", "greater"))
    print(f"  {k}: two-sided P = {fmt(p_two)}, smaller one-sided P = {fmt(p_one)}")
for p0 in (0.05, 0.10):
    print(f"  Binomial test of 7/18 against a {p0:.0%} response rate: one-sided P = "
          f"{fmt(stats.binomtest(7, 18, p0, alternative='greater').pvalue)}")

# ------------------------------------------------- 8. side observations
section("8. Side observations (minor, separate from the P value)")
c = int(ctc[lo].sum())
print(f"Day-14 CTC at <=20 mg/m2: {c}/{lo_n} = {c/lo_n:.1%}  (Results text: '2 of 6 (17%)')")
cr4 = sum(r == "CR" for _, r in TABLE4); cri4 = sum(r == "CRi" for _, r in TABLE4)
cr4_l3 = sum(1 for lv, r in TABLE4 if lv == 3 and r == "CR")
print(f"CR / CRi split: Table 3 total {cr3}/{cri3} (Results text: 1 CRi + 5 CR, 2 CRi = 5/3); "
      f"Table 4 rows {cr4}/{cri4}; level 3: Table 3 '3 (2/1)', Table 4 {cr4_l3} CR")
print(f"Complex karyotype: 2 of 10 + 5 of 14 = {2+5} responders among {10+14} patients (total responders {R}); "
      f"2/10 = {2/10:.0%}, 5/14 = {5/14:.0%}")
print(f"Total day-14 CTC: {ctc.sum()}/{N} = {ctc.sum()/N:.0%}")

# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e6e5e1"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
    "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
    "axes.edgecolor": "#bdbcb7", "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


def plabel(p):
    return f"P = {p:.4f}" if p < 0.01 else (f"P = {p:.3f}" if p < 0.1 else f"P = {p:.2f}")


# Figure 1: reported P vs P values recomputed from the paper's own counts, one log axis
rows = [("Reported in the paper\n(Fisher exact test)", REPORTED_P, ORANGE)]
rows += [(k, v, BLUE) for k, v in results.items()]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
ys = np.arange(len(rows))[::-1].astype(float)
ax.axvspan(1e-4, p_min_dir, color="#efeeea", zorder=0, lw=0)
ax.text(1.2e-4, ys[0] + 0.95, f"Shaded: P values no table with these margins\ncan reach in the reported direction (P < {p_min_dir:.3f})",
        fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(0.05, color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
ax.text(0.05 * 0.93, ys[-1] - 0.62, "P = 0.05", ha="right", va="center", color=INK2, fontsize=10)
for y, (lab, p, col) in zip(ys, rows):
    ax.plot([p], [y], "o", ms=9, color=col, mec=BG, mew=1.2, zorder=3)
    if p < 0.01:
        ax.text(p * 1.4, y, plabel(p), va="center", ha="left", fontsize=10)
    else:
        ax.text(p * 0.8, y, plabel(p), va="center", ha="right", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xscale("log"); ax.set_xlim(1e-4, 1.6)
ax.set_ylim(ys[-1] - 1.0, ys[0] + 1.6)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("P value (log scale)")
ax.set_title("CR/CRi below vs at or above 40 mg/m²: 1/6 vs 7/18 (Table 3)", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported (Discussion)"),
                   Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from Table 3 counts"),
                   ],
          loc="upper center", bbox_to_anchor=(0.3, -0.13), ncol=2, frameon=False, fontsize=10,
          handletextpad=0.4, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")

# Figure 2: two-sided Fisher P for every table with the paper's margins
fig, ax = plt.subplots(figsize=(7.2, 4.3))
xs = np.array(sorted(fisher_by_x)); ps = np.array([fisher_by_x[x] for x in xs])
for x, p in zip(xs, ps):
    obs = (x == lo_r)
    ax.plot([x], [p], "o", ms=11 if obs else 8, color=BLUE if obs else BG, mec=BLUE, mew=2, zorder=3)
ax.axhline(REPORTED_P, color=ORANGE, lw=2, zorder=2)
ax.text(2.0, REPORTED_P * 1.3, "Reported P = 0.0005", color=INK, ha="left", va="bottom", fontsize=10)
ax.annotate(f"Observed: 1/6 vs 7/18\n{plabel(fisher_by_x[lo_r])}", xy=(lo_r, fisher_by_x[lo_r]),
            xytext=(1.45, 0.07), fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"Most extreme table in the\nreported direction: 0/6 vs 8/18\n{plabel(fisher_by_x[0])}",
            xy=(0, fisher_by_x[0]), xytext=(0.1, 0.0028), fontsize=10,
            arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"All 6 low-dose patients respond\n(reverse pattern), {plabel(fisher_by_x[6])}",
            xy=(6, fisher_by_x[6]), xytext=(2.9, 0.00018), fontsize=10, va="center",
            arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_yscale("log"); ax.set_ylim(1e-4, 2)
ax.set_xlim(-0.4, 6.4); ax.set_xticks(xs)
ax.set_xticklabels([f"{x}/6\n{R-x}/18" for x in xs], fontsize=10)
ax.set_xlabel("CR/CRi below 40 mg/m² (top line) and at or above 40 mg/m² (bottom line)")
ax.set_ylabel("Two-sided Fisher P (log scale)")
ax.grid(axis="y", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.set_title("Every possible split of 8 responders between 6 and 18 patients", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, label="Reported in the paper"),
                   Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="Recomputed, observed table"),
                   Line2D([], [], marker="o", ls="", ms=8, mfc=BG, mec=BLUE, mew=2, label="Recomputed, other tables")],
          loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))

Hypertension–knowledge p = 0.007 cannot be reproduced from Table 1; its summary statistics give p ≈ 0.59

Diabetologia (2024) · doi:10.1007/s00125-024-06304-3 · PMC11732930

Summary

The paper surveys 1050 Jordanian adults. In its Abstract, Results and Table 1 it reports that diabetes knowledge scores differ significantly by hypertension status: 46.46 ± 13.11 with hypertension (n = 72) against 47.31 ± 12.87 without (n = 978), p = 0.007. The paper's own means, SDs and group sizes do not give that result. Student's t test, which the Table 1 footer names, gives t = −0.540 and p = 0.589 (Welch p = 0.596), with a 95% CI for the difference of −3.94 to 2.24 points. The companion behaviour comparison (p = 0.017) recomputes to p = 0.794. Both are printed as significant associations. The recomputed values are clearly non-significant, so the unadjusted link between hypertension and diabetes knowledge that the Abstract lists has no support in the paper's own numbers.

What the paper reports

Significant associations were found between diabetes knowledge and the following factors: […] hypertension (46.46 ± 13.11 for present vs 47.31 ± 12.87 for absent; p=0.007)

— Abstract, Results. The bracket marks four omitted factors (age, sex, marital status, smoking status). The sentence ends with diabetes status.

Behaviour scores correlated similarly with these factors except for diabetes and smoking status.

— Abstract, Results (this sentence presents the hypertension–behaviour association as significant as well)

Non-hypertensive individuals had higher knowledge scores (47.31 ± 12.87) than those with hypertension (46.46 ± 13.11, p=0.007). Behaviour scores were also higher in non-hypertensive individuals (85.10 ± 19.90) compared with hypertensive individuals (84.46 ± 21.75, p=0.017).

— Results, second paragraph

Yes | 72 (6.86) | 46.46 ± 13.11 | 0.007 | 84.46 ± 21.75 | 0.017 No | 978 (93.14) | 47.31 ± 12.87 | | 85.10 ± 19.90 |

— Table 1, "Distribution of diabetes knowledge and behaviour scores across demographic and clinical categories", Hypertension rows. The columns are n (%), knowledge score, p value, behaviour score, p value.

based on Student’s t tests for variables with two categories and one-way ANOVA for variables with three or more categories

— Table 1 footer. The footer also says the scores are "presented as means ± SD".

Differences in knowledge and behaviour scores across categories were tested using unpaired t test for binary factors (e.g. sex or marital status) and one-way ANOVA for multi-level factors (e.g. age or BMI), with statistical significance set at p<0.05.

— Methods, "Statistical analysis"

Variables that demonstrated significant associations (p<0.05) in preliminary t tests and ANOVA were included in the regression models.

— Methods, "Statistical analysis"

Two later passages describe hypertension's adjusted effect. It points the opposite way from the unadjusted means:

Hypertension status had a significant positive coefficient (4.5346, p<0.0001), indicating higher adjusted knowledge scores among individuals with hypertension.

— Results, third paragraph (Table 2: "Hypertension | 4.5346 | 0.558 | <0.0001")

Participants with hypertension exhibited higher diabetes knowledge and behaviour scores, possibly reflecting a broader understanding of chronic disease management.

— Discussion, third paragraph

What the paper's own numbers show

Method. An unpaired two-sample t test needs only the group sizes, means and SDs, and all of them are printed in Table 1. Student's pooled-variance test (df = n₁ + n₂ − 2 = 1048) is the test that Table 1's footer names. Welch's unequal-variance test and a z test serve as sensitivity checks. For each printed p, the t statistic it requires is found from the inverse t distribution. From that follow the mean difference, the standard error and the 95% interval that the p value implies. To test whether the method recovers the paper's own reporting, the same calculation was applied to every comparison in Table 1: 20 two-group t tests, and 6 one-way ANOVAs computed from group n, mean and SD. Every number below is printed by recompute.py.txt (Python, numpy/scipy).

Quantity Location Reported Recomputed from the paper's numbers
Knowledge, mean difference (hypertension yes − no) Abstract; Results; Table 1 46.46 vs 47.31 −0.85 points
Knowledge, Student's t test (pooled SD 12.89, SE 1.574, df 1048) Abstract; Results; Table 1 p = 0.007 t = −0.540, two-sided p = 0.589
Knowledge, Welch t test (SE √(2.387 + 0.169) = 1.599, df 81.4) sensitivity — t = −0.532, p = 0.596
Knowledge, z test / one-sided Student p sensitivity — 0.595 / 0.295
Knowledge, 95% CI for the difference derived — −3.94 to 2.24 points
Knowledge, |t| needed for p = 0.007 (df 1048) derived p = 0.007 2.702
Knowledge, difference needed for p = 0.007 / p = 0.05 with the printed SDs and n derived 0.85 4.25 / 3.09 points
Knowledge, SE implied by a 0.85 difference with p = 0.007 derived — 0.315, one-fifth of the SE the printed SDs give
Knowledge, common SD that p = 0.007 would need with n = 72 and 978 derived 13.11 and 12.87 2.58
Knowledge, smallest p with each mean and SD moved anywhere within its rounding interval derived p = 0.007 0.585
Knowledge, largest |t| reachable with 978 controls, however large the hypertensive group derived — 2.07 (p = 0.039), still short of 2.70
Knowledge, equal group size needed for p ≤ 0.007 derived 72 and 978 3400 per group (study total 1050)
Behaviour, mean difference (yes − no) Results; Table 1 84.46 vs 85.10 −0.64 points
Behaviour, Student's t test (SE 2.446) Results; Table 1 p = 0.017 t = −0.262, p = 0.794
Behaviour, Welch t test (SE 2.641, df 80.0) sensitivity — t = −0.242, p = 0.809
Behaviour, 95% CI for the difference derived — −5.44 to 4.16 points
Behaviour, difference needed for p = 0.017 derived 0.64 5.85 points
Control: smoking, knowledge (45.78 vs 48.22, n = 452 / 598) Abstract; Results; Table 1 p = 0.003 t = −3.10, p = 0.0020
Control: sex, marital status (knowledge), age (both scores) Abstract; Results; Table 1 p < 0.0001 2.0 × 10⁻⁵ to 8.5 × 10⁻¹²
Control: education, employment and diabetes status (behaviour) Table 1 0.913, 0.429, 0.294 0.91, 0.43, 0.29
All 26 comparisons in Table 1 Table 1 — 14 reproduce; 12 do not; 3 of the 12 cross p = 0.05: hypertension knowledge, hypertension behaviour, physical-activity behaviour
Hypertension coefficient, adjusted model Table 2 / Table 3 +4.5346 (SE 0.558) / +0.5410 (SE 0.133), p < 0.0001 Opposite sign to the unadjusted −0.85 / −0.64; a separate analysis, not a source of 0.007 or 0.017

"Reproduce" means that a printed "<" bound holds, or that the recomputed p lies within 10% of the printed value or within one unit of its last printed digit. Many comparisons reproduce closely, among them the knowledge comparisons for four of the five other factors that the Abstract lists next to hypertension (age, sex, marital status, smoking). The fifth, diabetes status, does not reproduce: its knowledge comparison recomputes to p = 0.0052 (Welch 0.0084) against a printed "<0.001", although it remains significant at p < 0.05. So the paper's summary statistics and the stated test do recover many of its own p values. The hypertension rows are among those that do not, and they are two of the three where the recomputed value lands on the other side of the paper's p < 0.05 threshold. The other non-reproducing rows are listed in the script's output and shown in Figure 2.

Figures

Hypertension yes-minus-no differences in knowledge and behaviour scores, with the narrow 95% intervals implied by the printed p values and the wide recomputed 95% CIs

Figure 1. Mean score difference between participants with and without hypertension (yes minus no), in points on the 0–100 scale, on one axis. The thin vertical line marks no difference. Orange circles and intervals show the printed differences (−0.85 for knowledge, −0.64 for behaviour), each with the 95% interval its printed p value implies (p = 0.007: −1.47 to −0.23; p = 0.017: −1.17 to −0.11). Neither orange interval includes zero. Blue diamonds and intervals are the 95% CIs recomputed from the Table 1 SDs and group sizes with Student's t test: −3.94 to 2.24 (p = 0.59) and −5.44 to 4.16 (p = 0.79). Both blue intervals contain zero. For the printed p values to hold, the standard errors would have to be about one-fifth (knowledge) to one-ninth (behaviour) of what the printed SDs and group sizes give.

Printed versus recomputed p values for all 26 comparisons in Table 1, on a log axis

Figure 2. Two-sided p values printed in Table 1 (orange circles; an arrow marks a value printed as a "<" bound) against p values recomputed from each row's n, mean and SD (blue diamonds; Student's t for two-category variables, one-way ANOVA for three-category variables, as the Table 1 footer specifies). The dashed line marks p = 0.05. Where a printed value is reproduced, the two markers sit together (often the diamond covers the circle) and the row label is in grey. For rows printed as a "<" bound, a reproduced value can lie far to the left of the printed bound; the thin grey line then marks agreement, not a gap. Row labels in bold mark comparisons whose printed p is not reproduced. The shaded rows are the hypertension comparisons, two of the three rows whose recomputed p falls on the other side of p = 0.05 (the third is physical-activity behaviour). Many rows reproduce, including age, sex, marital status and smoking (knowledge), four of the five other factors listed in the Abstract; the fifth, diabetes status, does not reproduce (0.0052 vs printed <0.001). The method therefore recovers the paper's reporting in general. Gaps also occur in rows other than hypertension (for example dyslipidaemia and physical-activity behaviour).

Why it matters

The hypertension result is printed in the Abstract, as one of six factors with "Significant associations" with diabetes knowledge, and the Results text repeats it. The Abstract's next sentence ("Behaviour scores correlated similarly with these factors except for diabetes and smoking status") presents the behaviour association as significant too. Recomputed from the paper's own Table 1, neither comparison is significant: p = 0.59 for knowledge and p = 0.79 for behaviour. The 95% CI for the knowledge difference, −3.94 to 2.24 points, is compatible with no difference. The direction of the printed means is unchanged, with hypertensive participants about 0.85 points lower on a 0–100 scale. What changes is the conclusion that the difference is statistically detectable.

The error has two further consequences.

The paper's central conclusions do not depend on these rows. These are the association between knowledge and behaviour (the knowledge–attitude–behaviour framing) and the differences by age and sex, which do reproduce. The error affects a secondary result that is printed in the Abstract.

A related observation, separate from the claim above: the Table 1 rows for different variables imply different overall knowledge means for the same 1050 participants. About 47.2 follows from most variables (for example sex 47.23, hypertension 47.25, smoking 47.17), but 41.33 from age, 63.09 from diabetes status and 75.98 from location. This does not change the hypertension recomputation, which uses only the hypertension rows. It does show that Table 1 has other internal inconsistencies.

Explanations considered

From the paper's numbers alone it is not possible to tell whether the p values, the means or the SDs in the hypertension rows are in error. Taken together, the printed means, SDs and group sizes are consistent with each other and with the Table 1 overall mean of about 47.2, and they give p ≈ 0.59 and p ≈ 0.79.

Status

As of 2026-10-23:

Sources: doi.org/10.1007/s00125-024-06304-3 · link.springer.com article · PMC11732930 · Europe PMC record · Crossref works record

Suggested correction

The authors could re-run the unadjusted comparisons by hypertension status from the original data and report the group sizes, means, SDs and p values exactly as produced, so that they agree with each other.

It would also help to reconcile the other Table 1 p values that do not reproduce from their rows (Figure 2, script output section 3), and the differing overall knowledge means implied by the diabetes-status, location and age rows. The text could explain how the near-zero unadjusted difference relates to the adjusted hypertension coefficients in Tables 2 and 3.

Recomputation script (python recompute.py.txt)
"""Recompute the unadjusted hypertension comparisons of PMC11732930
(Diabetologia 2024, doi:10.1007/s00125-024-06304-3) and regenerate the figures.

Run:   python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, and five candidate figure
designs (A-E) into its candidates/ subfolder (fig1 = design A, fig2 = design B).

All inputs are copied from Table 1 of the paper (n, mean +/- SD, printed p).
The Table 1 footer states that two-category variables were compared with
Student's t tests and variables with three or more categories with one-way
ANOVA. Student's t is computed with pooled variance (df = n1 + n2 - 2); Welch's
t is shown as a sensitivity check.
"""
import os
import shutil

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------------------
# Table 1 inputs: label, (n, mean, SD) group 1, (n, mean, SD) group 2, printed p
# Differences are always group 1 minus group 2, in the order of Table 1.
# ---------------------------------------------------------------------------
TWO_GROUP = [
    ("Sex (male vs female)", "knowledge", (474, 45.09, 13.27), (576, 49.00, 12.35), "<0.0001"),
    ("Sex (male vs female)", "behaviour", (474, 81.81, 22.00), (576, 88.00, 18.14), "<0.0001"),
    ("Marital status", "knowledge", (652, 45.39, 13.10), (398, 50.92, 11.69), "<0.0001"),
    ("Marital status", "behaviour", (652, 83.47, 20.91), (398, 88.53, 18.11), "0.0003"),
    ("Education", "knowledge", (176, 47.04, 14.58), (874, 47.27, 12.56), "0.457"),
    ("Education", "behaviour", (176, 85.06, 21.33), (874, 85.24, 19.98), "0.913"),
    ("Employment", "knowledge", (431, 47.70, 12.35), (619, 46.91, 13.29), "0.745"),
    ("Employment", "behaviour", (431, 85.80, 20.42), (619, 84.80, 20.05), "0.429"),
    ("Hypertension (yes vs no)", "knowledge", (72, 46.46, 13.11), (978, 47.31, 12.87), "0.007"),
    ("Hypertension (yes vs no)", "behaviour", (72, 84.46, 21.75), (978, 85.10, 19.90), "0.017"),
    ("Dyslipidaemia", "knowledge", (16, 47.12, 11.34), (1034, 47.25, 12.98), "0.076"),
    ("Dyslipidaemia", "behaviour", (16, 86.25, 22.45), (1034, 85.00, 19.98), "0.228"),
    ("Coronary artery disease", "knowledge", (11, 47.45, 10.58), (1039, 47.19, 12.77), "0.975"),
    ("Coronary artery disease", "behaviour", (11, 82.73, 20.58), (1039, 85.20, 20.00), "0.849"),
    ("Smoking", "knowledge", (452, 45.78, 13.22), (598, 48.22, 12.15), "0.003"),
    ("Smoking", "behaviour", (452, 84.09, 21.32), (598, 86.15, 19.43), "0.076"),
    ("Diabetes status", "knowledge", (52, 69.49, 17.35), (998, 62.76, 16.88), "<0.001"),
    ("Diabetes status", "behaviour", (52, 88.08, 20.39), (998, 85.06, 20.19), "0.294"),
    ("Location", "knowledge", (921, 75.60, 20.90), (129, 78.71, 19.35), "0.1108"),
    ("Location", "behaviour", (921, 84.99, 20.29), (129, 86.74, 19.53), "0.3571"),
]
MULTI_GROUP = [
    ("Age (3 groups)", "knowledge", [(542, 39.73, 9.95), (382, 42.61, 9.18), (126, 44.32, 9.53)], "<0.0001"),
    ("Age (3 groups)", "behaviour", [(542, 82.42, 21.84), (382, 88.06, 17.72), (126, 88.57, 18.27)], "<0.0001"),
    ("Physical activity (3 groups)", "knowledge", [(851, 46.37, 12.73), (91, 47.90, 11.88), (108, 48.35, 13.07)], "0.200"),
    ("Physical activity (3 groups)", "behaviour", [(851, 83.40, 20.56), (91, 87.77, 18.22), (108, 90.08, 17.95)], "0.507"),
    ("BMI (3 groups)", "knowledge", [(535, 46.52, 12.99), (345, 48.60, 11.82), (170, 47.13, 12.20)], "0.089"),
    ("BMI (3 groups)", "behaviour", [(535, 84.52, 20.12), (345, 86.70, 18.98), (170, 83.05, 21.64)], "0.108"),
]


def pooled(a, b):
    (n1, m1, s1), (n2, m2, s2) = a, b
    df = n1 + n2 - 2
    sp = np.sqrt(((n1 - 1) * s1 ** 2 + (n2 - 1) * s2 ** 2) / df)
    se = sp * np.sqrt(1 / n1 + 1 / n2)
    t = (m1 - m2) / se
    return dict(diff=m1 - m2, sp=sp, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))


def welch(a, b):
    (n1, m1, s1), (n2, m2, s2) = a, b
    v1, v2 = s1 ** 2 / n1, s2 ** 2 / n2
    se = np.sqrt(v1 + v2)
    df = (v1 + v2) ** 2 / (v1 ** 2 / (n1 - 1) + v2 ** 2 / (n2 - 1))
    t = (m1 - m2) / se
    return dict(diff=m1 - m2, v1=v1, v2=v2, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))


def anova(groups):
    N = sum(g[0] for g in groups)
    k = len(groups)
    gm = sum(g[0] * g[1] for g in groups) / N
    ssb = sum(g[0] * (g[1] - gm) ** 2 for g in groups)
    ssw = sum((g[0] - 1) * g[2] ** 2 for g in groups)
    F = (ssb / (k - 1)) / (ssw / (N - k))
    return F, k - 1, N - k, stats.f.sf(F, k - 1, N - k)


def parse_p(s):
    """Return (value, is_upper_bound, half of last printed digit)."""
    ub = s.startswith("<")
    v = s.lstrip("<")
    dec = len(v.split(".")[1])
    return float(v), ub, 10 ** (-dec)


def agrees(printed, rec):
    """Printed p reproduces if a '<x' bound holds, or if the recomputed value,
    rounded to the printed precision, is within one unit of the last digit,
    or if it lies within 10% of the printed value."""
    v, ub, unit = parse_p(printed)
    if ub:
        return rec < v
    return abs(round(rec / unit) - round(v / unit)) <= 1 or abs(rec - v) <= 0.10 * v


def ci95(res):
    q = stats.t.ppf(0.975, res["df"])
    return res["diff"] - q * res["se"], res["diff"] + q * res["se"]


def fmt_p(p):
    return f"{p:.2g}" if p >= 0.001 else f"{p:.1e}"


print("=" * 78)
print("1. Hypertension, diabetes KNOWLEDGE score (Table 1; Abstract; Results)")
print("=" * 78)
HY_K = ((72, 46.46, 13.11), (978, 47.31, 12.87))
HY_B = ((72, 84.46, 21.75), (978, 85.10, 19.90))
sk, wk = pooled(*HY_K), welch(*HY_K)
lo_k, hi_k = ci95(sk)
print(f"mean difference (yes - no)            {sk['diff']:.2f}")
print(f"Welch variance components             {wk['v1']:.3f} + {wk['v2']:.3f} = {wk['v1'] + wk['v2']:.3f}")
print(f"Welch SE                              {wk['se']:.3f}")
print(f"Welch t, df, two-sided p              t = {wk['t']:.3f}, df = {wk['df']:.1f}, p = {wk['p']:.3f}")
print(f"pooled SD                             {sk['sp']:.2f}")
print(f"Student SE                            {sk['se']:.4f}")
print(f"Student t, df, two-sided p            t = {sk['t']:.3f}, df = {sk['df']}, p = {sk['p']:.3f}")
print(f"Student one-sided p                   {sk['p'] / 2:.3f}")
print(f"z-test (Welch SE) two-sided p         {2 * stats.norm.sf(abs(wk['t'])):.3f}")
print(f"95% CI for the difference (Student)   {lo_k:.2f} to {hi_k:.2f}")
t_007 = stats.t.ppf(1 - 0.007 / 2, sk["df"])
t_05 = stats.t.ppf(1 - 0.05 / 2, sk["df"])
print(f"|t| needed for p = 0.007 (df 1048)    {t_007:.3f}")
print(f"difference needed for p = 0.007       {t_007 * sk['se']:.2f} points")
print(f"difference needed for p = 0.05        {t_05 * sk['se']:.2f} points")
se_imp_k = abs(sk["diff"]) / t_007
print(f"SE implied by diff 0.85 and p = 0.007 {se_imp_k:.3f}  (ratio to Student SE {sk['se'] / se_imp_k:.1f})")
imp_lo_k, imp_hi_k = sk["diff"] - t_05 * se_imp_k, sk["diff"] + t_05 * se_imp_k
print(f"95% CI implied by diff and p = 0.007  {imp_lo_k:.2f} to {imp_hi_k:.2f}")
# Common SD (if both groups shared one SD) needed for p = 0.007
sd_need = abs(sk["diff"]) / (t_007 * np.sqrt(1 / 72 + 1 / 978))
print(f"common SD that p = 0.007 would need   {sd_need:.2f} (printed SDs 13.11 and 12.87)")
# Even an unlimited hypertensive group cannot reach p = 0.007 with 978 controls:
t_lim = abs(sk["diff"]) / (12.87 / np.sqrt(978))
print(f"largest |t| with 978 non-hypertensive and infinitely many hypertensive: {t_lim:.2f}"
      f" (p = {2 * stats.norm.sf(t_lim):.3f})")
# Equal group sizes needed
for n in range(10, 200000):
    if pooled((n, 46.46, 13.11), (n, 47.31, 12.87))["p"] <= 0.007:
        print(f"equal n per group needed for p <= 0.007: {n} (total {2 * n}; study total 1050)")
        break

# Rounding: move every printed mean and SD to the edge of its rounding interval
pmin = 1.0
for d1 in (-0.005, 0.005):
    for d2 in (-0.005, 0.005):
        for e1 in (-0.005, 0.005):
            for e2 in (-0.005, 0.005):
                r = pooled((72, 46.46 + d1, 13.11 + e1), (978, 47.31 + d2, 12.87 + e2))
                pmin = min(pmin, r["p"])
print(f"smallest Student p over rounding of means and SDs   {pmin:.3f}")
# If the +/- values were SEs rather than SDs
se_alt = np.sqrt(13.11 ** 2 + 12.87 ** 2)
print(f"if +/- were SEs: t = {0.85 / se_alt:.3f}, z-test p = {2 * stats.norm.sf(0.85 / se_alt):.2f};"
      f" implied SD in the hypertensive group = {13.11 * np.sqrt(72):.0f} on a 0-100 score")

print()
print("=" * 78)
print("2. Hypertension, diabetes BEHAVIOUR score (Table 1; Results)")
print("=" * 78)
sb, wb = pooled(*HY_B), welch(*HY_B)
lo_b, hi_b = ci95(sb)
print(f"mean difference (yes - no)            {sb['diff']:.2f}")
print(f"Student SE, t, p                      SE = {sb['se']:.3f}, t = {sb['t']:.3f}, p = {sb['p']:.3f}")
print(f"Welch SE, t, df, p                    SE = {wb['se']:.3f}, t = {wb['t']:.3f}, df = {wb['df']:.1f}, p = {wb['p']:.3f}")
print(f"95% CI for the difference (Student)   {lo_b:.2f} to {hi_b:.2f}")
t_017 = stats.t.ppf(1 - 0.017 / 2, sb["df"])
print(f"difference needed for p = 0.017       {t_017 * sb['se']:.2f} points")
se_imp_b = abs(sb["diff"]) / t_017
imp_lo_b, imp_hi_b = sb["diff"] - t_05 * se_imp_b, sb["diff"] + t_05 * se_imp_b
print(f"SE implied by diff 0.64 and p = 0.017 {se_imp_b:.3f}  (ratio to Student SE {sb['se'] / se_imp_b:.1f})")
print(f"95% CI implied by diff and p = 0.017  {imp_lo_b:.2f} to {imp_hi_b:.2f}")

print()
print("=" * 78)
print("3. Every Table 1 comparison: printed p vs p recomputed from Table 1 summaries")
print("=" * 78)
print(f"{'variable':30s} {'score':9s} {'diff':>6s} {'Student p':>10s} {'Welch p':>9s} {'printed':>8s}  reproduces?")
ALL = []
for lab, sc, a, b, rp in TWO_GROUP:
    s, w = pooled(a, b), welch(a, b)
    ok = agrees(rp, s["p"])
    ALL.append((lab, sc, rp, s["p"], ok))
    print(f"{lab:30s} {sc:9s} {s['diff']:6.2f} {fmt_p(s['p']):>10s} {fmt_p(w['p']):>9s} {rp:>8s}  {'yes' if ok else 'NO'}")
for lab, sc, gs, rp in MULTI_GROUP:
    F, d1, d2, p = anova(gs)
    ok = agrees(rp, p)
    ALL.append((lab, sc, rp, p, ok))
    print(f"{lab:30s} {sc:9s} {'F=' + format(F, '.2f'):>6s} {fmt_p(p):>10s} {'':>9s} {rp:>8s}  {'yes' if ok else 'NO'}")
n_ok = sum(r[4] for r in ALL)
print(f"\nreproduce: {n_ok} of {len(ALL)}; do not reproduce: {len(ALL) - n_ok}")
flip = [r for r in ALL if (parse_p(r[2])[0] < 0.05) != (r[3] < 0.05)]
print("comparisons whose significance at 0.05 changes:", [f"{r[0]} / {r[1]}" for r in flip])
print("smoking knowledge (control): Student t = "
      f"{pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['t']:.2f}, p = {pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['p']:.4f}")
dm = pooled((52, 69.49, 17.35), (998, 62.76, 16.88))
dmw = welch((52, 69.49, 17.35), (998, 62.76, 16.88))
print(f"diabetes-status knowledge: Student p = {dm['p']:.4f}, Welch p = {dmw['p']:.4f} (printed <0.001)")

print()
print("=" * 78)
print("4. Sample-wide knowledge mean implied by each variable's Table 1 rows")
print("=" * 78)
groups_k = {lab: [a, b] for lab, sc, a, b, rp in TWO_GROUP if sc == "knowledge"}
groups_k.update({lab: gs for lab, sc, gs, rp in MULTI_GROUP if sc == "knowledge"})
for lab, gs in groups_k.items():
    N = sum(g[0] for g in gs)
    print(f"  {lab:30s} N = {N:5d}  implied mean = {sum(g[0] * g[1] for g in gs) / N:6.2f}")

print()
print("=" * 78)
print("5. Adjusted regression rows (Tables 2 and 3), for comparison only")
print("=" * 78)
for lab, b_, se_ in [("knowledge (Table 2)", 4.5346, 0.558), ("behaviour (Table 3)", 0.5410, 0.133)]:
    z = b_ / se_
    print(f"  hypertension, {lab}: coefficient {b_:+.4f}, SE {se_}, z = {z:.2f}, p = {2 * stats.norm.sf(z):.1e};"
          f" sign {'+' if b_ > 0 else '-'} vs unadjusted difference sign -")

# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, BLUE_L = "#eb6834", "#2a78d6", "#a9c8ee"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "axes.spines.top": False, "axes.spines.right": False, "axes.linewidth": 0.8,
    "svg.fonttype": "none",
})
LW, MS = 2.0, 9  # 2 px lines (at 72 pt/in) and markers >= 8 px


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(CAND, f"{name}.{ext}"), dpi=200, bbox_inches="tight")
    plt.close(fig)


def grid_x(ax):
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)


# Design A: dot-and-interval, mean difference with 95% CI, reported vs recomputed
def design_a():
    fig, ax = plt.subplots(figsize=(7.2, 3.4))
    rows = [
        ("Knowledge score", sk["diff"], (imp_lo_k, imp_hi_k), (lo_k, hi_k), "0.007", sk["p"]),
        ("Behaviour score", sb["diff"], (imp_lo_b, imp_hi_b), (lo_b, hi_b), "0.017", sb["p"]),
    ]
    y = 3.0
    yt, yl = [], []
    for lab, d, imp, rec, rp, pp in rows:
        ax.plot(imp, [y + 0.28] * 2, color=ORANGE, lw=LW, solid_capstyle="round", zorder=3)
        ax.plot(d, y + 0.28, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
        ax.text(imp[1] + 0.25, y + 0.28, f"interval implied by printed p = {rp}", va="center", fontsize=10, color=INK)
        ax.plot(rec, [y - 0.28] * 2, color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
        ax.plot(d, y - 0.28, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
        ax.text(rec[1] + 0.25, y - 0.28, f"recomputed 95% CI, p = {pp:.2f}", va="center", fontsize=10, color=INK)
        yt.append(y)
        yl.append(lab)
        y -= 1.6
    ax.plot([0, 0], [0.6, 3.6], color=INK2, lw=0.8, zorder=1)
    ax.text(0, 3.65, "no difference", ha="center", va="bottom", fontsize=10, color=INK2)
    ax.set_yticks(yt, yl, fontsize=10.5, color=INK)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    ax.set_ylim(0.6, 3.9)
    ax.set_xlim(-6.5, 10.5)
    ax.set_xlabel("Mean score difference, hypertension yes minus no (points on the 0-100 scale)")
    grid_x(ax)
    h1, = ax.plot([], [], "o-", color=ORANGE, lw=LW, ms=MS, label="Paper: printed difference with the 95% interval its p value implies")
    h2, = ax.plot([], [], "D-", color=BLUE, lw=LW, ms=MS - 1, label="Recomputed from Table 1 means, SDs and n (Student's t)")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.22), frameon=False, fontsize=10, ncol=1)
    save(fig, "A_difference_intervals")


# Design B: printed vs recomputed p for every Table 1 comparison (log scale)
def design_b():
    order = list(range(len(ALL)))
    fig, ax = plt.subplots(figsize=(7.2, 7.6))
    labels = []
    for i, idx in enumerate(order):
        lab, sc, rp, rec, ok = ALL[idx]
        y = len(order) - 1 - i
        v, ub, _ = parse_p(rp)
        is_h = lab.startswith("Hypertension")
        if is_h:
            ax.axhspan(y - 0.5, y + 0.5, color="#f3f2ee", zorder=0)
        ax.plot([min(v, rec), max(v, rec)], [y, y], color=INK2 if ok else INK, lw=0.8 if ok else 1.4, zorder=2)
        if ub:
            ax.annotate("", xy=(v / 3, y), xytext=(v, y),
                        arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=1.5, mutation_scale=10), zorder=3)
        ax.plot(v, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.2, zorder=4)
        ax.plot(max(rec, 1e-12), y, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.2, zorder=5)
        labels.append((y, f"{lab.split(' (')[0]}: {sc}", ok))
    ax.set_xscale("log")
    ax.set_xlim(1e-12, 1.6)
    ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (4, 3)), zorder=1)
    ax.text(0.05 * 1.25, len(order) - 0.45, "p = 0.05", ha="left", va="center", fontsize=10, color=INK2)
    ax.set_yticks([l[0] for l in labels], [l[1] for l in labels], fontsize=10)
    for tick, (_, _, ok) in zip(ax.get_yticklabels(), labels):
        tick.set_color(INK if not ok else INK2)
        tick.set_fontweight("bold" if not ok else "normal")
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    ax.set_ylim(-0.7, len(order) + 0.2)
    ax.set_xlabel("Two-sided p value (log scale)")
    grid_x(ax)
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Printed in Table 1 (arrow: printed as '<' bound)")
    h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from Table 1 summaries (t test or ANOVA)")
    ax.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(-0.02, -0.07), frameon=False, fontsize=10, ncol=1)
    ax.text(-0.02, -0.165, "Bold row label: printed p not reproduced (differs by more than 10% and by more than one unit "
            "in its last printed digit).\nShaded rows: hypertension.",
            transform=ax.transAxes, ha="left", va="top", fontsize=10, color=INK2)
    save(fig, "B_all_table1_pvalues")


# Design C: observed difference vs the difference each p value would need
def design_c():
    fig, ax = plt.subplots(figsize=(7.2, 2.9))
    items = [
        (abs(sk["diff"]), ORANGE, "o", "Printed difference\n|46.46 - 47.31| = 0.85", +1),
        (t_05 * sk["se"], BLUE, "D", f"Needed for p = 0.05:\n{t_05 * sk['se']:.2f}", -1),
        (t_007 * sk["se"], BLUE, "D", f"Needed for the printed\np = 0.007: {t_007 * sk['se']:.2f}\n"
         f"({t_007 * sk['se'] / abs(sk['diff']):.1f} times 0.85)", +1),
    ]
    ax.plot([0, 6], [1, 1], color=GRID, lw=LW, zorder=1)
    ax.plot([abs(sk["diff"]), t_007 * sk["se"]], [1, 1], color=INK2, lw=1.0, zorder=2)
    for x, c, m, txt, side in items:
        ax.plot(x, 1, m, color=c, ms=MS + 1, mec=BG, mew=1.5, zorder=4)
        ax.text(x, 1 + 0.1 * side, txt, ha="center", va="bottom" if side > 0 else "top", fontsize=10, color=INK)
    ax.set_xlim(0, 6)
    ax.set_ylim(0.55, 1.75)
    ax.set_yticks([])
    ax.spines["left"].set_visible(False)
    ax.set_xlabel("Absolute mean difference in knowledge score, hypertension yes vs no (points, 0-100 scale)\n"
                  "with the printed SDs (13.11, 12.87) and group sizes (72, 978), Student's t")
    grid_x(ax)
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="From the paper (Table 1)")
    h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from the paper's SDs and n")
    ax.legend(handles=[h1, h2], loc="upper left", frameon=False, fontsize=10, ncol=2, bbox_to_anchor=(0, 1.08))
    save(fig, "C_difference_needed")


# Design D: each group's mean with its own 95% CI
def design_d():
    fig, ax = plt.subplots(figsize=(7.2, 3.0))
    gs = [("Hypertension: yes (n = 72)", 72, 46.46, 13.11), ("Hypertension: no (n = 978)", 978, 47.31, 12.87)]
    for i, (lab, n, m, s) in enumerate(gs):
        y = 1 - i
        h = stats.t.ppf(0.975, n - 1) * s / np.sqrt(n)
        ax.plot([m - h, m + h], [y, y], color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
        ax.plot(m, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
        ax.text(m + h + 0.3, y, f"{m:.2f}  (95% CI {m - h:.2f} to {m + h:.2f})", va="center", fontsize=10)
    ax.set_yticks([1, 0], [g[0] for g in gs], fontsize=10.5, color=INK)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    ax.set_ylim(-0.6, 1.9)
    ax.set_xlim(40, 60)
    ax.text(40.6, 1.6, f"Printed: p = 0.007", color=INK, fontsize=10.5, va="center")
    ax.plot(40.25, 1.6, "o", color=ORANGE, ms=8, clip_on=False)
    ax.text(48.6, 1.6, f"Recomputed (Student's t): p = {sk['p']:.2f}", color=INK, fontsize=10.5, va="center")
    ax.plot(48.25, 1.6, "D", color=BLUE, ms=7, clip_on=False)
    ax.set_xlabel("Mean diabetes knowledge score (0-100 scale; axis shows 40-60)")
    grid_x(ax)
    h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Group mean printed in Table 1")
    h2, = ax.plot([], [], "-", color=BLUE, lw=LW, label="95% CI recomputed from the printed SD and n")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), frameon=False, fontsize=10, ncol=2)
    save(fig, "D_group_means")


# Design E: where the test statistic sits on the null t distribution
def design_e():
    fig, ax = plt.subplots(figsize=(7.2, 3.3))
    x = np.linspace(-4.2, 4.2, 801)
    yv = stats.t.pdf(x, sk["df"])
    ax.plot(x, yv, color=INK2, lw=1.2, zorder=2)
    ta = abs(sk["t"])
    m = np.abs(x) >= ta
    ax.fill_between(x, 0, yv, where=m & (x < 0), color=BLUE_L, lw=0, zorder=1)
    ax.fill_between(x, 0, yv, where=m & (x > 0), color=BLUE_L, lw=0, zorder=1)
    ax.axvline(sk["t"], color=BLUE, lw=LW, zorder=3)
    ax.plot(sk["t"], stats.t.pdf(sk["t"], sk["df"]), "D", color=BLUE, ms=MS - 1, mec=BG, zorder=4)
    for s in (-1, 1):
        ax.axvline(s * t_007, color=ORANGE, lw=LW, ls=(0, (5, 3)), zorder=3)
    ax.text(sk["t"] - 0.08, 0.43, f"recomputed t = {sk['t']:.2f}\nshaded tails: p = {sk['p']:.2f}",
            ha="right", va="top", fontsize=10, color=INK)
    ax.text(-t_007 - 0.08, 0.2, f"|t| needed for\nprinted p = 0.007:\n{t_007:.2f}", ha="right", va="center",
            fontsize=10, color=INK)
    ax.set_ylim(0, 0.45)
    ax.set_xlim(-4.2, 4.2)
    ax.set_yticks([])
    ax.spines["left"].set_visible(False)
    ax.set_xlabel("Student's t statistic under no difference (df = 1048), hypertension knowledge comparison")
    grid_x(ax)
    h1, = ax.plot([], [], "-", color=BLUE, lw=LW, label="Recomputed from Table 1 summaries")
    h2, = ax.plot([], [], "--", color=ORANGE, lw=LW, label="Implied by the printed p = 0.007")
    ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), frameon=False, fontsize=10, ncol=2)
    save(fig, "E_null_distribution")


design_a()
design_b()
design_c()
design_d()
design_e()
for src, dst in [("A_difference_intervals", "fig1"), ("B_all_table1_pvalues", "fig2")]:
    for ext in ("svg", "png"):
        shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nfigures written: fig1 (design A), fig2 (design B), candidates/A-E (.svg, .png)")

Abstract calls a non-significant subgroup difference at seroconversion significant; Results and Table 3 report none

Diabetologia (2025) · doi:10.1007/s00125-025-06573-6 · PMC12686032

Summary

This DIPP cohort analysis compares 1334 children with islet autoimmunity in three groups: an affected first-degree relative (FDR) at birth (FDR+, n = 145), an FDR diagnosed during follow-up (FDR− FDR+, n = 87), and no affected FDR (FDR−, n = 1102). The abstract's Results state that at seroconversion both FDR+ and FDR− FDR+ children were "significantly more often positive for GADA and multiple autoantibodies" than FDR− children. The paper's own Table 3 gives non-significant results for the FDR− FDR+ group: GADA OR 1.3 (95% CI 0.9, 2.1), p = 0.201, and multipositivity OR 1.5 (0.9, 2.4), p = 0.106. The Results text and the Discussion both say this group did not differ from FDR− children at seroconversion, and a recomputation from the Table 2 counts agrees (p = 0.19 and 0.11). The abstract is the only place where the claim appears. A reader who sees only the abstract could reasonably conclude that a family history arising after birth already marks a distinct autoantibody profile at seroconversion, which the body of the paper does not show.

What the paper reports

Text is quoted from the PubMed Central archive, which writes the group label as "FDR- FDR+" with a hyphen; the version of record typesets it with a minus sign (FDR− FDR+).

Abstract, Results:

At seroconversion, both FDR+ and FDR- FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR-).

Results, fourth paragraph (beginning “We compared the islet autoantibody profiles”):

At seroconversion, there was a statistically significant difference between the three groups for GADA positivity and multipositivity (Table 2). In pairwise comparisons, at seroconversion, FDR+ children had a higher risk for GADA positivity and multipositivity than FDR- children, but no difference was observed between FDR- FDR+ and FDR- children (Table 3).

Table 3, "At seroconversion" rows (columns: FDR+ vs FDR− | p value | FDR− FDR+ vs FDR− | p value; footnote: "Values are OR (95% CI) by logistic regression analysis"):

GADA+ | 1.5 (1.1, 2.2) | 0.020 | 1.3 (0.9, 2.1) | 0.201

Multipositive | 1.8 (1.2, 2.6) | 0.002 | 1.5 (0.9, 2.4) | 0.106

Table 2, "At seroconversion" rows (columns: FDR+ (n=145) | FDR− FDR+ (n=87) | FDR− (n=1102) | p value; data are % (n); footnote: "The exact χ2 test was used for categorical variables"):

GADA+ | 62.8 (91) | 59.8 (52) | 52.5 (578) | 0.036

Multipositive | 35.2 (51) | 31.0 (27) | 23.3 (257) | 0.004

Discussion, seventh paragraph (beginning “In terms of the timing of type 1 diabetes in the FDRs”):

FDR- FDR+ children did not differ from FDR- children at seroconversion, but showed a significantly higher risk of being positive for IAA, GADA, IA-2A or multiple autoantibodies during the follow-up period than FDR- children (Table 3).

Methods, "Statistical analyses":

The standard normal distribution (SND) test was used for pairwise comparisons of categorical variables. Logistic regression analysis was used to estimate the risk of various islet autoantibody profiles and progression to type 1 diabetes in children with and without an FDR with the disease.

Abstract, Conclusions/interpretation (for comparison; this sentence does not mention the FDR− FDR+ group):

At seroconversion, genetically susceptible children who had one or more FDRs with type 1 diabetes, especially an affected father, were more often positive for GADA and multiple islet autoantibodies.

What the paper's own numbers show

Method. From the Table 2 counts I rebuilt each 2×2 table (group vs FDR−; positive vs not positive) at seroconversion. For each I computed the unadjusted odds ratio with a Woolf 95% CI and Wald p value. These are exactly what a logistic regression of the outcome on a single group indicator gives, and the Methods name no covariates. I also computed the pooled two-proportion z test (the "SND test" named in the Methods), Fisher's exact test, the likelihood-ratio test and the Yates-corrected χ² test, and repeated the three-group χ² tests from Table 2. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Group sizes Abstract; Table 2 145 / 87 / 1102; total 1334 145 + 87 + 1102 = 1334
GADA+ at seroconversion, FDR− FDR+ vs FDR− Table 2; Table 3 59.8% (52) vs 52.5% (578); OR 1.3 (0.9, 2.1), p = 0.201 52/87 vs 578/1102; OR 1.35 (0.86, 2.10); Wald p = 0.189; SND p = 0.188; Fisher p = 0.220; LR p = 0.186
Multipositive at seroconversion, FDR− FDR+ vs FDR− Table 2; Table 3 31.0% (27) vs 23.3% (257); OR 1.5 (0.9, 2.4), p = 0.106 27/87 vs 257/1102; OR 1.48 (0.92, 2.38); Wald p = 0.106; SND p = 0.104; Fisher p = 0.117; LR p = 0.114
GADA+ at seroconversion, FDR+ vs FDR− Table 3 OR 1.5 (1.1, 2.2), p = 0.020 91/145 vs 578/1102; OR 1.53 (1.07, 2.18); p = 0.020
Multipositive at seroconversion, FDR+ vs FDR− Table 3 OR 1.8 (1.2, 2.6), p = 0.002 51/145 vs 257/1102; OR 1.78 (1.23, 2.58); p = 0.002
Three-group test, GADA+ at seroconversion Table 2 p = 0.036 (exact χ²) Pearson χ² = 6.71, df 2, p = 0.035
Three-group test, multipositive at seroconversion Table 2 p = 0.004 (exact χ²) Pearson χ² = 11.31, df 2, p = 0.004
Abstract statement on FDR− FDR+ vs FDR− at seroconversion Abstract, Results "significantly more often positive for GADA and multiple autoantibodies" not significant on either measure under any of the tests above; both 95% CIs include 1

The two disputed cells of Table 3 reproduce from Table 2: the odds ratios and intervals match to the printed precision, and the p values are close (printed 0.201, recomputed 0.189 for GADA; printed 0.106, recomputed 0.106 for multipositivity). The body of the paper is internally consistent. Only the abstract sentence departs from it.

Side observation, not part of the error. When every Table 3 cell is recomputed in the same way from the Table 2 counts, six other cells do not reproduce (rule applied in recompute.py.txt, section 7: the recomputed OR, rounded to one decimal, differs from the print by more than 0.1, or the p value differs by more than 0.05). Five are in the FDR− FDR+ vs FDR− column: IAA+ at seroconversion (printed 1.3 (0.8, 2.1), p = 0.181; recomputed 1.52 (0.97, 2.39), p = 0.070), IA-2A+ at seroconversion (printed 1.0 (0.5, 1.6), p = 0.937; recomputed 0.95 (0.52, 1.75), p = 0.869), ZnT8A+ at seroconversion (printed 0.8 (0.3, 1.5), p = 0.484; recomputed 0.89 (0.43, 1.81), p = 0.739), GADA+ during follow-up (printed 2.8 (1.1, 3.7), p = 0.021; recomputed 2.05 (1.12, 3.76), p = 0.020) and ZnT8A+ during follow-up (printed 1.2 (0.8, 1.8), p = 0.361; recomputed 1.48 (0.96, 2.30), p = 0.077). One is in the FDR+ vs FDR− column: ZnT8A+ during follow-up (printed 1.2 (0.8, 1.7), p = 0.227; recomputed 1.21 (0.86, 1.71), p = 0.278). For GADA+ during follow-up, the printed estimate 2.8 is far from the geometric midpoint of its interval (1.1, 3.7), which is 2.0, while the recomputed OR is 2.05; this suggests a typographical slip in the point estimate. None of these six changes which side of p = 0.05 a result falls on, and none bears on the abstract sentence.

Figure

The abstract and the body of the paper give opposite verdicts on the same comparison

Figure 1. The left panel quotes the abstract's claim that FDR− FDR+ children were significantly more often GADA-positive and multipositive at seroconversion than FDR− children. The right panel quotes the Results sentence and the Table 3 estimates for the same comparison, which are non-significant. Values the paper reports are in orange. The blue panel gives the estimates recomputed from the Table 2 counts (unadjusted logistic regression, equivalent to the 2×2 odds ratio), with Fisher exact p values for comparison. Both recomputed intervals include OR = 1.

Reported and recomputed odds ratios for the four pairwise comparisons at seroconversion

Figure 2. Odds ratios with 95% confidence intervals for the four pairwise comparisons at seroconversion, on a log scale with a thin reference line at OR = 1 (no difference). Orange circles show Table 3 as printed; blue diamonds show the values recomputed from the Table 2 counts, with the p value for each printed at the right. The two shaded rows are the FDR− FDR+ comparisons, which the abstract calls significant but Table 3 and the Results text do not. In both, the printed and the recomputed intervals cross 1. The two unshaded FDR+ rows, which the abstract also mentions, are significant and reproduce.

Why it matters

Explanations considered

Status

Suggested correction

In the abstract's Results, replace

At seroconversion, both FDR+ and FDR− FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−).

with

At seroconversion, FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−), whereas FDR− FDR+ children did not differ significantly from FDR− children (GADA OR 1.3 [95% CI 0.9, 2.1]; multipositivity OR 1.5 [0.9, 2.4]).

No change to the Results, Tables or Discussion is needed for this point. Separately, the authors may wish to check the six cells of Table 3 listed above (five in the FDR− FDR+ column, one in the FDR+ column) that do not reproduce from the Table 2 counts. One example is GADA+ during follow-up, printed as 2.8 (1.1, 3.7), where the counts give 2.05 (1.12, 3.76).

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12686032 (Diabetologia 2025, doi:10.1007/s00125-025-06573-6)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1 and fig2
# (SVG and 200 dpi PNG) in this script's folder.
# Inputs are the counts and estimates printed in the paper's Table 2 and Table 3.

import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs
N = {"FDR+": 145, "FDR- FDR+": 87, "FDR-": 1102}
# Table 2, % (n): (FDR+, FDR- FDR+, FDR-)
T2 = {
    ("seroconversion", "IAA+"): (83, 55, 585),
    ("seroconversion", "GADA+"): (91, 52, 578),
    ("seroconversion", "IA-2A+"): (25, 13, 172),
    ("seroconversion", "ZnT8A+"): (17, 9, 127),
    ("seroconversion", "Multipositive"): (51, 27, 257),
    ("follow-up", "IAA+"): (104, 71, 756),
    ("follow-up", "GADA+"): (124, 74, 810),
    ("follow-up", "IA-2A+"): (98, 56, 587),
    ("follow-up", "ZnT8A+"): (71, 47, 487),
    ("follow-up", "Multipositive"): (107, 64, 622),
    ("follow-up", "T1D diagnosis"): (81, 50, 429),
}
T2_pct = {  # printed percentages
    ("seroconversion", "GADA+"): (62.8, 59.8, 52.5),
    ("seroconversion", "Multipositive"): (35.2, 31.0, 23.3),
}
T2_p = {("seroconversion", "GADA+"): 0.036, ("seroconversion", "Multipositive"): 0.004,
        ("seroconversion", "IAA+"): 0.143, ("seroconversion", "IA-2A+"): 0.862,
        ("seroconversion", "ZnT8A+"): 0.940}
# Table 3, OR (95% CI), p: FDR+ vs FDR-, then FDR- FDR+ vs FDR-
T3 = {
    ("seroconversion", "IAA+"): ((1.2, 0.8, 1.7, 0.356), (1.3, 0.8, 2.1, 0.181)),
    ("seroconversion", "GADA+"): ((1.5, 1.1, 2.2, 0.020), (1.3, 0.9, 2.1, 0.201)),
    ("seroconversion", "IA-2A+"): ((1.1, 0.7, 1.7, 0.612), (1.0, 0.5, 1.6, 0.937)),
    ("seroconversion", "ZnT8A+"): ((1.0, 0.5, 1.7, 0.944), (0.8, 0.3, 1.5, 0.484)),
    ("seroconversion", "Multipositive"): ((1.8, 1.2, 2.6, 0.002), (1.5, 0.9, 2.4, 0.106)),
    ("follow-up", "IAA+"): ((1.2, 0.8, 1.7, 0.472), (2.0, 1.2, 3.5, 0.014)),
    ("follow-up", "GADA+"): ((2.1, 1.3, 3.4, 0.002), (2.8, 1.1, 3.7, 0.021)),
    ("follow-up", "IA-2A+"): ((1.8, 1.3, 2.7, 0.001), (1.6, 1.0, 2.5, 0.049)),
    ("follow-up", "ZnT8A+"): ((1.2, 0.8, 1.7, 0.227), (1.2, 0.8, 1.8, 0.361)),
    ("follow-up", "Multipositive"): ((2.2, 1.5, 3.2, 0.0005), (2.1, 1.3, 3.5, 0.002)),  # "<0.001" drawn as 0.0005
    ("follow-up", "T1D diagnosis"): ((2.0, 1.4, 2.8, 0.0005), (2.1, 1.4, 3.3, 0.0005)),
}

# ---------------------------------------------------------------- methods
def woolf(a, n1, c, n0):
    """Unadjusted OR, Woolf 95% CI, Wald p. Identical to a logistic regression
    of the outcome on a single group indicator (no covariates)."""
    b, d = n1 - a, n0 - c
    lor = np.log(a * d / (b * c))
    se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
    z = lor / se
    return np.exp(lor), np.exp(lor - 1.96 * se), np.exp(lor + 1.96 * se), 2 * stats.norm.sf(abs(z)), z

def snd(a, n1, c, n0):
    """Pooled two-proportion z test (the 'SND test' named in the Methods)."""
    p1, p0, pp = a / n1, c / n0, (a + c) / (n1 + n0)
    z = (p1 - p0) / np.sqrt(pp * (1 - pp) * (1 / n1 + 1 / n0))
    return z, 2 * stats.norm.sf(abs(z)), stats.norm.sf(z)

def wilson(x, n, z=1.96):
    p = x / n
    den = 1 + z * z / n
    c = (p + z * z / (2 * n)) / den
    h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
    return 100 * p, 100 * (c - h), 100 * (c + h)

def table22(a, n1, c, n0):
    return np.array([[a, n1 - a], [c, n0 - c]])

R = {}  # recomputed results reused by the figures

print("=" * 78)
print("1. Group sizes")
print(f"   145 + 87 + 1102 = {sum(N.values())} (paper: 1334); 145/1334 = {100*145/1334:.1f}% (paper: 10.9%)")

print("=" * 78)
print("2. Table 2 percentages at seroconversion (recomputed from the printed n)")
for key in [("seroconversion", "GADA+"), ("seroconversion", "Multipositive")]:
    x = T2[key]
    pct = [100 * xi / ni for xi, ni in zip(x, N.values())]
    print(f"   {key[1]:14s} {pct[0]:.1f} / {pct[1]:.1f} / {pct[2]:.1f}   (paper: {T2_pct[key]})")

print("=" * 78)
print("3. Three-group tests at seroconversion (Table 2 p column)")
for key in [("seroconversion", k) for k in ["IAA+", "GADA+", "IA-2A+", "ZnT8A+", "Multipositive"]]:
    x = T2[key]
    tab = np.array([[xi, ni - xi] for xi, ni in zip(x, N.values())])
    chi2, p3, _, _ = stats.chi2_contingency(tab, correction=False)
    print(f"   {key[1]:14s} Pearson chi2 = {chi2:.2f}, df 2, p = {p3:.3f}   (paper, exact chi2: {T2_p[key]})")

print("=" * 78)
print("4. The disputed comparison: FDR- FDR+ (n=87) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
    x1, x2, x0 = T2[("seroconversion", k)]
    o, lo, hi, p, zw = woolf(x2, 87, x0, 1102)
    zs, ps, p1s = snd(x2, 87, x0, 1102)
    pf = stats.fisher_exact(table22(x2, 87, x0, 1102))[1]
    pyates = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=True)[1]
    plr = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=False, lambda_="log-likelihood")[1]
    rep = T3[("seroconversion", k)][1]
    R[("FDR- FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
    print(f"   {k}: {x2}/87 ({100*x2/87:.1f}%) vs {x0}/1102 ({100*x0/1102:.1f}%), "
          f"difference {100*(x2/87 - x0/1102):.1f} percentage points")
    print(f"      unadjusted OR {o:.2f} (95% CI {lo:.2f}, {hi:.2f}); Wald z = {zw:.2f}, p = {p:.3f}")
    print(f"      SND / two-proportion z = {zs:.2f}, two-sided p = {ps:.3f}, one-sided p = {p1s:.3f}")
    print(f"      Fisher exact p = {pf:.3f}; Yates chi2 p = {pyates:.3f}; likelihood-ratio p = {plr:.3f}")
    print(f"      Table 3 prints: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]}")

print("=" * 78)
print("5. The comparison the abstract gets right: FDR+ (n=145) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
    x1, x2, x0 = T2[("seroconversion", k)]
    o, lo, hi, p, zw = woolf(x1, 145, x0, 1102)
    pf = stats.fisher_exact(table22(x1, 145, x0, 1102))[1]
    zs, ps, _ = snd(x1, 145, x0, 1102)
    rep = T3[("seroconversion", k)][0]
    R[("FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
    print(f"   {k}: {x1}/145 vs {x0}/1102; OR {o:.2f} ({lo:.2f}, {hi:.2f}), Wald p = {p:.3f}, "
          f"SND p = {ps:.3f}, Fisher p = {pf:.3f}   (Table 3: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]})")

print("=" * 78)
print("6. Alternative readings of the abstract sentence")
print("   (a) pooled 'ever FDR+' (145 + 87 = 232) vs FDR-  -- not what the sentence says ('both ... were')")
for k in ["GADA+", "Multipositive"]:
    x1, x2, x0 = T2[("seroconversion", k)]
    o, lo, hi, p, _ = woolf(x1 + x2, 232, x0, 1102)
    print(f"       {k}: {x1+x2}/232 vs {x0}/1102, OR {o:.2f} ({lo:.2f}, {hi:.2f}), p = {p:.4f}")
print("   (b) FDR- FDR+ vs FDR+ (are the two family-history groups different from each other?)")
for k in ["GADA+", "Multipositive"]:
    x1, x2, x0 = T2[("seroconversion", k)]
    print(f"       {k}: {x2}/87 vs {x1}/145, Fisher p = {stats.fisher_exact(table22(x2, 87, x1, 145))[1]:.3f}")
print("   (c) three-group test without the FDR- FDR+ group's contribution is not needed:"
      " the omnibus test only says the three groups are not all equal.")
print("   (d) smallest one-sided p for FDR- FDR+ vs FDR- (GADA, multipositive):",
      f"{snd(52, 87, 578, 1102)[2]:.3f}, {snd(27, 87, 257, 1102)[2]:.3f}")

print("=" * 78)
print("7. Side observations: every Table 3 cell recomputed as an unadjusted OR from Table 2 counts")
print("   (flag '<--': rounded OR differs from the print by more than 0.1, or p differs by more than 0.05;"
      " none bears on the abstract claim)")
FLAGGED = []
for key, (r1, r2) in T3.items():
    x1, x2, x0 = T2[key]
    for lab, a, n1, r in [("FDR+ vs FDR-", x1, 145, r1), ("FDR- FDR+ vs FDR-", x2, 87, r2)]:
        o, lo, hi, p, _ = woolf(a, n1, x0, 1102)
        flag = "  <--" if abs(round(o, 1) - r[0]) > 0.1 + 1e-9 or (r[3] > 0.001 and abs(p - r[3]) > 0.05) else ""
        if flag:
            FLAGGED.append((key, lab))
        rp = "<0.001" if r[3] == 0.0005 else r[3]
        ps_ = "<0.001" if p < 0.001 else f"{p:.3f}"
        print(f"   {key[0]:14s} {key[1]:14s} {lab:18s} OR {o:.2f} ({lo:.2f}, {hi:.2f}) p={ps_}"
              f"   printed {r[0]} ({r[1]}, {r[2]}) p={rp}{flag}")

print(f"   flagged cells: {len(FLAGGED)} "
      f"({sum(l == 'FDR- FDR+ vs FDR-' for _, l in FLAGGED)} FDR- FDR+ vs FDR-, "
      f"{sum(l == 'FDR+ vs FDR-' for _, l in FLAGGED)} FDR+ vs FDR-)")
r = T3[("follow-up", "GADA+")][1]
print(f"   GADA+ during follow-up, FDR- FDR+ vs FDR-: geometric midpoint of printed CI ({r[1]}, {r[2]}) = "
      f"{np.sqrt(r[1] * r[2]):.1f}; printed OR {r[0]}")
# ================================================================= figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAYBOX = "#eb6834", "#2a78d6", "#e4e3df", "#52514e"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": FG,
    "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
    "axes.edgecolor": "#b9b8b3", "svg.fonttype": "none",
    "figure.facecolor": BG, "axes.facecolor": BG,
})
W = 7.2
LW = 2.0            # 2 px-equivalent line weight in points
MS = 8.5            # marker size (pt)

def save(fig, path):
    fig.savefig(path + ".svg", facecolor=BG)
    fig.savefig(path + ".png", dpi=200, facecolor=BG)
    plt.close(fig)

def box(ax, x, y, w, h, edge, lw=2.0):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
                                fc=BG, ec=edge, lw=lw, transform=ax.transData))

def fmt_or(o, lo, hi, nd=1):
    return f"{o:.{nd}f} ({lo:.{nd}f}, {hi:.{nd}f})"

G = R[("FDR- FDR+", "GADA+")]; M = R[("FDR- FDR+", "Multipositive")]
GP = R[("FDR+", "GADA+")]; MP = R[("FDR+", "Multipositive")]

# ---------------------------------------------- Figure 1: two text panels
def fig_text_panels(path):
    fig = plt.figure(figsize=(W, 5.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 78); ax.axis("off")
    # left: abstract
    box(ax, 2, 30, 41, 45, ORANGE)
    ax.text(5, 71, "Abstract, Results", weight="bold", fontsize=12.5, va="top")
    ax.text(5, 65.5, "“At seroconversion, both FDR+ and\n"
                     "FDR− FDR+ children were\n"
                     "significantly more often positive for\n"
                     "GADA and multiple autoantibodies\n"
                     "than children without affected FDRs\n"
                     "(FDR−).”",
            fontsize=10.5, va="top", linespacing=1.45)
    ax.text(5, 39.5, "Claim for FDR− FDR+ vs FDR−:", color=FG2, fontsize=10.5, va="top")
    ax.text(5, 35.5, "significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
    # right: body
    box(ax, 57, 30, 41, 45, GRAYBOX)
    ax.text(60, 71, "Results text and Table 3", weight="bold", fontsize=12.5, va="top")
    ax.text(60, 65.5, "“…but no difference was observed\n"
                      "between FDR− FDR+ and FDR−\n"
                      "children (Table 3).”", fontsize=10.5, va="top", linespacing=1.45)
    ax.text(60, 52, "Table 3, FDR− FDR+ vs FDR−,\nOR (95% CI), p:", color=FG2, fontsize=10.5, va="top",
            linespacing=1.3)
    ax.text(60, 44.5, "GADA+: 1.3 (0.9, 2.1), p = 0.201\n"
                      "Multipositive: 1.5 (0.9, 2.4), p = 0.106",
            fontsize=10, va="top", color=ORANGE, linespacing=1.5)
    ax.text(60, 35.5, "not significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
    # connector
    ax.add_patch(FancyArrowPatch((44, 52), (56, 52), arrowstyle="<|-|>", mutation_scale=16, lw=LW, color=FG))
    ax.text(50, 56, "conflict", ha="center", weight="bold", fontsize=11)
    ax.text(50, 47.5, "same\ncomparison,\nopposite\nverdict", ha="center", va="top", color=FG2, fontsize=9.5,
            linespacing=1.3)
    # bottom: recomputed
    box(ax, 2, 3, 96, 23, BLUE)
    ax.text(5, 22.5, "Recomputed from the Table 2 counts (unadjusted logistic regression)",
            color=BLUE, weight="bold", fontsize=10.5, va="top")
    ax.text(5, 16.5,
            f"GADA+:  52/87 vs 578/1102,  OR {fmt_or(G['or_'], G['lo'], G['hi'], 2)},  "
            f"p = {G['p']:.2f}  (Fisher exact {G['pf']:.2f})\n"
            f"Multipositive:  27/87 vs 257/1102,  OR {fmt_or(M['or_'], M['lo'], M['hi'], 2)},  "
            f"p = {M['p']:.2f}  (Fisher exact {M['pf']:.2f})\n"
            "Both intervals include OR = 1, as Table 3 and the Results text state.",
            fontsize=10, va="top", linespacing=1.6)
    save(fig, path)

# ---------------------------------------------- Figure 2: forest plot (reported vs recomputed)
ROWS = [  # label, reported tuple, recomputed dict, disputed?
    ("GADA+\nFDR+ vs FDR−", T3[("seroconversion", "GADA+")][0], GP, False),
    ("GADA+\nFDR− FDR+ vs FDR−", T3[("seroconversion", "GADA+")][1], G, True),
    ("Multipositive\nFDR+ vs FDR−", T3[("seroconversion", "Multipositive")][0], MP, False),
    ("Multipositive\nFDR− FDR+ vs FDR−", T3[("seroconversion", "Multipositive")][1], M, True),
]

def forest(ax, rows, ptext=True):
    ax.set_xscale("log")
    ax.set_xlim(0.6, 4.0)
    ticks = [0.6, 0.8, 1, 1.5, 2, 3, 4]
    ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
    ax.axvline(1, color=FG2, lw=0.9, zorder=1)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    n = len(rows)
    for i, (lab, rep, rc, disp) in enumerate(rows):
        y = n - 1 - i
        if disp:
            ax.axhspan(y - 0.45, y + 0.45, color="#f3f2ee", zorder=0)
        ax.plot([rep[1], rep[2]], [y + 0.14] * 2, color=ORANGE, lw=LW, zorder=3, solid_capstyle="butt")
        ax.plot(rep[0], y + 0.14, "o", color=ORANGE, ms=MS, zorder=4)
        ax.plot([rc["lo"], rc["hi"]], [y - 0.14] * 2, color=BLUE, lw=LW, zorder=3, solid_capstyle="butt")
        ax.plot(rc["or_"], y - 0.14, "D", color=BLUE, ms=MS - 1, zorder=4)
        if ptext:
            ax.text(4.25, y + 0.14, f"p = {rep[3]:.3f}", color=ORANGE, va="center", fontsize=10)
            ax.text(4.25, y - 0.14, f"p = {rc['p']:.3f}", color=BLUE, va="center", fontsize=10)
    ax.set_yticks(range(n)); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10, color=FG)
    ax.set_ylim(-0.6, n - 0.4)
    ax.set_xlabel("Odds ratio at seroconversion (log scale); line at OR = 1 means no difference")
    for s in ["top", "right", "left"]:
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)

def legend_handles():
    from matplotlib.lines import Line2D
    return [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported (Table 3), 95% CI"),
            Line2D([], [], color=BLUE, marker="D", ms=MS - 1, lw=LW,
                   label="Recomputed from Table 2 counts, 95% CI")]

def fig_forest(path):
    fig, ax = plt.subplots(figsize=(W, 4.4))
    fig.subplots_adjust(left=0.27, right=0.83, top=0.80, bottom=0.14)
    forest(ax, ROWS)
    ax.legend(handles=legend_handles(), loc="lower left", bbox_to_anchor=(-0.36, 1.08), ncol=2,
              frameon=False, fontsize=9.6, handlelength=2.2)
    for y in [2, 0]:
        ax.text(0.62, y + 0.36, "abstract: “significantly more often”", fontsize=9.3, color=FG2,
                va="center", style="italic")
    save(fig, path)

if __name__ == "__main__":
    fig_text_panels(os.path.join(HERE, "fig1"))
    fig_forest(os.path.join(HERE, "fig2"))
    print("=" * 78)
    print("figures written: fig1.svg/.png, fig2.svg/.png")

The 2013 estimate of MSM living with HIV lies above its own 90% upper limit

Epidemiology (2016) · doi:10.1097/EDE.0000000000000423 · PMC4733816

Summary

The paper's main applied result is the number of men who have sex with men (MSM) living with HIV in the UK in 2013, given in the Abstract and Results as 48,310 (90% plausibility range 39,900–45,560). The Methods define the point estimate as the median and the range limits as the 5th and 95th percentiles of one distribution of model outputs, so the median cannot exceed the upper limit. Here it does, by 2,750. The point estimate is consistent with the paper's other numbers (its four CD4 strata sum to exactly 48,310), and the paper's own Figure 3A shows the 2013 range running from about 40,000 to about 60,400. The printed upper limit is therefore wrong, and as printed it understates the upward uncertainty of the headline estimate by roughly 15,000 people, in a paper whose stated contribution is a method that produces such ranges.

What the paper reports

Abstract:

In 2013, 48,310 (90% plausibility range: 39,900–45,560) men who have sex with men were estimated to be living with HIV in the UK, of whom 10,400 (6,160–17,350) were undiagnosed.

Results, third paragraph of "Calibration to Data on MSM in the UK and Resulting Estimates for 2013":

By 2013, an estimated 48,310 (90% plausibility range: 39,900–45,560) MSM were living with HIV in the UK (Figure 3), and 2,300, 5,500, 11,830, and 28,680 individuals had CD4 count ≤200, 201–350, 351–500, and >500 cells/mm3 respectively. It is thought that 10,400 (6,160–17,350) MSM were living with undiagnosed HIV, which means the undiagnosed proportion is 22% (13%–36%, Figure 3).

Methods, third paragraph of "How the Model Is Calibrated to Country Data", which defines the estimate and the range:

The median, 5th and 95th percentiles of the distribution of these model outputs at each calendar year are considered the point estimate and plausibility range limits, respectively.

Results, first paragraph of the same subsection, which says which runs the results come from:

The following results presented are those generated using the 742 parameter sets where the calibration score was <0.2 (median: 0.178, min: 0.107, max: 0.199) among the 1,000 resampled simulations.

Figure 3 legend:

Estimates of the (A) total number of MSM living with HIV in the UK and (B) total number of MSM living with undiagnosed HIV, by calendar year. Columns and error bars: Modeled median and 90% plausibility range.

Discussion, second paragraph (point estimate only):

The number of MSM living with HIV in the UK is now estimated to be 48,310 for 2013.

Discussion, final (tenth) paragraph, on what the method contributes:

This novel method allows us to also naturally produce plausibility ranges for these estimates, which describe the uncertainty surrounding the estimates, based on the data to which the model was calibrated.

What the paper's own numbers show

Method. I checked the reported estimate against its range using the paper's definition (median between the 5th and 95th percentiles; this holds for any distribution, and recompute.py.txt confirms it on four very different simulated ones). To see which of the three numbers is at fault, I summed the CD4 strata and the undiagnosed CD4 strata given in the same paragraph and recomputed the undiagnosed proportion. I checked every other estimate-with-range in the Results the same way, and expressed each as the position of the estimate within its range, (estimate − lower)/(upper − lower), which must lie between 0 and 1. Finally I read the 2013 column of the paper's Figure 3A, which the Results cite for this estimate. I fitted a straight line to the pixel rows of the eight labelled y-axis tick marks (0 to 70,000) on the PMC image ede-27-247-g004.jpg; one pixel row is 165 people, and the largest tick-mark residual is 154. I then converted the measured rows of the grey bar and the diamond. Every number below is printed by recompute.py.txt.

In the table, Results paragraphs are counted from 1 within the subsection "Calibration to Data on MSM in the UK and Resulting Estimates for 2013".

Quantity Location Reported Recomputed from the paper's own numbers
MSM living with HIV, 2013: is the estimate inside its range? Abstract; Results, 3rd paragraph 48,310 (39,900–45,560) No. 48,310 − 45,560 = 2,750 above the upper limit
Lower limit to estimate Abstract; Results, 3rd paragraph 39,900 8,410 below the estimate; possible
Required upper limit (95th percentile ≥ median) Methods, 3rd paragraph of "How the Model Is Calibrated to Country Data" 45,560 must be ≥ 48,310
Sum of CD4 strata, 2013 Results, 3rd paragraph 2,300 + 5,500 + 11,830 + 28,680 48,310, equal to the point estimate
Sum of undiagnosed CD4 strata, 2013 Results, 3rd paragraph 710 + 1,930 + 3,040 + 4,720 10,400, equal to the undiagnosed estimate
Undiagnosed proportion Results, 3rd paragraph 22% (13%–36%) 10,400 / 48,310 = 21.5%, rounds to 22%
Undiagnosed MSM, 2013 Abstract; Results, 3rd paragraph 10,400 (6,160–17,350) inside; position 0.38
Infections per year, 2010–2013 Abstract; Results, 2nd paragraph 3,210 (1,730–5,350) inside; position 0.41
Cumulative infections by end of 2012 Results, 2nd paragraph 67,720 (56,470–78,800) inside; position 0.50
Asymptomatic diagnosis rate per 3 months Results, 2nd paragraph 0.042 (0.025–0.069) inside; position 0.39
MSM living with HIV, 2013, as printed Abstract; Results, 3rd paragraph 48,310 (39,900–45,560) position 1.49
Figure 3A, 2013 column, read off Figure 3A median and 90% range drawn, no numbers median about 48,500; range about 40,000–60,400 (position 0.42)
Figure 3A, 2012 and 2011 columns, read off Figure 3A as above 2012: about 46,100 (38,200–56,400); 2011: about 43,800 (36,200–52,300)
Figure 3A minus text, 2013 Figure 3A vs Abstract — median +190, lower limit +100, upper limit +14,840
Check of the reading method: Figure 3B, undiagnosed, 2013 Figure 3B vs Results, 3rd paragraph 10,400 (6,160–17,350) read off as about 10,400 (6,100–17,400)
eAppendix row described as "the same as the main result" eAppendix eTable 6 (threshold 0.20, weights A) 121 parameter sets; 51,000 (41,400–61,000) upper limit above the median; but the set count and median differ from the main text (742 sets, 48,310)
Upper limits in all nine sensitivity scenarios eAppendix eTable 6 59,700 to 61,500 none is below 48,310

The point estimate fits every other number in the Results, and the lower limit agrees with Figure 3A to within reading precision. Only the upper limit fails, and Figure 3A, which plots the same quantity from the same model runs, puts it at about 60,400, not 45,560. The figure reading is approximate (about ±200 people), but the difference from the text is about 74 times larger than that. The same reading method applied to the 2013 column of Figure 3B (undiagnosed) gives about 10,400 (6,100–17,400), against the printed 10,400 (6,160–17,350), so the method recovers a range the text prints correctly.

Figure

The 2013 estimate and its 90% range as printed, compared with the paper's own Figure 3A and the supplement, on one axis

Figure 1. MSM living with HIV in the UK in 2013, on one axis of people (the axis starts at 30,000; no bar lengths are compared). Orange: the estimate and 90% plausibility range printed in the Abstract and Results, 48,310 (39,900–45,560). The printed upper limit lies 2,750 to the left of the estimate, which the Methods' definition (median ≤ 95th percentile) does not allow. Blue: the same quantity read from the 2013 column of the paper's Figure 3A, about 48,500 (40,000–60,400). Grey: the eAppendix eTable 6 row that the supplement describes as the main result, 51,000 (41,400–61,000), shown for context; it comes from 121 parameter sets, not the 742 the main text uses. The dashed line marks the reported estimate. The figure reading and the supplement agree that the upper limit is near 60,000.

Where each point estimate in the Results sits inside its own 90% range

Figure 2. For each estimate with a 90% range in the Results, the position of the point estimate within its range, (estimate − lower)/(upper − lower). A median must fall between 0 (the lower limit) and 1 (the upper limit). The five other ranges in the Results (blue) put the estimate at 0.38 to 0.50, as does the Figure 3A reading for the 2013 total (0.42). The printed 2013 total (orange) is at 1.49, outside its own range.

Why it matters

Explanations considered

Status

Suggested correction

  1. Replace the upper limit of the 2013 estimate in both the Abstract and the Results ("48,310 (90% plausibility range: 39,900–45,560)") with the 95th percentile of the 742 accepted runs. The paper's Figure 3A puts it at about 60,400, so the corrected text would read about "48,310 (90% plausibility range: 39,900–60,400)" if the figure is right. The authors' model output is needed for the exact value.
  2. Confirm that the lower limit (39,900) and the point estimate (48,310) were taken from the same output. Both match Figure 3A, so no change is expected.
  3. Optionally, clarify in the eAppendix how the eTable 6 row described as "the same as the main result" (121 parameter sets; 51,000, 41,400–61,000) relates to the main text's 742 runs and 48,310.
Recomputation script (python recompute.py.txt)
"""Recompute the 2013 estimate of MSM living with HIV in the UK and its 90% plausibility
range, PMC4733816 (Epidemiology 2016;27(2):247-256, doi:10.1097/EDE.0000000000000423).

Run:  python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and five candidate designs in candidates/ next to this file.

Inputs are copied from the paper as archived in PubMed Central:
  Abstract and Results ("Calibration to Data on MSM in the UK ...", first to third paragraphs)
  Methods ("How the Model Is Calibrated to Country Data", third paragraph): the point estimate is
    the median, and the range limits are the 5th and 95th percentiles of one distribution
  Figure 3A (image ede-27-247-g004.jpg, 800 x 1244 px): pixel rows measured once from the
    image and entered below; the image itself is not redrawn
  eAppendix (links.lww.com/EDE/A994), eTable 6: sensitivity analysis
"""
import os

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs from the paper
EST, LO, HI = 48310, 39900, 45560                 # Abstract; Results, 3rd paragraph
CD4_TOTAL = [2300, 5500, 11830, 28680]            # Results, 3rd paragraph: <=200, 201-350, 351-500, >500
CD4_UNDIAG = [710, 1930, 3040, 4720]              # Results, 3rd paragraph, undiagnosed
CD4_LABELS = ["CD4 ≤200", "201–350", "351–500", ">500"]
OTHER = [  # (label, location, estimate, lower, upper)
    ("Undiagnosed MSM, 2013", "Abstract; Results, 3rd paragraph", 10400, 6160, 17350),
    ("Undiagnosed proportion, 2013 (%)", "Results, 3rd paragraph", 22, 13, 36),
    ("Infections per year, 2010–2013", "Abstract; Results, 2nd paragraph", 3210, 1730, 5350),
    ("Cumulative infections by end 2012", "Results, 2nd paragraph", 67720, 56470, 78800),
    ("Asymptomatic diagnosis rate per 3 months", "Results, 2nd paragraph", 0.042, 0.025, 0.069),
]
# eAppendix eTable 6, row "threshold 0.20, weight A", which the eAppendix says "is the same as
# the main result presented in the main manuscript"
ETAB6_MAIN = dict(sets=121, est=51000, lo=41400, hi=61000)
ETAB6_ALL_HI = [60000, 60700, 60400, 61000, 61200, 60400, 61500, 59700, 61200]  # 9 scenarios
MAIN_SETS = 742                                   # Results, 1st paragraph

# Figure 3A: pixel rows of the labelled y-axis tick marks (y, value) and the 2011-2013 columns, measured on the image
GRID = [(46, 70000), (107.5, 60000), (167.5, 50000), (228.5, 40000),
        (288.5, 30000), (349.5, 20000), (411.5, 10000), (471, 0)]
FIG3A = {  # year: (top row of grey bar, bottom row of grey bar, diamond top row, diamond bottom row)
    2011: (154, 251, 201, 210),
    2012: (129, 239, 187, 196),
    2013: (105, 228, 173, 181),
}

# Figure 3B (undiagnosed), same image: y-axis tick-mark rows and the 2013 column, used as a check of the
# reading method against a range the text prints (10,400; 6,160-17,350)
GRID_B = [(696.5, 20000), (801.5, 15000), (908.5, 10000), (1015.5, 5000), (1121, 0)]
FIG3B_2013 = (752, 991, 895, 905)

# ------------------------------------------------------------------ calculations
print("=== 2013 MSM living with HIV: reported value and range ===")
print(f"Reported: {EST:,} (90% range {LO:,}-{HI:,})")
print(f"Estimate inside its own range? {LO <= EST <= HI}")
print(f"Estimate minus upper limit: {EST - HI:,}")
print(f"Estimate minus lower limit: {EST - LO:,}")
print(f"Reported range width: {HI - LO:,}")

print("\n=== Order constraint implied by the Methods (median <= 95th percentile) ===")
rng = np.random.default_rng(0)
ok = True
for name, draw in [("normal", rng.normal(48310, 5000, 10**5)),
                   ("lognormal", rng.lognormal(np.log(48310), 0.2, 10**5)),
                   ("right-skewed exponential", rng.exponential(1, 10**5)),
                   ("discrete mixture", rng.choice([1, 2, 50], size=10**5, p=[.6, .3, .1]))]:
    p5, p50, p95 = np.percentile(draw, [5, 50, 95])
    ok &= p5 <= p50 <= p95
    print(f"  {name:25s} P5={p5:10.2f} P50={p50:10.2f} P95={p95:10.2f}  ordered={p5 <= p50 <= p95}")
print(f"Ordering holds in every case: {ok} (it holds for any distribution by definition)")
print(f"So the 95th percentile must be >= {EST:,}; the printed {HI:,} is not.")

print("\n=== Is 48,310 the sound number? ===")
s_tot, s_und = sum(CD4_TOTAL), sum(CD4_UNDIAG)
print(f"Sum of CD4 strata: {' + '.join(f'{v:,}' for v in CD4_TOTAL)} = {s_tot:,}  (reported {EST:,})")
print(f"Sum of undiagnosed CD4 strata: {' + '.join(f'{v:,}' for v in CD4_UNDIAG)} = {s_und:,}  (reported 10,400)")
prop = 100 * 10400 / EST
print(f"Undiagnosed proportion 10,400 / 48,310 = {prop:.1f}%  (reported 22%)")
print(f"Mirror of the lower gap above the estimate: {EST:,} + {EST - LO:,} = {EST + (EST - LO):,} (indicative only)")

print("\n=== Every other estimate-with-range in the Results ===")
pos = {}
for lab, loc, e, lo, hi in OTHER:
    p = (e - lo) / (hi - lo)
    pos[lab] = p
    print(f"  {lab:42s} {e} ({lo}-{hi})  inside={lo <= e <= hi}  position in range={p:.2f}")
p_main = (EST - LO) / (HI - LO)
print(f"  {'MSM living with HIV, 2013':42s} {EST} ({LO}-{HI})  inside={LO <= EST <= HI}  position in range={p_main:.2f}")

print("\n=== Figure 3A read-off (paper's own chart of the same quantity) ===")
gy, gv = np.array(GRID).T
slope, icpt = np.polyfit(gy, gv, 1)
resid = gv - (slope * gy + icpt)
print(f"Calibration: {slope:.1f} people per pixel row; max tick-mark residual {np.abs(resid).max():.0f}")
fig3 = {}
for yr, (top, bot, d0, d1) in FIG3A.items():
    hi_v = slope * (top - 0.5) + icpt
    lo_v = slope * (bot + 0.5) + icpt
    med_v = slope * ((d0 + d1) / 2) + icpt
    fig3[yr] = (round(med_v, -2), round(lo_v, -2), round(hi_v, -2))
    print(f"  {yr}: median about {fig3[yr][0]:,.0f}; 90% range about {fig3[yr][1]:,.0f}-{fig3[yr][2]:,.0f}")
f_med, f_lo, f_hi = fig3[2013]
print(f"Figure 3A 2013 upper limit minus text upper limit: about {f_hi - HI:,.0f}")
print(f"Figure 3A 2013 lower limit minus text lower limit: about {f_lo - LO:,.0f}")
print(f"Figure 3A 2013 median minus text estimate: about {f_med - EST:,.0f}")
print(f"Reading precision: one pixel row = {abs(slope):.0f} people")
print(f"Figure 3A 2013 range width: about {f_hi - f_lo:,.0f}; above the reported estimate: about {f_hi - EST:,.0f}")
print(f"Printed range width: {HI - LO:,}; text-vs-figure upper-limit gap / 200-person reading error = {(f_hi - HI) / 200:.0f}")
gyb, gvb = np.array(GRID_B).T
sb, ib = np.polyfit(gyb, gvb, 1)
t, bt, d0, d1 = FIG3B_2013
print(f"Check on Figure 3B (undiagnosed, 2013): median about {round(sb * (d0 + d1) / 2 + ib, -2):,.0f}; "
      f"range about {round(sb * (bt + 0.5) + ib, -2):,.0f}-{round(sb * (t - 0.5) + ib, -2):,.0f} "
      f"(text: 10,400; 6,160-17,350); one pixel row = {abs(sb):.0f} people")

print("\n=== eAppendix eTable 6 ===")
print(f"Row described as the main result: {ETAB6_MAIN['sets']} parameter sets, "
      f"{ETAB6_MAIN['est']:,} ({ETAB6_MAIN['lo']:,}-{ETAB6_MAIN['hi']:,}); main text: {MAIN_SETS} sets, {EST:,} ({LO:,}-{HI:,})")
print(f"Upper limits across all 9 sensitivity scenarios: {min(ETAB6_ALL_HI):,}-{max(ETAB6_ALL_HI):,}")
print(f"Number of scenarios with upper limit below {EST:,}: {sum(h < EST for h in ETAB6_ALL_HI)}")

# ------------------------------------------------------------------ figure style
BG, INK, INK2, GRIDC = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GREY = "#eb6834", "#2a78d6", "#9a9893"
plt.rcParams.update({
    "font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "text.color": INK, "axes.spines.top": False, "axes.spines.right": False,
    "svg.fonttype": "none",
})
W = 7.2


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg", bbox_inches="tight")
    fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
    plt.close(fig)


def kfmt(ax):
    ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:,.0f}"))


# ---- Candidate A: dot-and-interval rows on one axis --------------------------------------
def cand_a(path):
    fig, ax = plt.subplots(figsize=(W, 3.9))
    rows = [
        ("Reported in Abstract and Results", ORANGE, EST, LO, HI, "o"),
        ("Paper's Figure 3A, 2013 column (read off)", BLUE, f_med, f_lo, f_hi, "D"),
        ("eAppendix eTable 6, row called the main result", GREY, ETAB6_MAIN["est"],
         ETAB6_MAIN["lo"], ETAB6_MAIN["hi"], "s"),
    ]
    ys = [3, 1.7, 0.4]
    ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
    ax.text(EST, 3.72, f"reported estimate {EST:,}", ha="center", va="bottom", color=INK2, fontsize=10,
            bbox=dict(fc=BG, ec="none", pad=1))
    for (lab, col, e, lo, hi, m), y in zip(rows, ys):
        ax.plot([lo, hi], [y, y], color=col, lw=2.5, solid_capstyle="butt", zorder=2)
        for x in (lo, hi):
            ax.plot([x, x], [y - 0.12, y + 0.12], color=col, lw=2, zorder=2)
        ax.plot(e, y, m, color=col, ms=9, mec=BG, mew=1.5, zorder=3)
        ax.text(30500, y + 0.36, lab, ha="left", va="center", fontsize=10.5, color=INK, zorder=4,
                bbox=dict(fc=BG, ec="none", pad=1.5))
        ax.text(max(hi, e) + 900, y, f"{e:,.0f} ({lo:,.0f}\u2013{hi:,.0f})", ha="left", va="center",
                fontsize=10, color=INK2)
    # gap between the printed upper limit and the estimate
    ax.annotate("", xy=(EST, 3 - 0.36), xytext=(HI, 3 - 0.36),
                arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=1.5, shrinkA=0, shrinkB=0))
    ax.text(EST + 900, 3 - 0.36, f"upper limit is {EST - HI:,} below the estimate",
            ha="left", va="center", fontsize=10, color=INK)
    ax.set_ylim(-0.05, 4.05)
    ax.set_xlim(30000, 72000)
    ax.set_yticks([])
    ax.spines["left"].set_visible(False)
    ax.grid(axis="x", color=GRIDC, lw=0.8)
    ax.set_axisbelow(True)
    kfmt(ax)
    ax.set_xlabel("MSM living with HIV in the UK, 2013 (median and 90% plausibility range)")
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=ORANGE, marker="o", lw=2.5, ms=8, label="Reported in the text"),
         Line2D([], [], color=BLUE, marker="D", lw=2.5, ms=8, label="Read from the paper's own Figure 3A"),
         Line2D([], [], color=GREY, marker="s", lw=2.5, ms=8, label="Supplement, eTable 6 (121 sets)")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3, frameon=False,
              fontsize=10, handlelength=2.2, columnspacing=1.2)
    save(fig, path)


# ---- Candidate B: one number line with the order constraint -----------------------------
def cand_b(path):
    fig, ax = plt.subplots(figsize=(W, 2.6))
    ax.set_xlim(36000, 52000)
    ax.set_ylim(-1.3, 1.75)
    ax.axhline(0, color=INK2, lw=1)
    marks = [(LO, "lower limit\n(5th percentile)", 39900), (HI, "upper limit\n(95th percentile)", 45560),
             (EST, "point estimate\n(median)", 48310)]
    for x, lab, _ in marks:
        ax.plot(x, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5, zorder=3)
        ax.text(x, 0.28, f"{x:,}", ha="center", va="bottom", fontsize=11, fontweight="bold")
        ax.text(x, -0.25, lab, ha="center", va="top", fontsize=10, color=INK2)
    ax.annotate("", xy=(HI, 0.95), xytext=(EST, 0.95), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
    ax.text((HI + EST) / 2, 1.03, f"the 95th percentile is {EST - HI:,} below the median",
            ha="center", va="bottom", fontsize=10.5)
    ax.text(36100, -1.2, "Required by the Methods' definition, for any distribution:  "
            "5th percentile ≤ median ≤ 95th percentile", ha="left", va="bottom", fontsize=10, color=BLUE)
    ax.text(36100, 1.6, "Reported values (Abstract; Results), in order along one axis", ha="left", va="bottom", fontsize=10, color=INK2)
    ax.set_yticks([])
    for s in ("left", "bottom"):
        ax.spines[s].set_visible(False)
    ax.set_xticks([])
    save(fig, path)


# ---- Candidate C: text panels joined by an annotated connector --------------------------
def cand_c(path):
    fig, ax = plt.subplots(figsize=(W, 5.0))
    ax.set_xlim(0, 10)
    ax.set_ylim(0, 6.85)
    ax.axis("off")

    def panel(x, y, w, h, title, body, edge):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
                                    fc="#ffffff", ec=edge, lw=1.6))
        ax.text(x + 0.18, y + h - 0.2, title, ha="left", va="top", fontsize=10, color=INK2, fontweight="bold")
        ax.text(x + 0.18, y + h - 0.62, body, ha="left", va="top", fontsize=10.5, color=INK, linespacing=1.4)

    panel(0.1, 3.6, 4.7, 3.15, "Abstract; Results, 3rd paragraph",
          "\u201cIn 2013, 48,310 (90% plausibility\nrange: 39,900\u201345,560) men who\n"
          "have sex with men were estimated\nto be living with HIV in the UK\u201d", ORANGE)
    panel(5.2, 3.6, 4.7, 3.15, "Methods, model calibration",
          "\u201cThe median, 5th and 95th\npercentiles of the distribution of\n"
          "these model outputs ... are\nconsidered the point estimate and\nplausibility range limits\u201d", INK2)
    panel(0.1, 0.1, 4.7, 2.5, "Results, 3rd paragraph: CD4 strata",
          "2,300 + 5,500 + 11,830 + 28,680\n= 48,310, the point estimate\n"
          "10,400 / 48,310 = 21.5%,\nthe stated 22% undiagnosed", BLUE)
    panel(5.2, 0.1, 4.7, 2.5, "What the definition requires",
          "median \u2264 95th percentile, so the\nupper limit must be \u2265 48,310\n"
          "printed upper limit 45,560\nfalls 2,750 short", BLUE)
    for xc in (2.45, 7.55):
        ax.plot([xc, xc], [3.6, 3.32], color=ORANGE, lw=1.6)
        ax.plot([xc, xc], [2.78, 2.6], color=BLUE, lw=1.6, ls=(0, (2, 2)))
    ax.text(5.0, 3.05, "conflict: median 48,310 > 95th percentile 45,560",
            ha="center", va="center", fontsize=10.5, color=INK, fontweight="bold",
            bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=1.5))
    save(fig, path)


# ---- Candidate D: where each estimate sits inside its own range -------------------------
def cand_d(path):
    labs = [o[0] for o in OTHER] + ["MSM living with HIV, 2013 (Fig. 3A read-off)",
                                    "MSM living with HIV, 2013 (as printed)"]
    vals = [pos[o[0]] for o in OTHER] + [(f_med - f_lo) / (f_hi - f_lo), p_main]
    cols = [BLUE] * (len(OTHER) + 1) + [ORANGE]
    fig, ax = plt.subplots(figsize=(W, 3.6))
    y = np.arange(len(labs))[::-1]
    ax.axvspan(0, 1, color="#f0efeb", zorder=0)
    for xv, t in ((0, "lower limit"), (1, "upper limit")):
        ax.axvline(xv, color=INK2, lw=1, zorder=1)
        ax.text(xv, len(labs) - 0.35, t, ha="center", va="bottom", fontsize=10, color=INK2,
                bbox=dict(fc=BG, ec="none", pad=1.5))
    for yi, v, c, lab in zip(y, vals, cols, labs):
        ax.plot([0, 1], [yi, yi], color=GRIDC, lw=1, zorder=1)
        ax.plot(v, yi, "o", color=c, ms=9, mec=BG, mew=1.5, zorder=3)
        ax.text(v + 0.04, yi, f"{v:.2f}", va="center", ha="left", fontsize=10, color=INK2)
    ax.set_yticks(y)
    ax.set_yticklabels(labs, fontsize=10)
    ax.tick_params(axis="y", length=0)
    ax.spines["left"].set_visible(False)
    ax.set_xlim(-0.1, 1.7)
    ax.set_ylim(-0.6, len(labs) - 0.1)
    ax.set_xticks([0, 0.5, 1, 1.5])
    ax.set_xlabel("Position of the point estimate within its own 90% range\n"
                  "(estimate − lower) / (upper − lower); must lie between 0 and 1")
    from matplotlib.lines import Line2D
    h = [Line2D([], [], color=ORANGE, marker="o", lw=0, ms=8, label="Reported 2013 total, as printed"),
         Line2D([], [], color=BLUE, marker="o", lw=0, ms=8, label="Other ranges in the paper, and the Fig. 3A read-off")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.3, -0.3), ncol=2, frameon=False, fontsize=10)
    save(fig, path)


# ---- Candidate E: calculation flow, CD4 strata summed against the range -----------------
def cand_e(path):
    fig, ax = plt.subplots(figsize=(W, 2.9))
    left = 0
    shades = ["#9cc0ea", "#6fa3e2", "#4a8cdc", BLUE]
    for v, lab, c in zip(CD4_TOTAL, CD4_LABELS, shades):
        ax.barh(2, v, left=left, height=0.5, color=c, ec=BG, lw=2)
        if v > 10000:
            ax.text(left + v / 2, 2, f"{lab}\n{v:,}", ha="center", va="center", fontsize=10,
                    color="#ffffff" if c in (BLUE, "#4a8cdc") else INK)
        left += v
    ax.text(0, 2.42, f"Sum of the four reported CD4 strata = {s_tot:,}", ha="left", va="bottom", fontsize=10.5)
    ax.text(0, 1.68, f"{CD4_LABELS[0]}: {CD4_TOTAL[0]:,}   {CD4_LABELS[1]}: {CD4_TOTAL[1]:,}",
            ha="left", va="top", fontsize=10, color=INK2)
    ax.plot([LO, HI], [1, 1], color=ORANGE, lw=2.5)
    for x in (LO, HI):
        ax.plot([x, x], [0.85, 1.15], color=ORANGE, lw=2)
    ax.plot(EST, 1, "o", color=ORANGE, ms=9, mec=BG, mew=1.5)
    ax.text(0, 1, "Reported 90% range\n39,900–45,560 and estimate", ha="left", va="center", fontsize=10.5)
    ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)))
    ax.text(EST + 400, 0.5, f"{EST:,}", ha="left", va="center", fontsize=10, color=INK2)
    ax.set_xlim(0, 60000)
    ax.set_ylim(0.3, 2.85)
    ax.set_yticks([])
    ax.spines["left"].set_visible(False)
    ax.grid(axis="x", color=GRIDC, lw=0.8)
    ax.set_axisbelow(True)
    kfmt(ax)
    ax.set_xlabel("MSM living with HIV in the UK, 2013")
    ax.text(HI - 200, 1.22, "upper limit ends 2,750 short of the estimate", ha="right", va="bottom",
            fontsize=10, color=INK)
    save(fig, path)


cand_a(os.path.join(CAND, "cand_A_interval_rows"))
cand_b(os.path.join(CAND, "cand_B_order_line"))
cand_c(os.path.join(CAND, "cand_C_text_panels"))
cand_d(os.path.join(CAND, "cand_D_position_in_range"))
cand_e(os.path.join(CAND, "cand_E_cd4_flow"))
cand_a(os.path.join(HERE, "fig1"))
cand_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)

Abstract reports a 31% fall in endotoxin activity on the prudent diet; Results report 38%

Gastroenterology (2012) · doi:10.1053/j.gastro.2012.01.034 · PMC3978718

Summary

This short crossover study fed 8 healthy inpatients a Western-style diet and a prudent-style diet for 1 month each and measured plasma endotoxin activity before and after each diet. The abstract says the prudent-style diet reduced endotoxin activity by 31%. The Results paragraph that cites Figure 1 says the decrease was 38%. Both passages describe the same subjects, assay and before/after comparison, and both give the same +71% for the Western-style diet. Rounding cannot produce a 7-point gap. The paper prints no group means for endotoxin activity, only the individual-subject plots in Figure 1. Reading those plots by eye gives a 37.5% fall in the group mean, which supports the Results figure. The abstract's 31% matches no single summary method that also gives +71% for the Western diet. The abstract is the number most likely to be quoted, so the size of the prudent-diet effect is now ambiguous in the literature. The direction of the effect, the Western-diet result and the paper's conclusions are unaffected.

What the paper reports

Abstract:

Placing 8 healthy subjects on a Western-style diet for 1 month induced a 71% increase in plasma levels of endotoxin activity (endotoxemia), whereas a prudent-style diet reduced levels by 31%.

Main text, Results paragraph (the paragraph that cites Figure 1 and Table 1):

Our study shows that 4 weeks of feeding a Western-style diet induced highly significant increases (by 71%) in plasma endotoxin activity levels, in contrast to a major decrease (by 38%) after the prudent-style diet (Figure 1) (P < .001).

Main text, same paragraph:

Baseline endotoxin activity levels were about identical

Main text, study description. This shows that both passages concern one group of subjects and one design:

Briefly, 8 healthy subjects were fed a Western-style and a “prudent-style” diet for 1 month as inpatients hospitalized in a metabolic ward in a crossover study separated by a 1-month washout period.

Figure 1 legend:

Data from 8 individual subjects shown as endotoxin activity units for the Western-style diet (WD) and the prudent-style diet (PD). Paired t tests were used to compare effects of the diets on endotoxin activity levels. The Mann–Whitney sum test was used to compare effects of the 2 diets on endotoxin activity levels.

In the Figure 1 image, panels A (Western diet, before vs after) and B (prudent diet, before vs after) are labelled "P≤.05". Panel D, which compares the two diets after the intervention, is labelled "P≤.001". Neither the text nor the figure gives a group mean, SD or percentage for endotoxin activity. Table 1 gives cytokine concentrations only, as before and after means with a "Change (%)" column.

What the paper's own numbers show

Method. The paper reports the endotoxin results only as two percentages and as individual-subject lines in Figure 1, panels A and B (8 subjects each). I read each subject's before and after values off the enlarged figure image by eye. Each reading is uncertain by about ±0.005 activity units. These readings are approximations, not the authors' data. From them I computed the percentage change in four standard ways: - the change in the group mean (mean after / mean before − 1); - the mean of each subject's own percentage change; - the geometric mean of the after/before ratios; - the change in the median.

To carry the reading error through, I added independent uniform ±0.005 noise to every point, repeated this 20,000 times and took the 2.5–97.5 percentile band of each statistic. As a check on which method the authors used elsewhere, I recomputed Table 1's cytokine percentages from its rounded means. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
Prudent diet, change in endotoxin activity Abstract −31% — (same quantity as the next row)
Prudent diet, change in endotoxin activity Results paragraph citing Fig. 1 −38% —
Gap between the two printed values Abstract vs Results — 7 percentage points (23% of the abstract value); post/baseline ratio 0.69 vs 0.62
Western diet, change in endotoxin activity Abstract and Results +71% (both) —
Prudent diet, group mean before → after Fig. 1B (read by eye) not reported 0.115 → 0.072 units
Western diet, group mean before → after Fig. 1A (read by eye) not reported 0.120 → 0.207 units
Prudent diet, change of group means Fig. 1B −31% / −38% −37.5% (band −39.5 to −35.4)
Western diet, change of group means Fig. 1A +71% +72.2% (band +69.0 to +75.5)
Prudent diet, mean of per-subject % changes Fig. 1B — −33.3% (band −35.9 to −30.4)
Western diet, mean of per-subject % changes Fig. 1A — +80.2% (band +75.6 to +85.3)
Prudent / Western, geometric mean ratio Fig. 1A–B — −35.7% (−38.4 to −33.1) / +71.3% (+66.9 to +75.9)
Prudent / Western, change of medians Fig. 1A–B — −28.7% (−33.5 to −24.1) / +78.3% (+71.8 to +85.5)
Within-diet paired t test, PD / WD Results text; Fig. 1A–B labels "P < .001" (text); "P≤.05" (figure) p ≈ 0.020 / 0.017 (from figure readings)
Table 1, PD cytokines: TNF-α, MCP-1, IL-6, IL-8 Table 1 −17, −9, −11, −28% −16.0, −8.7, −12.5, −26.3% (change of rounded means)
Table 1, WD cytokines: TNF-α, MCP-1, IL-6, IL-8 Table 1 −1, −9, −2, −13% 0.0, −9.1, 0.0, −11.8% (change of rounded means)

Three findings stand out: - The change in the group mean, read off Figure 1, reproduces both the Results' −38% and the shared +71%. - The geometric mean ratio also gives values close to −38% and +71%. - The only methods whose bands reach −31% are the mean of per-subject changes and the change of medians. Both give +78% to +80% for the Western diet, not the +71% printed beside the 31% in the abstract.

Table 1's printed percentages agree, within the rounding of its means, with the change-of-means method. This fits the 38% value, although rounded means cannot rule out other methods.

Figure

Abstract and Results excerpts giving different prudent-diet decreases for the same comparison

The two passages that report the prudent-diet result, as printed in the archived article. In orange are the paper's values: the abstract gives a 31% decrease and the Results paragraph citing Figure 1 gives a 38% decrease. Both concern the same 8 subjects, the same neutrophil-priming assay and the same before/after comparison, and both pair the value with an identical +71% for the Western-style diet. The footer gives the change of group means computed in recompute.py.txt from Figure 1 values read by eye.

Percent change in endotoxin activity under four summary methods, compared with the printed values

Percentage change in endotoxin activity recomputed from the individual values in Figure 1, read by eye (blue dots), under four summary methods. The bars show the 2.5–97.5 percentile range when each reading is perturbed by ±0.005 units. The dashed orange lines are the printed values: −31% (abstract), −38% (Results) and +71% (both). The left cluster is the prudent diet and the right cluster the Western diet, on one shared axis. The change of group means and the geometric mean ratio match −38% and +71% together. No method matches −31% and +71% together.

Why it matters

The abstract is where readers, reviews and meta-analyses usually take effect sizes from. Here it gives a prudent-diet reduction (31%, ratio 0.69) that differs from the body (38%, ratio 0.62) by 7 percentage points, or about 23% of the abstract value. A reader who compares the two diets gets a different contrast depending on the source: a post-diet ratio of 1.71/0.69 ≈ 2.48 between diets if the abstract is right, against 1.71/0.62 ≈ 2.76 if the body is right. Because the paper prints no group means or SDs for endotoxin activity, a reader cannot tell from the article which value is correct. Anyone sizing a follow-up study or pooling this result has to choose between two figures the paper gives for the same quantity.

The error is limited in scope: - The Western-diet +71%, the result named in the title, is the same in both places. - The prudent diet lowers endotoxin activity in both statements. - The cytokine results in Table 1 and the paper's qualitative interpretation do not depend on which value is correct.

A smaller, related point concerns the P value. The Results sentence attaches "P < .001" to the within-diet changes. The Figure 1 panels for those changes (A and B) are labelled "P≤.05", and the figure readings give paired-t p-values of about 0.02. The "P≤.001" label appears only on panel D, the between-diet comparison after the intervention. The Results sentence may be citing the panel D value. As written, though, it reads as applying to the within-diet changes.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The authors could state which value is correct for the change in plasma endotoxin activity after the prudent-style diet. They could also give the group mean (SD) endotoxin activity before and after each diet, and say how the percentage change was calculated (for example, change of group means or mean of per-subject changes). If the Results value is correct, the abstract would read: "…whereas a prudent-style diet reduced levels by 38%." If the abstract value is correct, the Results sentence would need the same correction. The authors could also say which comparison the "P < .001" in the Results sentence refers to, since Figure 1 labels the within-diet changes "P≤.05" and only the between-diet comparison after the intervention "P≤.001".

Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC3978718 write-up and regenerate fig1/fig2.

Run:  python recompute.py.txt   (needs numpy, scipy, matplotlib)

Inputs
------
* Reported values (paper): abstract -31% (prudent diet, PD), +71% (Western
  diet, WD); Results paragraph citing Figure 1: -38% (PD), +71% (WD);
  Table 1 cytokine means before/after and printed % changes.
* Individual endotoxin activity values: the paper gives these only as plotted
  points in Figure 1 (panels A and B, 8 subjects each). The values below were
  read by eye from the enlarged figure image (PMC file nihms561466f1.jpg);
  reading error is roughly +/-0.005 units per point. They are approximations,
  not the authors' data.
"""
import os
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- reported
REP = {"abstract_PD": -31, "results_PD": -38, "abstract_WD": 71, "results_WD": 71}

# ------------------------------------------------- Figure 1, read by eye
# subject order (by plotting symbol): light-blue '+', purple 'x', blue diamond,
# orange circle, green triangle, brown, cyan, red square
pd_before = np.array([0.246, 0.138, 0.132, 0.120, 0.080, 0.077, 0.067, 0.062])
pd_after  = np.array([0.120, 0.089, 0.080, 0.0625, 0.094, 0.047, 0.050, 0.034])
wd_before = np.array([0.235, 0.130, 0.172, 0.125, 0.105, 0.078, 0.050, 0.068])
wd_after  = np.array([0.290, 0.375, 0.210, 0.200, 0.275, 0.105, 0.100, 0.103])

CONV = {
    "Change of group means":        lambda b, a: (a.mean() / b.mean() - 1) * 100,
    "Mean of per-subject % changes": lambda b, a: np.mean(a / b - 1) * 100,
    "Geometric mean ratio":         lambda b, a: (np.exp(np.mean(np.log(a / b))) - 1) * 100,
    "Change of medians":            lambda b, a: (np.median(a) / np.median(b) - 1) * 100,
}

def jitter_band(b, a, f, n=20000, half=0.005, seed=0):
    rng = np.random.default_rng(seed)
    vals = np.empty(n)
    for i in range(n):
        bb = b + rng.uniform(-half, half, b.size)
        aa = a + rng.uniform(-half, half, a.size)
        vals[i] = f(bb, aa)
    return np.percentile(vals, [2.5, 97.5])

print("=== Figure 1 (read by eye), endotoxin activity units ===")
for name, b, a in [("Prudent (PD)", pd_before, pd_after), ("Western (WD)", wd_before, wd_after)]:
    t = stats.ttest_rel(a, b)
    print(f"{name}: mean before {b.mean():.3f}, after {a.mean():.3f}; paired t p = {t.pvalue:.3f}")
print("Paper reports: PD -31% (abstract), -38% (Results); WD +71% (both)")

RES = {}
print("\n=== % change under each summary convention (2.5-97.5% band from +/-0.005 reading error) ===")
for cname, f in CONV.items():
    pdv, wdv = f(pd_before, pd_after), f(wd_before, wd_after)
    pdb, wdb = jitter_band(pd_before, pd_after, f), jitter_band(wd_before, wd_after, f)
    RES[cname] = (pdv, pdb, wdv, wdb)
    print(f"{cname:32s} PD {pdv:+6.1f} [{pdb[0]:+.1f}, {pdb[1]:+.1f}]   "
          f"WD {wdv:+6.1f} [{wdb[0]:+.1f}, {wdb[1]:+.1f}]")

print("\n=== Which conventions reproduce the printed pairs? (|recomputed - reported| <= band) ===")
for cname, (pdv, pdb, wdv, wdb) in RES.items():
    ok31 = pdb[0] - 0.5 <= -31 <= pdb[1] + 0.5
    ok38 = pdb[0] - 0.5 <= -38 <= pdb[1] + 0.5
    ok71 = wdb[0] - 0.5 <= 71 <= wdb[1] + 0.5
    print(f"{cname:32s} PD=-31? {ok31!s:5s} PD=-38? {ok38!s:5s} WD=+71? {ok71!s:5s}")

print("\n=== Gap between the two printed PD values ===")
print(f"38 - 31 = {38-31} percentage points; relative to abstract value: {(38-31)/31*100:.0f}%")
print(f"post/baseline ratio: abstract {1-0.31:.2f}, Results {1-0.38:.2f}")
print(f"Diet contrast (post/baseline WD / PD): abstract {1.71/0.69:.2f}, Results {1.71/0.62:.2f}")
print(f"PD decrease expressed relative to post-diet mean (change of means): "
      f"{(pd_before.mean()-pd_after.mean())/pd_after.mean()*100:.0f}%")
print(f"Rounding: any value in [-31.5, -30.5) prints as 31%, any in [-38.5, -37.5) as 38%; the intervals do not overlap.")

print("\n=== Table 1: convention check (change of rounded means vs printed % change) ===")
T1 = [("PD TNF-a", 10, 8.4, -17), ("PD MCP-1", 195, 178, -9), ("PD IL-6", 1.6, 1.4, -11), ("PD IL-8", 19, 14, -28),
      ("WD TNF-a", 9, 9, -1), ("WD MCP-1", 198, 180, -9), ("WD IL-6", 1.8, 1.8, -2), ("WD IL-8", 17, 15, -13)]
for n, b, a, rep in T1:
    print(f"{n:9s} before {b:>5} after {a:>5}: change of means {(a/b-1)*100:+6.1f}%  printed {rep:+d}%")

# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

BG, INK, INK2, ORG, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
                     "text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2})

def fmt(v):
    return f"{v:+.1f}".replace("-", "\u2212")

# ---------- fig1: abstract vs Results text panels with connector
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")

def panel(x, title, lines, pdtxt):
    ax.add_patch(FancyBboxPatch((x, 15), 42, 42, boxstyle="round,pad=0.4,rounding_size=1.5",
                                fc="white", ec="#c9c8c3", lw=1.2))
    ax.text(x + 2, 54.5, title, fontsize=11, fontweight="bold", va="top", color=INK)
    y = 48.5
    for ln in lines:
        ax.text(x + 2, y, ln, fontsize=10, va="top", color=INK2)
        y -= 4.3
    ax.text(x + 21, 23.5, pdtxt, fontsize=17, fontweight="bold", ha="center", va="center", color=ORG)
    ax.text(x + 21, 18.3, "Western diet: +71%", fontsize=10.5, ha="center", va="center", color=ORG)

panel(3, "Abstract", ["\u201cPlacing 8 healthy subjects on a",
                      "Western-style diet for 1 month",
                      "induced a 71% increase \u2026",
                      "whereas a prudent-style diet",
                      "reduced levels by 31%.\u201d"],
      "Prudent diet: \u221231%")
panel(55, "Results (paragraph citing Fig. 1)", ["\u201c\u2026 increases (by 71%) in plasma",
                      "endotoxin activity levels, in contrast",
                      "to a major decrease (by 38%) after",
                      "the prudent-style diet (Figure 1)",
                      "(P < .001).\u201d"],
      "Prudent diet: \u221238%")
ax.annotate("", xy=(54.2, 23.5), xytext=(45.8, 23.5),
            arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(50, 26.5, "conflict", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(50, 6.8, "Same 8 subjects, same assay, same before/after comparison: the prudent-diet decrease is printed\n"
        f"as 31% in the abstract and 38% in the Results (gap {38-31} percentage points); the Western-diet value agrees.\n"
        f"Figure 1, read by eye, change of group means: prudent {fmt(RES['Change of group means'][0])}%, "
        f"Western {fmt(RES['Change of group means'][2])}% (shown in fig. 2).",
        ha="center", va="center", fontsize=9.5, color=INK, linespacing=1.5)
ax.text(50, 61, "Values in orange are as printed in the paper", ha="center", va="center", fontsize=10, color=ORG)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
    fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)

# ---------- fig2: conventions vs printed values, one axis
fig, ax = plt.subplots(figsize=(7.2, 4.0))
names = list(RES)
ys = np.arange(len(names))[::-1].astype(float)
top = len(names) - 0.5
ax.axvline(0, color=INK2, lw=0.8)
for v, lab, ha in [(-31, "abstract \u221231%", "left"), (-38, "Results \u221238%", "right"),
                   (71, "abstract and\nResults +71%", "right")]:
    ax.plot([v, v], [-0.5, top], color=ORG, lw=2, ls=(0, (4, 2)))
    ax.text(v + (0.8 if ha == "left" else -0.8), top + 0.05, lab, color=ORG, fontsize=9.5, ha=ha, va="bottom")
ax.text(-35, top + 0.75, "Prudent diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
ax.text(75, top + 0.75, "Western diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
for y, n in zip(ys, names):
    pdv, pdb, wdv, wdb = RES[n]
    for v, b in [(pdv, pdb), (wdv, wdb)]:
        ax.plot(b, [y, y], color=BLU, lw=2, solid_capstyle="butt")
        ax.plot(v, y, "o", ms=8, color=BLU)
    ax.text(-58, y, fmt(pdv), color=BLU, fontsize=10, ha="left", va="center")
    ax.text(110, y, fmt(wdv), color=BLU, fontsize=10, ha="right", va="center")
ax.set_yticks(ys); ax.set_yticklabels(names, color=INK)
ax.set_xlim(-60, 112); ax.set_ylim(-0.6, top + 1.35)
ax.set_xlabel("% change in endotoxin activity over 4 weeks on each diet")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
    ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o-", color=BLU, ms=8, lw=2, label="recomputed from Figure 1 values read by eye (bar: 95% reading-error band)")
ax.plot([], [], color=ORG, lw=2, ls=(0, (4, 2)), label="value printed in the paper")
ax.legend(loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
    fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print("\nfigures written to", HERE)

Abstract reverses the direction of the paper's own DDIT4 and gastric cancer survival finding

Gastroenterology (2020) · doi:10.1053/j.gastro.2020.09.016 · PMC7857017

Summary

The abstract states that "Lower DDIT4 expression correlated with longer survival" in gastric cancer. The rest of the paper reports the opposite for the only gastric cancer survival cohort it analyses, a tissue microarray (TMA) of 509 patients. The Introduction, Results, Figure 4 title and Discussion all say that high DDIT4 goes with longer survival, and the Kaplan-Meier curves in Figure 4B agree: the DDIT4-positive group (n = 200) lies above the DDIT4-negative group (n = 309), log-rank P = .021. The reversed sentence is also in the copyedited abstract indexed in PubMed, and no correction has been issued, so readers who see only the abstract get the direction of the paper's main human result backwards.

What the paper reports

Abstract, Results (PMC author manuscript):

Lower DDIT4 expression correlated with longer survival of gastric cancer patients.

Abstract, Results, as deposited in PubMed (PMID 32956680) and Europe PMC, in the journal's copyedited wording:

Lower DDIT4 expression correlated with longer survival of patients with gastric cancer.

Abstract, Conclusions:

During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation correlates with increased proliferation of cells harboring DNA damage, as well as increased tumor formation and growth in mice and humans.

Introduction, final (5th) paragraph:

Patients with low expression have poorer survival.

Results, section "Loss of DDIT4 causes cell-autonomous increased growth of human gastric…", paragraph 1:

We examined another annotated tissue microarray on which we had compiled 509 patients with gastric cancer. We saw statistically significant correlation of high DDIT4 expression with increased patient survival and more differentiated tumors, especially in patients who had received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage (Figure 4A–C; Table S3).

Figure 4 title:

In humans, loss of DDIT4 correlates with worse gastric cancer survival and causes cell-autonomous increased mTORC1 and proliferation despite chemotherapy

Figure 4 legend, panel B:

Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an event defined as a cancer-related death.

Discussion, paragraph 3:

Our results showing DDIT4 expression correlating with less proliferation in precancerous lesions and better survival of patients with gastric cancer confirm and extend on those of a previous report43.

Discussion, paragraph 4:

Paradoxically, however, studies across multiple organs have shown disparate results for DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, 47–50.

Figure 4B and 4C, legend boxes printed on the image (read from the PMC image file nihms-1661019-f0004.jpg, not from the text):

4B, Overall Patients: DDIT4 negative (n=309); DDIT4 positive (n=200); P = .021

4C, Patients receiving chemotherapy: DDIT4 negative (n=263); DDIT4 positive (n=170); P = .016

What the paper's own numbers show

Method. The paper reports no hazard ratio or survival estimate in its text, so there is no estimate to recompute. Instead I recorded the direction of the DDIT4 and survival association in every place the paper describes it, and checked the group sizes printed on Figure 4B and 4C against the cohort size in the Results. All quotes from the paper's text were matched verbatim against the PMC-archived text, and recompute.py.txt repeats this check for every one of them; the PubMed wording was checked against the PubMed efetch record, and the Figure 4B and 4C counts were read from the figure image. In Figure 4B and 4C the DDIT4-positive curve lies above the DDIT4-negative curve over most of follow-up. In this paper, "positive" is the high-expression group (Figure 4A shows cores scored "negative" or "positive").

Quantity Location Reported Recomputed / what the paper's own material shows
Direction of DDIT4 and survival Abstract, Results (PMC and PubMed) Low DDIT4 → longer survival Opposite to every statement below and to Figure 4B
Direction of DDIT4 and survival Introduction, final (5th) paragraph Low DDIT4 → poorer survival High DDIT4 → longer survival
Direction of DDIT4 and survival Results, 509-patient TMA High DDIT4 → increased survival High DDIT4 → longer survival
Direction of DDIT4 and survival Figure 4 title Loss of DDIT4 → worse survival High DDIT4 → longer survival
Direction of DDIT4 and survival Discussion, paragraph 3 DDIT4 expression → better survival High DDIT4 → longer survival
Direction of DDIT4 and survival Figure 4B and 4C curves Positive curve above negative curve High DDIT4 → longer survival
Body statements that agree Intro, Results, Fig 4 title, Discussion, Fig 4B n/a 5 of 5 agree with each other; the abstract contradicts all 5
Cohort size Results text vs Figure 4B 509 patients; 309 + 200 309 + 200 = 509 (matches)
Chemotherapy subgroup Figure 4C 263 + 170 433 patients, 85.1% of the 509
DDIT4-positive share Figure 4B / 4C 200 of 509; 170 of 433 39.3% in both
Log-rank P Figure 4B / 4C .021 / .016 Not recomputable from published material (no individual data)

Figure

The abstract sentence (orange) and the five places in the body of the paper (blue) that describe the same 509-patient survival result, joined by a connector marking the opposite direction

Figure 1. On the left (orange outline) is the abstract's Results sentence, quoted from the PMC author manuscript; the same sentence, in copyedited wording ("…of patients with gastric cancer"), is in the publisher-deposited abstract in PubMed. On the right (blue outline) are short excerpts from the Introduction, the Results, the Figure 4 title and the Discussion, plus the group sizes and log-rank P value printed on Figure 4B. They describe the same 509-patient gastric cancer tissue microarray (the Introduction sentence does not name the cohort, but the 509-patient TMA is the paper's only survival analysis) and the same endpoint (overall survival, with cancer-related death as the event). The abstract states that low DDIT4 goes with longer survival. Every body statement, and the Kaplan-Meier data, state or show that high DDIT4 goes with longer survival. The counts come from recompute.py.txt, which transcribes them from the Figure 4 image. The paper's own figure is not reproduced.

Why it matters

Explanations considered

Status

Suggested correction

  1. Change the abstract's Results sentence so that it matches the body of the paper and Figure 4B, for example: "Higher DDIT4 expression correlated with longer survival of patients with gastric cancer." The equivalent "Lower DDIT4 expression correlated with shorter survival…" would also work.
  2. Ideally, add the effect size to the abstract and Results (for example a hazard ratio with a 95% CI from a Cox model, alongside the log-rank P = .021 for the 509 patients), so the size and direction of the association can be read without the figure.
  3. Ask PubMed and Europe PMC to update their abstract records when the correction is published, since those are the copies most readers see.
Recomputation script (python recompute.py.txt)
"""Check the direction of the DDIT4-survival association stated in PMC7857017
(Gastroenterology 2020, doi:10.1053/j.gastro.2020.09.016).

Run:  python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.

There is no estimate to recompute: the paper reports no hazard ratio, only Kaplan-Meier
curves with log-rank P values (Figure 4B-C). The check is therefore (1) the direction
stated in each place the paper describes the 509-patient gastric cancer survival result,
and (2) the group sizes and P values printed on Figure 4B-C, read from the PMC image
nihms-1661019-f0004.jpg (they are not in the text, so they are typed in below).

If the repository's src/show.py is available (run from /home/ubuntu/papers), every quote
from the paper's text is also confirmed verbatim against the PMC-archived text.
"""
import os
import subprocess
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (from the paper)
# direction: +1 = high DDIT4 -> longer survival ; -1 = high (i.e. not low) DDIT4 -> shorter survival
STATEMENTS = [
    ("Abstract, Results",
     "Lower DDIT4 expression correlated with longer survival of gastric cancer patients.", -1),
    ("Introduction, final (5th) paragraph",
     "Patients with low expression have poorer survival.", +1),
    ("Results, human gastric cancer TMA",
     "We saw statistically significant correlation of high DDIT4 expression with increased "
     "patient survival and more differentiated tumors, especially in patients who had received "
     "chemotherapy", +1),
    ("Figure 4 title",
     "In humans, loss of DDIT4 correlates with worse gastric cancer survival", +1),
    ("Discussion, paragraph 3",
     "Our results showing DDIT4 expression correlating with less proliferation in precancerous "
     "lesions and better survival of patients with gastric cancer", +1),
]
# Other quotes from the PMC text used in writeup.md (checked verbatim, no direction coded)
OTHER_QUOTES = [
    ("Abstract, Conclusions",
     "During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation "
     "correlates with increased proliferation of cells harboring DNA damage, as well as increased "
     "tumor formation and growth in mice and humans."),
    ("Results, human gastric cancer TMA (full passage)",
     "We examined another annotated tissue microarray on which we had compiled 509 patients with "
     "gastric cancer. We saw statistically significant correlation of high DDIT4 expression with "
     "increased patient survival and more differentiated tumors, especially in patients who had "
     "received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage "
     "(Figure 4A–C; Table S3)."),
    ("Figure 4 legend, panel B",
     "Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an "
     "event defined as a cancer-related death."),
    ("Discussion, paragraph 3 (full sentence)",
     "Our results showing DDIT4 expression correlating with less proliferation in precancerous "
     "lesions and better survival of patients with gastric cancer confirm and extend on those of "
     "a previous report43."),
    ("Discussion, paragraph 4",
     "Paradoxically, however, studies across multiple organs have shown disparate results for "
     "DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, "
     "47–50."),
]
# The copyedited abstract deposited in PubMed (PMID 32956680) and Europe PMC:
PUBMED_ABSTRACT = "Lower DDIT4 expression correlated with longer survival of patients with gastric cancer."

TMA_N_TEXT = 509  # Results: "compiled 509 patients with gastric cancer"
# Figure 4B (overall) and 4C (patients receiving chemotherapy), legend boxes on the image
FIG4 = {
    "4B overall":      {"negative": 309, "positive": 200, "P": 0.021},
    "4C chemotherapy": {"negative": 263, "positive": 170, "P": 0.016},
}
# On both panels the DDIT4-positive (green) curve lies above the DDIT4-negative (blue) curve
# over most of follow-up, i.e. the data show high DDIT4 -> longer survival.
FIG4_DIRECTION = +1


def check_quotes():
    show = os.path.join(os.getcwd(), "src", "show.py")
    if not os.path.exists(show):
        print("src/show.py not found in the working directory; quote check skipped")
        return
    py = "/home/ubuntu/papers-data/venv/bin/python"
    py = py if os.path.exists(py) else "python3"
    txt = subprocess.run([py, show, "PMC7857017", "--full"], capture_output=True,
                         text=True).stdout
    quotes = [(loc, s) for loc, s, _ in STATEMENTS] + OTHER_QUOTES
    for loc, s in quotes:
        print(f"  verbatim in PMC text: {s in txt!s:5}  ({loc})")
    print(f"  PMC-text quotes found: {sum(s in txt for _, s in quotes)} of {len(quotes)}"
          "  (the PubMed wording and the Figure 4B-C counts are not in the PMC text)")


def main():
    print("== Direction of the DDIT4-survival association, by location")
    word = {+1: "high DDIT4 -> longer survival", -1: "low DDIT4 -> longer survival"}
    for loc, _, d in STATEMENTS:
        print(f"  {loc:36s} {word[d]}")
    print(f"  {'Figure 4B-C Kaplan-Meier curves':36s} {word[FIG4_DIRECTION]}")
    body = [d for _, _, d in STATEMENTS[1:]] + [FIG4_DIRECTION]
    abstract = STATEMENTS[0][2]
    print(f"  body statements agreeing with each other: {sum(d == body[0] for d in body)} of {len(body)}")
    print(f"  abstract opposite to every body statement: {all(d == -abstract for d in body)}")
    print(f"  PubMed/Europe PMC abstract wording: {PUBMED_ABSTRACT!r}")
    check_quotes()

    print("\n== Figure 4B-C group sizes")
    b, c = FIG4["4B overall"], FIG4["4C chemotherapy"]
    nb = b["negative"] + b["positive"]
    nc = c["negative"] + c["positive"]
    print(f"  4B: {b['negative']} negative + {b['positive']} positive = {nb}"
          f"  (Results text: {TMA_N_TEXT}; match = {nb == TMA_N_TEXT}); log-rank P = {b['P']}")
    print(f"  4C: {c['negative']} negative + {c['positive']} positive = {nc}"
          f" = {100 * nc / nb:.1f}% of the 4B cohort; log-rank P = {c['P']}")
    print(f"  DDIT4-positive share: 4B {100 * b['positive'] / nb:.1f}%, 4C {100 * c['positive'] / nc:.1f}%")
    make_figure(b)


# ------------------------------------------------------------------ figure
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"


def box(ax, x, y, w, h, color):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                linewidth=2, edgecolor=color, facecolor="white"))


def make_figure(b):
    plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none"})
    fig = plt.figure(figsize=(7.2, 5.9), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off"); ax.set_facecolor(BG)

    # left panel: abstract
    lx, lw, ly, lh = 0.03, 0.30, 0.36, 0.48
    box(ax, lx, ly, lw, lh, ORANGE)
    ax.text(lx + 0.02, ly + lh - 0.035, "Abstract (Results)", fontsize=11, weight="bold",
            color=INK, va="top")
    ax.text(lx + 0.02, ly + lh - 0.085, "PMC author manuscript", fontsize=10,
            color=INK2, va="top")
    q = textwrap.fill("“Lower DDIT4 expression correlated with longer survival of gastric "
                      "cancer patients.”", 24)
    ax.text(lx + 0.02, ly + lh - 0.135, q, fontsize=10.5, color=INK, va="top", style="italic")
    ax.text(lx + 0.02, ly + 0.11, "Same sentence, in\ncopyedited wording, is\nin the PubMed abstract", fontsize=10,
            color=INK2, va="bottom")
    ax.text(lx + 0.02, ly + 0.03, "Stated: LOW DDIT4 →\nlonger survival", fontsize=10.5,
            weight="bold", color=INK, va="bottom")

    # right panel: body + figure data
    rx, rw, ry, rh = 0.44, 0.53, 0.17, 0.79
    box(ax, rx, ry, rw, rh, BLUE)
    ax.text(rx + 0.02, ry + rh - 0.03, "Body of the paper (509-patient TMA)", fontsize=11,
            weight="bold", color=INK, va="top")
    items = [
        ("Introduction", "“Patients with low expression have poorer survival.”"),
        ("Results", "“high DDIT4 expression with increased patient survival”"),
        ("Figure 4 title", "“loss of DDIT4 correlates with worse gastric cancer survival”"),
        ("Discussion", "“DDIT4 expression correlating with … better survival”"),
        ("Figure 4B data", f"DDIT4-positive curve (n={b['positive']}) above DDIT4-negative "
                           f"(n={b['negative']}); log-rank P = {str(b['P'])[1:]}"),
    ]
    y = ry + rh - 0.095
    for head, body in items:
        ax.text(rx + 0.02, y, head, fontsize=10, weight="bold", color=INK2, va="top")
        lines = textwrap.fill(body, 47)
        ax.text(rx + 0.02, y - 0.036, lines, fontsize=10, color=INK, va="top")
        y -= 0.036 + 0.031 * (lines.count("\n") + 1) + 0.025
    ax.text(rx + 0.02, ry + 0.025, "Stated and shown: HIGH DDIT4 →\nlonger survival",
            fontsize=10.5, weight="bold", color=INK, va="bottom")

    # connector
    arr = FancyArrowPatch((lx + lw + 0.005, ly + lh / 2), (rx - 0.005, ly + lh / 2),
                          arrowstyle="<->", mutation_scale=14, linewidth=2, color=INK2)
    ax.add_patch(arr)
    ax.text((lx + lw + rx) / 2, ly + lh / 2 + 0.03, "opposite\ndirection", ha="center",
            va="bottom", fontsize=10, color=INK, weight="bold")
    ax.text((lx + lw + rx) / 2, ly + lh / 2 - 0.03, "same\ncohort,\nsame\nendpoint", ha="center",
            va="top", fontsize=10, color=INK2)

    ax.legend(handles=[Patch(facecolor="white", edgecolor=ORANGE, linewidth=2,
                             label="As reported in the abstract"),
                       Patch(facecolor="white", edgecolor=BLUE, linewidth=2,
                             label="Paper's own body text and Figure 4B data")],
              loc="lower left", bbox_to_anchor=(0.03, 0.02), frameon=False, fontsize=10,
              ncol=1, handlelength=1.6)
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
    plt.close(fig)
    print("\nwrote fig1.svg, fig1.png")


if __name__ == "__main__":
    main()

Abstract hazard ratio for SLiM-CRAB status lies outside its own confidence interval and contradicts the Results

J Clin Oncol (2024) · doi:10.1200/JCO.23.02771 · PMC11404760

Summary

This single-arm phase II trial in high-risk smoldering myeloma (90 patients) reports in its abstract that having any SLiM-CRAB criterion predicted progression to multiple myeloma (MM), with "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03". Those three numbers cannot belong together. The point estimate lies below the lower confidence limit, and the interval contains 1, so a Wald test based on it gives P ≈ 0.084, not .03. The Results section reports the same comparison as HR 0.10 (95% CI 0.01 to 0.96; P = .046). Those values are internally consistent (the interval's centre is 0.098, Wald P 0.046) and roughly match the paper's raw counts. The correct result keeps its direction and nominal significance. But the abstract figures, the version most readers see, cannot be read as printed, and the body's figures show that the estimate rests on 5 events and has an interval spanning about two orders of magnitude.

What the paper reports

Abstract, Results:

The presence of any SLiM CRAB criteria predicted progression to MM (four of the five patients; hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03).

Results, "Time-To-Event End Points", paragraph 4 of that section (same comparison, same outcome, same figure):

As mentioned above, 28 patients presented with any one SLiM biomarker. Although their response rate was not different from that observed in the standard high-risk–SMM population (Appendix Table A2), the presence of any SLiM biomarker predicted progression to MM (HR, 0.10 [95% CI, 0.01 to 0.96]; P = .046; Fig 3B).

The subgroup sizes. Abstract, Results:

90 patients were included, and 31% met at least one SixtyLightchain MRI (SLiM)-hypercalcemia, renal impairment, anemia, bone disease (CRAB) criterion.

Table 1:

SLiM-CRAB criteria, No. (%) | 28 (31)

Appendix Table A2 gives the columns "High Risk (n = 62)" and "Ultra High Risk (n = 28)". Its footnote b reads:

Four patients biochemically progressed and one of them progressed to myeloma.

The model. Methods, "Statistical Analysis":

The Kaplan-Meier method was used to characterize event time distributions, and the corresponding treatment hazard ratio was estimated using a stratified Cox regression model to compare outcomes based on baseline characteristics and MRD status.

The conclusion that rests on this result. Discussion, paragraph 6:

In our study, the only feature predicting progression to MM was the presence of any SLiM biomarker, now incorporated into the definition of MM, which would reinforce this hypothesis.

What the paper's own numbers show

Method. A Cox-model 95% CI is symmetric on the log scale, so the reported HR should lie close to the interval's geometric centre, sqrt(lower × upper). The standard error of log HR is ln(upper/lower)/(2 × 1.96), and the two-sided Wald P is 2Φ(−|ln centre|/SE). I applied this check to every hazard ratio printed in the paper (five in all), which shows whether the paper follows one consistent convention. I then checked whether rounding or a single misplaced digit could reconcile the abstract. As a rough check on direction and size that is independent of the Cox model, I computed a crude risk ratio from the counts (1 of 62 vs 4 of 28 progressed to MM). It ignores follow-up time and is not the paper's estimate. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Is the HR inside its 95% CI? Abstract, Results HR 0.12; CI 0.14 to 1.13 No: 0.12 < 0.14
Centre of the CI, sqrt(lo × hi) Abstract, Results HR 0.12 0.398
Does the CI exclude 1? Abstract, Results P = .03 No: 1.13 > 1, so any test matching this interval gives P ≥ .050
Wald P from the CI Abstract, Results P = .03 0.084 (SE of log HR 0.533)
Can rounding explain HR < lower limit? Abstract, Results 0.12 vs 0.14 No: values rounding to 0.12 are < 0.125; values rounding to 0.14 are ≥ 0.135
Centre of the CI Results, Time-To-Event End Points; Fig 3B HR 0.10; CI 0.01 to 0.96 0.098
Wald P from the CI Results, Time-To-Event End Points; Fig 3B P = .046 0.046 from the CI centre; 0.048 from HR 0.10 (SE 1.164)
The paper's other three HRs: CI centre and Wald P Results, Time-To-Event End Points (Fig 2B, Appendix Fig A1A, A1B) 0.29 (0.13 to 0.66), P = .003; 0.42 (0.22 to 0.81), P = .01; 0.40 (0.18 to 0.85), P = .018 0.293, P 0.003; 0.422, P 0.009; 0.391, P 0.018: all consistent
Abstract with the lower limit read as 0.014 hypothetical single typo n/a Centre 0.126 (fits 0.12), but the CI still contains 1; Wald P 0.058 to 0.064
CI that HR 0.12 with P = .03 would imply hypothetical n/a 0.018 to 0.814
Progression to MM by SLiM status Abstract ("four of the five"); Table 1; Table A2 4 of 5 events in the SLiM group; n = 28 vs 62 4/28 = 14.3% vs 1/62 = 1.6%
Crude risk ratio, no SLiM vs SLiM from the counts above n/a 0.113 (95% CI 0.013 to 0.965); Wald P 0.046; Fisher exact P 0.031
Share of patients with any SLiM-CRAB criterion Abstract; Table 1 31%; 28 (31) 28/90 = 31.1% (consistent)

Of the five hazard ratios printed in the paper, the abstract's is the only one whose point estimate, interval and P value do not fit together. The other four, including the Results-section statement of this same comparison, agree with the Wald method to within rounding.

Further observations, not part of the main finding. The following are reported here so that a correction can address them together. I have not investigated them further.

Quantity Location Reported Check
70-month TTP Abstract; Results, Time-To-Event End Points 94% (95% CI, 84 to 89) Estimate lies outside its own CI
70-month OS Abstract; Results, Time-To-Event End Points 92% (95% CI, 82 to 89) Estimate lies outside its own CI
70-month TTBP Results, Time-To-Event End Points 59% (95% CI, 66 to 76) Estimate lies outside its own CI
Patients with any SLiM criterion Results, Time-To-Event End Points, paragraph 2 "12 [38.7%] of 31 patients" vs "18 [30.5%] of 59 patients" 31 here vs 28 in Table 1, Table A2 and the Results paragraph quoted above; 12 + 18 = 30, against 36 biochemical progressions reported in the same section
Sustained uMRD 4 years after ASCT Abstract / Discussion and Table A1 / CONTEXT box "29 patients (31%)" / "28 patients (31%)" and "28 (31)" / "40 (44%)" 29/90 = 32.2%; 28/90 = 31.1%; 40/90 = 44.4%. The three counts differ

Figure

Dot-and-interval plot on a log ratio axis comparing the abstract's and the Results' printed hazard ratios with the values recomputed from their own intervals and from the raw counts.

Figure 1. One comparison, any SLiM-CRAB criterion vs none and progression to MM, as printed in two places and as recomputed. Orange circles and lines show values printed in the paper; blue diamonds show values recomputed from the paper's own numbers. The abstract's point estimate (0.12) lies to the left of its own interval (0.14 to 1.13), and that interval crosses the reference line at 1. The interval's own centre is 0.40, with a Wald P of 0.084. The Results-section estimate (0.10, 0.01 to 0.96) sits at the centre of its interval and implies P = 0.046, as printed. A crude risk ratio from the counts (1/62 vs 4/28), which ignores follow-up time, lands near the Results value. The horizontal axis is logarithmic, and the vertical line marks a ratio of 1.

Dot plot of printed P values against the Wald P values implied by the printed confidence intervals for the five hazard ratios in the paper.

Figure 2. Every hazard ratio in the paper, comparing the printed P value (orange circle) with the Wald P implied by its printed 95% CI (blue diamond). Within each row the circle and diamond are offset slightly up and down so that both stay visible. For four of the five, the two agree to within rounding. Only the abstract's statement crosses the P = .05 line, at .03 printed against 0.084 implied by its interval. This suggests that the paper's intervals and P values otherwise follow one convention, and that the abstract figures are an exception rather than a different method.

Why it matters

The error does not touch the trial's primary end point (uMRD 3 months after ASCT, 56 of 90) or its sustained-uMRD secondary end point. Nor, on the evidence of the body text and Fig 3B, does it reverse the reported finding: the internally consistent value (HR 0.10, 95% CI 0.01 to 0.96, P = .046) has the same direction and is nominally significant.

It matters for three reasons:

  1. The abstract is uninterpretable as printed. A reader who has only the abstract sees an interval (0.14 to 1.13) that includes no effect, next to a claim that SLiM status "predicted" progression and a P value of .03. Read at face value, the interval says the association is not statistically significant, which contradicts the sentence it supports.
  2. The body values show how fragile the result is. The finding rests on 5 progression events, 4 of them in the SLiM group. Its interval spans roughly two orders of magnitude, and its P value (.046) sits just under the threshold. The Discussion uses this result, "the only feature predicting progression to MM", to support the hypothesis that early treatment of true high-risk smoldering myeloma raises the chance of operational cure. A reader should weigh it as a hypothesis-generating signal.
  3. The direction convention is not stated. "Presence ... predicted progression" corresponds to an HR above 1 for SLiM present vs absent. Both printed HRs are below 1, so the reference group must be the SLiM group (absent vs present). The crude proportions (1.6% vs 14.3%) support that reading, but the paper does not state it.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. In the abstract, replace "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03" with the values from the fitted model. If the model is the one reported in the Results and Fig 3B, these are "hazard ratio, 0.10; 95% CI, 0.01 to 0.96; P = .046".
  2. State the reference group for the SLiM hazard ratio (it appears to be SLiM present as reference), so that an HR below 1 reads correctly alongside "presence ... predicted progression". Consider reporting it as SLiM present vs absent (HR ≈ 10) instead.
  3. Given 5 events, consider stating the number of events with the HR, and describing the association as exploratory in the abstract and Discussion.
  4. Separately, check the 70-month TTP, OS and TTBP intervals (each excludes its own estimate). Also check the SLiM subgroup count (31 vs 28) in the biochemical-progression paragraph, and the sustained-uMRD count at 4 years (29 in the abstract, 28 in the Discussion and Table A1, 40 in the CONTEXT box).
Recomputation script (python recompute.py.txt)
"""Recompute the SLiM-CRAB hazard ratio statements of PMC11404760
(J Clin Oncol 2024, doi:10.1200/JCO.23.02771).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
  fig1.svg / fig1.png             (= candidate A, dot-and-interval plot)
  fig2.svg / fig2.png             (= candidate C, reported vs CI-implied P for every HR)
  candidates/cand_A..E.svg / .png (five candidate designs for review)

All inputs are copied from the article as archived in PubMed Central:
  Abstract, Results:  HR 0.12; 95% CI 0.14 to 1.13; P = .03 (any SLiM-CRAB -> MM)
  Results, Time-To-Event End Points p3:  HR 0.10 [95% CI 0.01 to 0.96]; P = .046 (Fig 3B)
  Results, Time-To-Event End Points p0-p1:  the three other HRs printed in the paper
  Table 1 / Appendix Table A2:  28 of 90 patients with any SLiM-CRAB criterion (62 without)
  Abstract / Appendix Table A2 footnotes:  4 of the 5 progressions to MM were in the SLiM group
Wald method: for a ratio with 95% CI (lo, hi) on the log scale,
  SE = ln(hi/lo) / (2 * 1.959964),  centre = sqrt(lo * hi),  z = ln(centre) / SE.
"""
import math
import os

import numpy as np
from scipy.stats import norm, fisher_exact

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

Z = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (from the paper)
# (label, location, HR, lo, hi, printed P)
HRS = [
    ("Any SLiM-CRAB vs none -> MM (Abstract)", "Abstract, Results", 0.12, 0.14, 1.13, 0.03),
    ("Any SLiM vs none -> MM (Results, Fig 3B)", "Results, TTE p3", 0.10, 0.01, 0.96, 0.046),
    ("uMRD vs detectable MRD -> biochem. progression", "Results, TTE p0 (Fig 2B)", 0.29, 0.13, 0.66, 0.003),
    ("HRCA vs none -> biochem. progression", "Results, TTE p1 (Fig A1A)", 0.42, 0.22, 0.81, 0.01),
    ("HRCA incl. 1q vs none -> biochem. progression", "Results, TTE p1 (Fig A1B)", 0.40, 0.18, 0.85, 0.018),
]
N_TOTAL = 90
N_SLIM = 28            # Table 1 "SLiM-CRAB criteria, No. (%) | 28 (31)"; Table A2 n = 28
N_NOSLIM = 62          # Table A2 n = 62
MM_SLIM, MM_NOSLIM = 4, 1   # abstract: "four of the five patients"; five progressed in total
# Side observations: (label, location, estimate %, lo %, hi %)
KM = [
    ("70-month TTP", "Abstract; Results TTE p3", 94, 84, 89),
    ("70-month OS", "Abstract; Results TTE p4", 92, 82, 89),
    ("70-month TTBP", "Results TTE p0", 59, 66, 76),
]
SUSTAINED = [("Abstract", 29), ("Discussion p1 / Table A1", 28), ("CONTEXT box", 40)]

# ---------------------------------------------------------------- helpers
def wald(hr, lo, hi):
    se = math.log(hi / lo) / (2 * Z)
    centre = math.sqrt(lo * hi)
    p_centre = 2 * norm.sf(abs(math.log(centre)) / se)
    p_hr = 2 * norm.sf(abs(math.log(hr)) / se)
    return se, centre, p_centre, p_hr


def fmt_p(p):
    return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"


R = []
print("=== 1. Every hazard ratio printed in the paper, checked against its own 95% CI ===")
for lab, loc, hr, lo, hi, p in HRS:
    se, c, pc, ph = wald(hr, lo, hi)
    R.append(dict(lab=lab, loc=loc, hr=hr, lo=lo, hi=hi, p=p, se=se, c=c, pc=pc, ph=ph))
    print(f"\n{lab}  [{loc}]")
    print(f"  printed: HR {hr}; 95% CI {lo} to {hi}; P = {p}")
    print(f"  HR inside its CI? {lo <= hr <= hi};  CI includes 1? {lo <= 1 <= hi}")
    print(f"  CI centre sqrt(lo*hi) = {c:.3f};  SE(log HR) = {se:.3f}")
    print(f"  Wald P from CI centre = {fmt_p(pc)};  Wald P from printed HR with that SE = {fmt_p(ph)}")

A, B = R[0], R[1]

print("\n=== 2. Can rounding explain the abstract? ===")
print("  values that round to HR 0.12 lie in [0.115, 0.125); values that round to a lower limit of 0.14 lie in [0.135, 0.145)")
print(f"  the two ranges overlap: {0.125 > 0.135}")
print(f"  smallest z compatible with a 95% CI that includes 1 is 1.96, i.e. P >= {2 * norm.sf(Z):.3f}")

print("\n=== 3. Single-typo hypothesis: lower limit 0.014 instead of 0.14 ===")
se, c, pc, ph = wald(0.12, 0.014, 1.13)
print(f"  CI 0.014 to 1.13: centre {c:.3f}; SE {se:.3f}; Wald P from centre {pc:.3f}; from HR 0.12 {ph:.3f}; includes 1: True")
z03 = norm.isf(0.03 / 2)
se03 = abs(math.log(0.12)) / z03
print(f"  for HR 0.12 and P = .03 the Wald CI would be {math.exp(math.log(0.12) - Z * se03):.3f} to {math.exp(math.log(0.12) + Z * se03):.3f}")
TYPO = dict(c=c, se=se, pc=pc, ph=ph)

print("\n=== 4. Crude check from counts (ignores follow-up time; not the Cox model) ===")
r1, r0 = MM_NOSLIM / N_NOSLIM, MM_SLIM / N_SLIM
rr = r1 / r0
se_rr = math.sqrt(1 / MM_NOSLIM - 1 / N_NOSLIM + 1 / MM_SLIM - 1 / N_SLIM)
rr_lo, rr_hi = math.exp(math.log(rr) - Z * se_rr), math.exp(math.log(rr) + Z * se_rr)
p_rr = 2 * norm.sf(abs(math.log(rr)) / se_rr)
_, p_fisher = fisher_exact([[MM_NOSLIM, N_NOSLIM - MM_NOSLIM], [MM_SLIM, N_SLIM - MM_SLIM]])
print(f"  progression to MM: no SLiM {MM_NOSLIM}/{N_NOSLIM} = {100 * r1:.1f}%;  SLiM {MM_SLIM}/{N_SLIM} = {100 * r0:.1f}%")
print(f"  crude risk ratio (no SLiM / SLiM) = {rr:.3f}; 95% CI {rr_lo:.3f} to {rr_hi:.3f}; Wald P = {p_rr:.3f}; Fisher exact P = {p_fisher:.3f}")
print(f"  N_SLIM + N_NOSLIM = {N_SLIM + N_NOSLIM} (= {N_TOTAL}); 28/90 = {100 * N_SLIM / N_TOTAL:.1f}%")
CRUDE = dict(rr=rr, lo=rr_lo, hi=rr_hi, p=p_rr)

print("\n=== 5. Side observations ===")
for lab, loc, e, lo, hi in KM:
    print(f"  {lab} [{loc}]: {e}% (95% CI {lo} to {hi}) -> estimate inside its CI? {lo <= e <= hi}")
print("  biochemical progression by SLiM (Results TTE p1): 12/31 = %.1f%%, 18/59 = %.1f%%; 31 + 59 = %d; 12 + 18 = %d (36 biochemical progressions reported)"
      % (1200 / 31, 1800 / 59, 31 + 59, 12 + 18))
for loc, n in SUSTAINED:
    print(f"  sustained uMRD at 4 years [{loc}]: {n}/90 = {100 * n / 90:.1f}%")

# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})
W = 7.2
MS = 9       # marker size in points (>= 8 px)
LW = 1.5     # 1.5 pt = 2 px at 96 dpi


def save(fig, name, folder=HERE):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def log_axis(ax, lo, hi, ticks, label):
    ax.set_xscale("log")
    ax.set_xlim(lo, hi)
    ax.set_xticks(ticks)
    ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.minorticks_off()
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_xlabel(label, color=INK2)


LEGEND = [
    Line2D([0], [0], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported in the paper"),
    Line2D([0], [0], color=BLUE, lw=LW, marker="D", ms=MS - 1, label="Recomputed from the paper's own numbers"),
]


# ---------------------------------------------------------------- candidate A: dot-and-interval
def cand_A():
    rows = [
        ("Abstract, as printed", ORANGE, "o", A["hr"], A["lo"], A["hi"],
         f"HR 0.12 (0.14 to 1.13); P = .03"),
        ("Abstract interval: its centre\nand Wald P", BLUE, "D", A["c"], A["lo"], A["hi"],
         f"{A['c']:.2f} (0.14 to 1.13); P = {A['pc']:.3f}"),
        ("Results text and Fig 3B,\nas printed", ORANGE, "o", B["hr"], B["lo"], B["hi"],
         "HR 0.10 (0.01 to 0.96); P = .046"),
        ("Results interval: its centre\nand Wald P", BLUE, "D", B["c"], B["lo"], B["hi"],
         f"{B['c']:.3f} (0.01 to 0.96); P = {B['pc']:.3f}"),
        ("Crude risk ratio from counts\n1/62 vs 4/28 (no Cox model)", BLUE, "D", CRUDE["rr"], CRUDE["lo"], CRUDE["hi"],
         f"{CRUDE['rr']:.3f} ({CRUDE['lo']:.3f} to {CRUDE['hi']:.2f}); P = {CRUDE['p']:.3f}"),
    ]
    fig, ax = plt.subplots(figsize=(W, 3.9))
    y = np.arange(len(rows))[::-1].astype(float)
    y[2:] -= 0.35   # gap between abstract pair and body pair
    for yi, (lab, col, mk, est, lo, hi, txt) in zip(y, rows):
        ax.plot([lo, hi], [yi, yi], color=col, lw=LW, solid_capstyle="butt", zorder=2)
        ax.plot(est, yi, marker=mk, ms=MS if mk == "o" else MS - 1, color=col,
                mec=BG, mew=1.2, zorder=3)
        ax.text(3.4, yi, txt, va="center", ha="left", fontsize=10, color=INK)
    ax.axvline(1, color=INK2, lw=0.8, zorder=1)
    ax.text(0.93, y[0] + 0.75, "ratio = 1 (no association)", ha="right", va="bottom", fontsize=10, color=INK2)
    ax.annotate("0.12 lies below its own\nlower limit (0.14)", xy=(0.12, y[0]), xytext=(0.0125, y[0] + 0.05),
                fontsize=10, color=INK, va="center",
                arrowprops=dict(arrowstyle="-|>", color=INK2, lw=0.9, shrinkB=6))
    ax.set_yticks(y)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
    log_axis(ax, 0.004, 3.0, [0.01, 0.03, 0.1, 0.3, 1, 3], "Ratio, SLiM-CRAB absent vs present (log scale)")
    ax.set_ylim(y[-1] - 0.6, y[0] + 1.25)
    ax.text(3.4, y[0] + 0.75, "Estimate (95% CI); P", fontsize=10, color=INK2, va="bottom")
    ax.legend(handles=LEGEND, loc="upper left", bbox_to_anchor=(0.0, -0.2), ncol=2, frameon=False, fontsize=10)
    ax.set_title("Same comparison, two incompatible results: any SLiM-CRAB criterion and progression to MM",
                 fontsize=11, loc="left", color=INK, pad=10)
    return fig


# ---------------------------------------------------------------- candidate B: two text panels
def cand_B():
    fig = plt.figure(figsize=(W, 3.3))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(8, 50)
    ax.axis("off")

    def panel(x, y, w, h, title, quote, check):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                    fc="white", ec=INK2, lw=1.0))
        ax.text(x + 1.5, y + h - 2.0, title, fontsize=11, fontweight="bold", va="top")
        ax.add_patch(plt.Rectangle((x + 1.5, y + h - 15.8), 0.6, 10.6, color=ORANGE, lw=0))
        ax.text(x + 3.2, y + h - 5.4, quote, fontsize=10, va="top", color=INK)
        ax.text(x + 1.5, y + h - 17.6, "Recomputed check:", fontsize=10, va="top", color=INK2)
        ax.plot(x + 2.3, y + h - 22.0, marker="D", ms=MS - 1, color=BLUE, mec=BG)
        ax.text(x + 4.2, y + h - 20.2, check, fontsize=10, va="top", color=INK)

    panel(1.5, 19, 42, 29, "Abstract, Results",
          "“...predicted progression to MM\n(four of the five patients; hazard\nratio, 0.12; 95% CI, 0.14 to 1.13;\nP = .03).”",
          f"HR 0.12 is below the lower limit 0.14\nInterval contains 1\nCI centre {A['c']:.2f}; Wald P {A['pc']:.3f}")
    panel(56.5, 19, 42, 29, "Results text (Fig 3B)",
          "“...predicted progression to MM\n(HR, 0.10 [95% CI, 0.01 to 0.96];\nP = .046; Fig 3B).”",
          f"HR 0.10 is inside its interval\nInterval excludes 1\nCI centre {B['c']:.3f}; Wald P {B['pc']:.3f}")
    ax.annotate("", xy=(55.6, 34), xytext=(44.4, 34),
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
    ax.text(50, 35.5, "same\nanalysis", ha="center", va="bottom", fontsize=10, color=INK2)
    ax.text(50, 30.5, "different\nHR, CI, P", ha="center", va="top", fontsize=10, color=INK)
    ax.text(50, 10.5, "Conflict: the two statements describe the same analysis (any SLiM criterion vs none, progression to MM,\n"
                   "ITT population) but give different HR, CI and P. Only the Results values are internally consistent.",
            ha="center", va="bottom", fontsize=10, color=INK)
    return fig


# ---------------------------------------------------------------- candidate C: reported vs CI-implied P
def cand_C():
    fig, ax = plt.subplots(figsize=(W, 3.4))
    y = np.arange(len(R))[::-1]
    for yi, r in zip(y, R):
        ax.plot([r["p"], r["pc"]], [yi + 0.13, yi - 0.13], color=GRID, lw=2, zorder=1)
        ax.plot(r["p"], yi + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
        ax.plot(r["pc"], yi - 0.13, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
        ax.text(1.15, yi, f"{r['p']:g} vs {r['pc']:.3f}", va="center", fontsize=10)
    ax.axvline(0.05, color=INK2, lw=0.8)
    ax.text(0.047, y[0] + 0.6, "P = .05", ha="right", va="bottom", fontsize=10, color=INK2)
    ax.text(1.15, y[0] + 0.6, "printed vs from CI", fontsize=10, color=INK2, va="bottom")
    ax.set_yticks(y)
    ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
    log_axis(ax, 0.001, 1.0, [0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1], "Two-sided P value (log scale)")
    ax.set_ylim(-0.6, y[0] + 1.1)
    leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="P printed in the paper"),
           Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Wald P implied by the printed 95% CI")]
    ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=2, frameon=False, fontsize=10)
    ax.set_title("Every hazard ratio in the paper: printed P vs the P its own interval implies",
                 fontsize=11, loc="left", pad=10)
    return fig


# ---------------------------------------------------------------- candidate D: printed HR vs CI centre, all HRs
def cand_D():
    fig, ax = plt.subplots(figsize=(W, 3.4))
    y = np.arange(len(R))[::-1]
    for yi, r in zip(y, R):
        ax.plot([r["lo"], r["hi"]], [yi, yi], color=INK2, lw=LW, alpha=0.55, zorder=1, solid_capstyle="butt")
        ax.plot(r["c"], yi - 0.2, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
        ax.plot(r["hr"], yi + 0.2, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=4)
        ax.text(2.4, yi, f"{r['hr']:.2f} vs {r['c']:.3f}", va="center", fontsize=10)
    ax.axvline(1, color=INK2, lw=0.8)
    ax.text(2.4, y[0] + 0.6, "printed vs centre", fontsize=10, color=INK2, va="bottom")
    ax.set_yticks(y)
    ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
    log_axis(ax, 0.006, 2.0, [0.01, 0.03, 0.1, 0.3, 1], "Hazard ratio (log scale); gray line = printed 95% CI")
    ax.set_ylim(-0.6, y[0] + 1.1)
    leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="HR printed in the paper"),
           Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Centre of the printed CI, sqrt(lo × hi)"),
           Line2D([0], [0], color=INK2, alpha=0.55, lw=LW, label="Printed 95% CI")]
    ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=3, frameon=False, fontsize=10)
    ax.set_title("Every hazard ratio in the paper: only the abstract's sits off its interval's centre",
                 fontsize=11, loc="left", pad=10)
    return fig


# ---------------------------------------------------------------- candidate E: consistency checklist
def cand_E():
    checks = [
        ("Printed HR lies inside its 95% CI", ("No: 0.12 < 0.14", False), ("Yes: 0.01 ≤ 0.10 ≤ 0.96", True)),
        ("CI excludes 1, as P < .05 requires", ("No: interval contains 1", False), ("Yes: upper limit 0.96", True)),
        ("CI centre sqrt(lo × hi) ≈ printed HR", (f"No: centre {A['c']:.2f}", False), (f"Yes: centre {B['c']:.3f}", True)),
        ("Wald P from CI ≈ printed P", (f"No: {A['pc']:.3f} vs .03", False), (f"Yes: {B['pc']:.3f} vs .046", True)),
    ]
    fig = plt.figure(figsize=(W, 3.3))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 100)
    ax.set_ylim(0, 50)
    ax.axis("off")
    cx = [2, 38, 72]
    for x, t1, t2 in ((cx[1], "Abstract", "HR 0.12 (0.14 to 1.13); P = .03"),
                      (cx[2], "Results text / Fig 3B", "HR 0.10 (0.01 to 0.96); P = .046")):
        ax.text(x, 46, t1, fontsize=10, va="top", fontweight="bold")
        ax.text(x, 42.6, t2, fontsize=10, va="top")
        ax.add_patch(plt.Rectangle((x - 1.2, 38.6), 0.6, 7.4, color=ORANGE, lw=0))
    ax.text(cx[0], 46, "Check, recomputed from the\npaper's own numbers", fontsize=10, va="top", color=INK2)
    for i, (lab, a, b) in enumerate(checks):
        yy = 32 - i * 8
        ax.plot([1, 99], [yy + 4.4, yy + 4.4], color=GRID, lw=1)
        ax.text(cx[0], yy, lab, fontsize=10, va="center")
        for x, (txt, ok) in ((cx[1], a), (cx[2], b)):
            ax.plot(x + 0.6, yy, marker="D", ms=MS - 2, color=BLUE, mec=BG)
            ax.text(x + 2.2, yy, ("✓ " if ok else "✗ ") + txt, fontsize=10, va="center",
                    fontweight="normal" if ok else "bold")
    ax.text(cx[0], 1.5, "Orange bar: values printed in the paper. Blue diamond: check recomputed from them (Wald method, log scale).",
            fontsize=10, color=INK2, va="center")
    return fig


for name, fn in zip("ABCDE", (cand_A, cand_B, cand_C, cand_D, cand_E)):
    save(fn(), f"cand_{name}", CAND)
save(cand_A(), "fig1")
save(cand_C(), "fig2")
print("\nwrote fig1.*, fig2.* and candidates/cand_A..E.* in", HERE)

Abstract reports two obesity-group odds ratios as the overweight and obesity estimates for EPT use

JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.19236 · PMC12235495

Summary

The study pools 31 case-control studies to estimate associations between exogenous hormone use and intrinsic-like breast cancer subtypes, stratified by body mass index (BMI). The abstract says the associations of current estrogen-progestin therapy (EPT) use with luminal A–like and luminal B–ERBB2-like tumors "were attenuated but remained" for women with overweight (OR 1.40, 95% CI 1.02–1.92) or obesity (OR 1.68, 95% CI 1.01–2.78). In the paper's own Results text and in Supplement 1 eTable 3, both of these values come from the obesity stratum: 1.40 is the luminal A–like OR and 1.68 is the luminal B–ERBB2-like OR. The overweight estimates, 2.18 (1.83–2.61) for luminal A–like and 1.83 (1.32–2.55) for luminal B–ERBB2-like, do not appear in the abstract. No direction or significance changes, but an abstract reader would take the overweight luminal A–like excess odds to be about a third of what the paper's own data show, and would see a BMI pattern (1.40 for overweight, then 1.68 for obesity) that the full results do not show.

What the paper reports

Abstract, Results:

Current EPT users with healthy weight were more likely to be diagnosed with luminal A–like (odds ratio [OR], 2.51 [95% CI, 2.26-2.80]) or luminal B–ERBB2-like (OR, 1.95 [95% CI, 1.61-2.37]) subtypes. These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78]).

Results, "Combined EPT Use and Tumor Subtypes", paragraph 2:

In women with healthy weight, current EPT users were more likely to be diagnosed with luminal A–like (OR, 2.51 [95% CI, 2.26-2.80]), luminal B–like (OR, 1.47 [95% CI, 1.17-1.86]), or luminal B–ERBB2–like (OR, 1.95 [95% CI, 1.61-2.37]) breast tumors, with significant heterogeneity across subtypes (P < .001 for heterogeneity).

Results, "Combined EPT Use and Tumor Subtypes", paragraph 3:

In women with overweight, current EPT use was positively associated with luminal A–like (OR, 2.18 [95% CI, 1.83-2.61]) and luminal B–ERBB2-like (OR, 1.83 [95% CI, 1.32-2.55]) tumors but not with other subtypes (P < .001 for heterogeneity). In women with obesity, associations for luminal A–like (OR, 1.40 [95% CI, 1.02-1.92]) and luminal B–ERBB2-like (OR, 1.68 [95% CI, 1.01-2.78]) tumors were seen, with smaller effect sizes and nonsignificant heterogeneity (P < .11 for heterogeneity).

Methods, Statistical Analysis (the only BMI definition in the paper):

To evaluate heterogeneity in associations by BMI, associations were stratified by BMI categories as follows: lean or healthy weight (hereinafter, healthy weight) (18.5-<25 [calculated as weight in kilograms divided by height in meters squared]), overweight (25-<30), or obesity (≥30).

Supplement 1, eTable 3 (EPT use by intrinsic-like subtype, stratified by BMI), "Current use" rows: Lean/normal, luminal A-like 2.51 (2.26,2.80) and luminal B-ERBB2-like 1.95 (1.61,2.37). Overweight, 2.18 (1.83,2.61) and 1.83 (1.32,2.55). Obese, 1.40 (1.02,1.92) and 1.68 (1.01,2.78).

All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC12235495 --grep. The eTable values were read from the supplement PDF.

What the paper's own numbers show

Method. I took every current-EPT odds ratio and 95% CI for luminal A–like and luminal B–ERBB2-like tumors from the Results text, and checked that each CI is symmetric about its OR on the log scale. I checked the text values against Supplement 1 eTable 3. I then matched each (OR, lower, upper) triple in the abstract against every BMI-by-subtype cell in the Results. To size the discrepancy, I compared the abstract's "overweight" value with the paper's overweight estimates as a ratio of ORs and as excess odds (OR − 1). I computed a z statistic for the difference of log ORs, using standard errors taken from the CI widths. That z treats the overweight and obesity strata as independent, which holds because they are disjoint groups of women. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
Healthy weight, luminal A–like Abstract; Results para. 2; eTable 3 2.51 (2.26–2.80) Abstract and Results identical; geometric CI midpoint 2.516
Healthy weight, luminal B–ERBB2-like Abstract; Results para. 2; eTable 3 1.95 (1.61–2.37) Abstract and Results identical; midpoint 1.953
Overweight, luminal A–like Results para. 3; eTable 3 2.18 (1.83–2.61) Midpoint 2.185, consistent; not in the abstract
Overweight, luminal B–ERBB2-like Results para. 3; eTable 3 1.83 (1.32–2.55) Midpoint 1.835, consistent; not in the abstract
Obesity, luminal A–like Results para. 3; eTable 3 1.40 (1.02–1.92) Midpoint 1.399, consistent
Obesity, luminal B–ERBB2-like Results para. 3; eTable 3 1.68 (1.01–2.78) Midpoint 1.676, consistent
Abstract value labelled "overweight" Abstract, Results 1.40 (1.02–1.92) Equals only the obesity, luminal A–like cell; matches no overweight cell
Abstract value labelled "obesity" Abstract, Results 1.68 (1.01–2.78) Equals only the obesity, luminal B–ERBB2-like cell
Overweight luminal A–like vs abstract "overweight" Results vs Abstract 2.18 vs 1.40 Ratio of ORs 1.56; excess odds 1.18 vs 0.40, so the abstract shows 34% of it; z = 2.39 for the difference
Overweight luminal B–ERBB2-like vs abstract "overweight" Results vs Abstract 1.83 vs 1.40 Ratio of ORs 1.31; excess odds 0.83 vs 0.40 (48%); z = 1.15
BMI gradient, luminal A–like Results paras. 2–3 2.51 → 2.18 → 1.40 Falls steadily with BMI
BMI gradient, luminal B–ERBB2-like Results paras. 2–3 1.95 → 1.83 → 1.68 Falls steadily with BMI
BMI gradient as the abstract presents it Abstract overweight 1.40, obesity 1.68 Rises from overweight to obesity, which no subtype in the Results does
Significance, overweight and obesity cells Results para. 3 All four CIs exclude 1 z (log OR / SE) = 8.60, 3.60, 2.09, 2.01; unchanged
Fully adjusted model (sensitivity) eTable 7 Overweight 2.18 (1.82–2.61), 1.81 (1.30–2.52); obese 1.41 (1.02–1.93), 1.65 (0.99–2.75) No overweight cell equals 1.40 (1.02–1.92)

Every value in the Results text agrees with eTable 3, and every CI is consistent with its point estimate. The other four current-EPT values in the abstract (2.51, 1.95 and 1.47 for healthy weight, 1.68 for obesity) match Results values for the BMI group they are given under. The only mismatch is the BMI label on the first of the two higher-BMI values.

Figure

The abstract's "overweight" and "obesity" values are both the obesity-row estimates in the Results

Figure 1. On the left (orange) is the abstract's sentence on attenuation, with the value attached to each BMI group as printed. On the right (blue) are the current-EPT odds ratios (95% CI) for women with overweight and obesity from Results paragraph 3, which are identical to the Supplement 1 eTable 3 "Current use" rows. Arrows link each abstract value to the only Results cell that has the same OR and CI. Both arrows end in the obesity row. The abstract's "overweight" value is the obesity, luminal A–like estimate. The shaded overweight row (2.18 and 1.83) is not reported anywhere in the abstract.

Current-EPT odds ratios by BMI group: abstract values vs Results text

Figure 2. Odds ratios with 95% CIs for current EPT use vs never use, on a log axis with a thin reference line at OR = 1. Blue: values from the Results text and eTable 3 (circles, luminal A–like; squares, luminal B–ERBB2-like). Orange diamonds: the abstract's values, drawn under the BMI group the abstract names. In the healthy-weight and obesity groups each orange interval repeats a blue one. In the overweight group the abstract's interval (1.02–1.92) matches neither overweight estimate. It reproduces the obesity, luminal A–like interval one group lower. The blue luminal A–like estimates fall steadily from 2.51 to 2.18 to 1.40.

Why it matters

The BMI stratification is central to how the paper frames its EPT findings: the Key Points and Conclusions stress associations "particularly in participants with healthy weight", and the Discussion describes the luminal A–like and luminal B–ERBB2-like associations as "consistent across BMI groups, albeit attenuated with greater adiposity". The abstract is the only part of the paper that most readers, and most citing articles, see. A reader who relies on it would conclude that among women with overweight, current EPT use is associated with an OR of about 1.40. By the paper's own estimates the overweight luminal A–like OR is 2.18. That is roughly three times the excess odds (1.18 vs 0.40), and the difference is larger than sampling error alone would plausibly produce (z ≈ 2.4). The mislabeling also distorts the dose-response picture by BMI: the abstract's numbers rise from overweight (1.40) to obesity (1.68), while the full results fall steadily with BMI for both subtypes. Because neither value in the sentence is tied to a subtype, a reader also cannot tell that the two numbers refer to different tumor subtypes.

The consequences are limited. All four overweight and obesity estimates are above 1 with CIs that exclude 1, so the abstract's qualitative statement ("attenuated but remained") is true for both BMI groups. The title, Key Points, abstract Conclusions, body Results, Discussion, Figure 1 and the supplement are correct. The estrogen-only therapy and oral contraceptive findings, and the heterogeneity tests, are not affected. The error is a misreported secondary estimate in the abstract, not a change to any headline conclusion.

Explanations considered

The pattern fits a mislabeled sentence in the abstract: the two obesity-row values were placed after "overweight" and "obesity" in place of an overweight value and an obesity value for the same subtype. Nothing in the paper shows how this happened.

Status

As of 2026-10-23:

Sources: publisher full text; PMC12235495; Europe PMC REST full-text XML and supplementary files for PMC12235495; Crossref REST API for doi:10.1001/jamanetworkopen.2025.19236.

Suggested correction

Abstract, Results. Replace "These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78])." with:

These associations were attenuated but remained for individuals with overweight (luminal A–like: OR, 2.18 [95% CI, 1.83-2.61]; luminal B–ERBB2-like: OR, 1.83 [95% CI, 1.32-2.55]) or obesity (luminal A–like: OR, 1.40 [95% CI, 1.02-1.92]; luminal B–ERBB2-like: OR, 1.68 [95% CI, 1.01-2.78]).

If word limits require a shorter sentence, the luminal A–like values alone (overweight 2.18 [1.83-2.61]; obesity 1.40 [1.02-1.92]) would give the BMI gradient correctly. All values are those already given in the Results text and in Supplement 1 eTable 3. No other part of the paper needs to change.

Recomputation script (python recompute.py.txt)
"""Check the BMI-stratified current-EPT odds ratios in the abstract of PMC12235495
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.19236) against the paper's own
Results text and Supplement 1 eTable 3, and redraw fig1/fig2 in this file's folder.

Run:  python recompute.py.txt      (needs numpy, matplotlib)
All inputs are copied from the published article (Abstract; Results, "Combined EPT Use
and Tumor Subtypes", paragraphs 2 and 3) and from Supplement 1 (eTables 3 and 7).
"""
import os
import math
import numpy as np

HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964

# ---------------------------------------------------------------- inputs
# Results text: current EPT use vs never use, by BMI stratum and intrinsic-like subtype.
# (OR, lower 95% CI, upper 95% CI). Identical values in Supplement 1 eTable 3, "Current use" rows.
RES = {
    ("healthy weight", "luminal A-like"):        (2.51, 2.26, 2.80),  # Results para 2
    ("healthy weight", "luminal B-like"):        (1.47, 1.17, 1.86),  # Results para 2
    ("healthy weight", "luminal B-ERBB2-like"):  (1.95, 1.61, 2.37),  # Results para 2
    ("overweight", "luminal A-like"):            (2.18, 1.83, 2.61),  # Results para 3
    ("overweight", "luminal B-ERBB2-like"):      (1.83, 1.32, 2.55),  # Results para 3
    ("obesity", "luminal A-like"):               (1.40, 1.02, 1.92),  # Results para 3
    ("obesity", "luminal B-ERBB2-like"):         (1.68, 1.01, 2.78),  # Results para 3
}
# Supplement 1, eTable 3 (base model), "Current use" rows, as printed.
ETABLE3 = {
    ("healthy weight", "luminal A-like"):        (2.51, 2.26, 2.80),
    ("healthy weight", "luminal B-ERBB2-like"):  (1.95, 1.61, 2.37),
    ("overweight", "luminal A-like"):            (2.18, 1.83, 2.61),
    ("overweight", "luminal B-ERBB2-like"):      (1.83, 1.32, 2.55),
    ("obesity", "luminal A-like"):               (1.40, 1.02, 1.92),
    ("obesity", "luminal B-ERBB2-like"):         (1.68, 1.01, 2.78),
}
# Supplement 1, eTable 7 (fully adjusted model), "Current use" rows.
ETABLE7 = {
    ("overweight", "luminal A-like"):            (2.18, 1.82, 2.61),
    ("overweight", "luminal B-ERBB2-like"):      (1.81, 1.30, 2.52),
    ("obesity", "luminal A-like"):               (1.41, 1.02, 1.93),
    ("obesity", "luminal B-ERBB2-like"):         (1.65, 0.99, 2.75),
}
# Abstract, Results: the BMI group each value is attached to (subtype as stated, if any).
ABS = [
    ("healthy weight", "luminal A-like",       (2.51, 2.26, 2.80)),
    ("healthy weight", "luminal B-ERBB2-like", (1.95, 1.61, 2.37)),
    ("overweight",     "(no subtype named)",   (1.40, 1.02, 1.92)),
    ("obesity",        "(no subtype named)",   (1.68, 1.01, 2.78)),
    ("healthy weight", "luminal B-like",       (1.47, 1.17, 1.86)),
]


def se_log(lo, hi):
    return (math.log(hi) - math.log(lo)) / (2 * Z)


def gmid(lo, hi):
    return math.exp((math.log(lo) + math.log(hi)) / 2)


def fmt(t):
    return f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"


print("=== 1. Internal consistency of the Results values (CI symmetric on the log scale) ===")
for k, (o, lo, hi) in RES.items():
    print(f"  {k[0]:15s} {k[1]:22s} OR {o:.2f}  geometric midpoint of CI {gmid(lo, hi):.3f}  "
          f"SE(log OR) {se_log(lo, hi):.3f}")

print("\n=== 2. Results text vs Supplement 1 eTable 3 ===")
for k, v in ETABLE3.items():
    print(f"  {k[0]:15s} {k[1]:22s} text {fmt(RES[k])}  eTable 3 {fmt(v)}  identical: {RES[k] == v}")

print("\n=== 3. Which Results estimate does each abstract value equal? (exact OR + CI match) ===")
for bmi, sub, v in ABS:
    hits = [f"{b} / {s}" for (b, s), rv in RES.items() if rv == v]
    same = any(b == bmi for (b, s), rv in RES.items() if rv == v)
    print(f"  abstract '{bmi}' {sub:22s} {fmt(v)} -> Results: {hits}; same BMI group: {same}")
ow = {s: v for (b, s), v in RES.items() if b == "overweight"}
print(f"  Results overweight estimates: " + "; ".join(f"{s} {fmt(v)}" for s, v in ow.items()))
print(f"  Does 1.40 (1.02-1.92) appear among any overweight estimate? "
      f"{any(v == (1.40, 1.02, 1.92) for v in ow.values())}")
print("  Fully adjusted eTable 7 values for the same cells: " +
      "; ".join(f"{b}/{s} {fmt(v)}" for (b, s), v in ETABLE7.items()))
print(f"  Does the abstract's 'overweight' triple equal any eTable 7 overweight cell? "
      f"{any(v == (1.40, 1.02, 1.92) for (b, s), v in ETABLE7.items() if b == 'overweight')}")

print("\n=== 4. Size of the discrepancy for women with overweight ===")
a = ABS[2][2]
for s, v in ow.items():
    ex_rep, ex_true = a[0] - 1, v[0] - 1
    d = math.log(v[0]) - math.log(a[0])
    se_d = math.sqrt(se_log(v[1], v[2]) ** 2 + se_log(a[1], a[2]) ** 2)
    print(f"  {s:22s} abstract 1.40 vs Results {v[0]:.2f}: ratio of ORs {v[0]/a[0]:.2f}; "
          f"excess odds {ex_rep:.2f} vs {ex_true:.2f} -> abstract shows {100*ex_rep/ex_true:.0f}% "
          f"of the excess (Results value is {ex_true/ex_rep:.1f}x)")
    print(f"  {'':22s} difference of log ORs {d:.3f}, SE {se_d:.3f}, z {d/se_d:.2f} "
          f"(treating the overweight and obesity strata as independent samples)")
print(f"  Overweight CI lower bounds vs abstract value: luminal A-like lower bound 1.83 > 1.40: {1.83 > 1.40}; "
      f"the two CIs overlap only between 1.83 and 1.92")

print("\n=== 5. BMI gradient as shown by the abstract and by the Results ===")
for s in ["luminal A-like", "luminal B-ERBB2-like"]:
    seq = [RES[(b, s)][0] for b in ["healthy weight", "overweight", "obesity"]]
    print(f"  Results {s:22s} healthy -> overweight -> obesity: {seq}; monotone decreasing: "
          f"{all(x > y for x, y in zip(seq, seq[1:]))}")
print(f"  Abstract 'overweight' 1.40 < 'obesity' 1.68: looks non-monotone: {1.40 < 1.68}")

print("\n=== 6. Significance unchanged ===")
for k in [("overweight", "luminal A-like"), ("overweight", "luminal B-ERBB2-like"),
          ("obesity", "luminal A-like"), ("obesity", "luminal B-ERBB2-like")]:
    o, lo, hi = RES[k]
    print(f"  {k[0]:11s} {k[1]:22s} CI excludes 1: {lo > 1}  z = {math.log(o)/se_log(lo, hi):.2f}")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch
from matplotlib.lines import Line2D

BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
                     "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
    fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
    plt.close(fig)


# ---- Figure 1: abstract vs Results text diagram
fig = plt.figure(figsize=(7.2, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-7, 62); ax.axis("off")

ax.add_patch(FancyBboxPatch((1.5, 6), 34.5, 50, boxstyle="round,pad=0.4,rounding_size=1.5",
                            fc="white", ec=ORA, lw=2))
ax.text(4, 52.5, "Abstract, Results", fontsize=11.5, weight="bold", va="top")
ax.text(4, 47.5, "As labelled in the abstract", fontsize=10.5, color=ORA, weight="bold", va="top")
ax.text(4, 42, "\u201cThese associations were\nattenuated but remained for\nindividuals with \u2026\u201d",
        fontsize=10.5, style="italic", va="top", linespacing=1.35)
ax.text(4, 27.6, "overweight", fontsize=10.5, color=SEC, va="center")
ax.text(4, 23.6, "OR 1.40 (1.02\u20131.92)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 16.6, "obesity", fontsize=10.5, color=SEC, va="center")
ax.text(4, 12.6, "OR 1.68 (1.01\u20132.78)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 8.4, "No subtype named for either", fontsize=10, color=SEC, va="center")

# Results grid (row labels on the right)
x0, cw, ch = 57, 15.5, 11
ax.text(x0 + cw, 58, "Results text, para. 3", fontsize=11.5, weight="bold", ha="center", va="top")
ax.text(x0 + cw, 53.5, "(identical in eTable 3)", fontsize=10.5, color=BLU, ha="center", va="top")
ax.text(x0 + cw / 2, 45, "Luminal A\u2013like", ha="center", fontsize=10.5, color=SEC)
ax.text(x0 + 1.5 * cw, 45, "Luminal B\u2013\nERBB2-like", ha="center", fontsize=10.5, color=SEC,
        linespacing=1.1)
rows = [("overweight", 32), ("obesity", 21)]
for b, y in rows:
    ax.text(x0 + 2 * cw + 1.2, y + ch / 2, b.capitalize(), ha="left", va="center", fontsize=10.5, color=SEC)
    for j, s in enumerate(["luminal A-like", "luminal B-ERBB2-like"]):
        o, lo, hi = RES[(b, s)]
        ax.add_patch(Rectangle((x0 + j * cw, y), cw, ch, fc="#e8f0fb" if b == "overweight" else "white",
                               ec=BLU, lw=2 if b == "obesity" else 1.2))
        ax.text(x0 + j * cw + cw / 2, y + ch * 0.64, f"{o:.2f}", ha="center", va="center",
                fontsize=12, weight="bold", color=BLU)
        ax.text(x0 + j * cw + cw / 2, y + ch * 0.28, f"({lo:.2f}\u2013{hi:.2f})", ha="center",
                va="center", fontsize=10, color=SEC)
ax.text(x0 - 1.5, 37.5, "Overweight row\n(shaded): not in\nthe abstract", ha="right", va="center",
        fontsize=10, color=SEC, linespacing=1.2)
ax.text(x0 + 2 * cw + 1.2, 16, "Current EPT\nvs never use", ha="left", va="top", fontsize=10,
        color=SEC, linespacing=1.2)

# connectors
ax.add_patch(FancyArrowPatch((32, 23.6), (x0 - 0.3, 26.5), arrowstyle="-|>", mutation_scale=13,
                             lw=2, color=TXT, shrinkA=2, shrinkB=2))
ax.add_patch(FancyArrowPatch((32, 12.6), (x0 + 1.5 * cw, 21 - 0.3), arrowstyle="-|>", mutation_scale=13,
                             lw=2, color=TXT, connectionstyle="arc3,rad=0.3", shrinkA=2, shrinkB=2))
ax.text(50, -1.8, "Conflict: both abstract values are the obesity-row estimates. The value labelled\n"
        "\u201coverweight\u201d is the obesity, luminal A\u2013like OR; the overweight luminal A\u2013like\n"
        "OR is 2.18 (1.83\u20132.61) and the overweight luminal B\u2013ERBB2-like OR is 1.83 (1.32\u20132.55).",
        ha="center", va="center", fontsize=10, linespacing=1.3)
save(fig, "fig1")

# ---- Figure 2: dot-and-interval plot, abstract labels vs Results values, by BMI
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
ax.set_facecolor(BG)
rows = []  # (label for y group, kind, value, text)
plan = [
    ("healthy weight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
                        ("abs", 0, "abstract, luminal A\u2013like"), ("abs", 1, "abstract, luminal B\u2013ERBB2-like")]),
    ("overweight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
                    ("abs", 2, "abstract, \u201coverweight\u201d")]),
    ("obesity", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
                 ("abs", 3, "abstract, \u201cobesity\u201d")]),
]
mk = {"luminal A-like": "o", "luminal B-ERBB2-like": "s"}
y, step, gap, centers = 0.0, 0.34, 0.55, {}
for b, items in plan:
    ys = []
    for it in items:
        if it[0] == "res":
            o, lo, hi = RES[(b, it[1])]
            col, m, lab = BLU, mk[it[1]], f"{o:.2f} {it[1].replace('-like', '')}"
        else:
            o, lo, hi = ABS[it[1]][2]
            col, m, lab = ORA, "D", f"{o:.2f} {it[2]}"
        ax.plot([lo, hi], [-y, -y], color=col, lw=2, solid_capstyle="butt")
        ax.plot(o, -y, marker=m, ms=8, color=col, ls="none")
        ax.text(hi * 1.03, -y, lab, va="center", fontsize=10, color=col)
        ys.append(-y); y += step
    centers[b] = np.mean(ys); y += gap
ax.axvline(1, color=SEC, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.9, 7.0)
ticks = [1, 1.5, 2, 3, 4]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_yticks(list(centers.values()))
ax.set_yticklabels(["Healthy\nweight", "Overweight", "Obesity"], fontsize=10.5)
ax.set_ylim(-y + gap - 0.3, 0.3)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
    ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, current EPT use vs never use (log scale; 95% CI)", fontsize=10.5)
h = [Line2D([], [], color=BLU, marker="o", ms=8, lw=2, label="Results text / eTable 3, luminal A\u2013like"),
     Line2D([], [], color=BLU, marker="s", ms=8, lw=2, label="Results text / eTable 3, luminal B\u2013ERBB2-like"),
     Line2D([], [], color=ORA, marker="D", ms=8, lw=2, label="Abstract, under the BMI group it names")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.15), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)

Abstract reports a diastolic blood pressure difference that the trial's own analyses do not show

JAMA Network Open (2021) · doi:10.1001/jamanetworkopen.2021.12417 · PMC8176331

Summary

This randomized trial of a smartphone lifestyle app (204 adults with type 2 diabetes in Singapore) lists diastolic blood pressure in its abstract among the outcomes with "intergroup differences favoring the intervention". The paper's Table 2 gives the diastolic between-group differences as −2.6 mm Hg (95% CI −6.2 to 0.9; P = .15) at 3 months and −2.4 mm Hg (95% CI −5.7 to 1.0; P = .17) at 6 months, and the complete-case analysis in Supplement 1 gives P = .13 and .08. All four intervals include zero, recomputation from the printed means and SDs agrees with them, and the paper's own Results text says that between-group differences in blood pressure were not observed at 6 months. The primary outcome (weight) and the glycemic results are unaffected, but the abstract credits the app with a blood pressure benefit that none of the paper's reported analyses supports at its stated threshold of 2-sided P < .05.

What the paper reports

Abstract, Results (the sentence follows the 6-month weight, HbA1c and medication results):

Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.

Results, "Metabolic Outcomes":

Between-group differences in blood pressure and lipids were not observed at 6 months.

Table 2, "Change in diastolic blood pressure, mm Hg" (blood pressure block: control group, 72 participants; intervention group, 67 participants). Columns: participants, control mean (SD) change, intervention mean (SD) change, mean difference (95% CI), P value, Cohen d. Superscript b is reproduced as "b":

3 mo | 139 | -1.8 (11.0) | -4.4 (9.9)b | -2.6 (-6.2 to 0.9) | .15 | 0.25

6 mo | 139 | -3.1 (9.2)b | -5.4 (11.1)b | -2.4 (-5.7 to 1.0) | .17 | 0.23

Table 2, footnote b (it marks within-group change from baseline, not between-group differences):

Statistically significant change from baseline to postintervention at P < .05, after Benjamini-Hochberg correction with false discovery rate at 0.20 and with 76 participants.

Supplement 1, eTable 1 (complete-case analysis), "∆ Diastolic blood pressure (mmHg)", as extracted from the PDF:

3 months 132 -1.6 (10.6) -4.1 (8.8)* -2.6 (-5.9 to 0.8) .13 0.26

6 months 131 -2.5 (7.9)* -5.4 (10.9)* -2.9 (-6.2 to 0.4) .08 0.30

Methods, "Statistical Analysis":

Type I error for multiple comparisons was adjusted using the Benjamini-Hochberg procedure with false discovery rate at 0.20.

Statistical significance was set at 2-sided P < .05.

Results, "Participants":

Baseline characteristics were similar between study groups, except for a significantly higher diastolic blood pressure in the control group.

What the paper's own numbers show

Method. The paper fitted generalized linear mixed models to change from baseline on 5 multiply imputed data sets, with recruitment source as a random effect. That model cannot be refitted from published summaries, so I checked the printed numbers against each other in two ways. (a) From the printed mean (SD) changes and group sizes (72 and 67 for blood pressure), I computed the intervention-minus-control difference with a pooled-variance t 95% CI and 2-sided P, a Welch P, and Cohen d. (b) I back-calculated the 2-sided P implied by each printed 95% CI (normal approximation, SE = CI width / 3.92). eTable 1 does not give the group split behind its n = 132 and 131, so its rows assume 66/66 and 66/65; across every split consistent with at most 72 control and 67 intervention participants, the recomputed P ranges only from .142 to .148 (3 mo) and .080 to .085 (6 mo). Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Diastolic BP difference, 3 mo (multiple imputation) Table 2 −2.6 (−6.2 to 0.9); P = .15; d = 0.25 −2.60 (−6.12 to 0.92); P = .146 (Welch .145; from printed CI .151); d = 0.25
Diastolic BP difference, 6 mo (multiple imputation) Table 2 −2.4 (−5.7 to 1.0); P = .17; d = 0.23 −2.30 from the rounded arm means (−5.71 to 1.11); P = .185 (Welch .188; from printed CI .160); d = 0.23
Diastolic BP difference, 3 mo (complete case) Supplement 1, eTable 1 −2.6 (−5.9 to 0.8); P = .13 −2.50 (−5.85 to 0.85); P = .143 (from printed CI .128)
Diastolic BP difference, 6 mo (complete case) Supplement 1, eTable 1 −2.9 (−6.2 to 0.4); P = .08 −2.90 (−6.19 to 0.39); P = .083 (from printed CI .085)
One-sided P implied by the printed CIs (not the paper's convention) Table 2; eTable 1 — .076 and .080 (multiple imputation, 3 and 6 mo); .064 and .042 (complete case)
Systolic BP difference, 3 mo / 6 mo (for comparison) Table 2 −4.2 (−8.8 to 0.3), P = .07 / −2.8 (−7.7 to 2.0), P = .25 −4.30 (−8.98 to 0.38), P = .072 / −2.80 (−7.66 to 2.06), P = .257
Systolic BP difference, 3 mo (complete case) Supplement 1, eTable 1 −4.7 (−9.3 to −0.2); P = .04 −4.70 (−9.25 to −0.15); P = .043, the only blood pressure comparison in the paper below .05
Fasting blood glucose difference, 6 mo (named in the same abstract sentence) Table 2 −12.6 (−23.4 to −3.6); P = .01 −12.6 (−21.4 to −3.8); P = .005
Calorie intake difference, 6 mo (named in the same abstract sentence, "dietary changes") Table 2 −305.8 (−441.0 to −170.7); P < .001 −305.5 (−441.1 to −169.9); P < .001
Baseline diastolic BP, control vs intervention Table 1; Results 85.7 (9.8) vs 82.7 (8.8); "significantly higher" in control difference 3.0 mm Hg; t = 2.30, P = .023
Participants with blood pressure data Table 2 139 of 204 randomized 68.1%

Both multiple-imputation analyses and both complete-case analyses of diastolic blood pressure have 95% CIs that include zero, and recomputation reproduces each printed P value to within rounding. Of the outcomes named in the abstract sentence, fasting blood glucose and the dietary outcomes have between-group P values below .05 in Table 2. Diastolic blood pressure does not, in any analysis the paper reports.

Figure

The abstract sentence set against the Results text and the Table 2 diastolic rows

Figure 1. The conflicting statements, quoted from the article. Left: the abstract lists diastolic blood pressure among the outcomes with intergroup differences favoring the intervention. Right: the Results text says between-group differences in blood pressure were not observed at 6 months, and Table 2 prints diastolic between-group differences whose 95% CIs include zero at both time points (orange: values printed in the paper). Below (blue): the 2-sided P values recomputed from Table 2's own mean (SD) changes and group sizes (.146 and .185), and those implied by the printed CIs (.151 and .160). The paper sets significance at 2-sided P < .05.

Dot-and-interval plot of the diastolic blood pressure between-group differences, reported and recomputed

Figure 2. Between-group difference in change in diastolic blood pressure (intervention minus control, mm Hg) with 95% CIs, for every analysis the paper reports: multiple imputation (Table 2) and complete case (Supplement 1, eTable 1). Orange circles and lines are the printed values. Blue squares and lines are recomputed from the printed mean (SD) changes with a pooled-variance t interval; the complete-case rows assume a 66/66 and 66/65 group split, which eTable 1 does not print. The vertical line marks no difference. Every interval, printed or recomputed, crosses it, and the P values at right are all above .05.

Why it matters

Blood pressure is a prespecified secondary outcome (Methods, "Outcomes"). The abstract's Results paragraph is where a reader learns which secondary outcomes differed between arms, and as written it tells that reader that the app improved diastolic blood pressure relative to usual care. The paper's analyses instead show point estimates of about −2.4 to −2.9 mm Hg with intervals running from about −6 to about +1 mm Hg. That result is compatible with no effect, and the paper's own Results text describes it as no difference. Abstracts are what is indexed, screened for systematic reviews and quoted in secondary literature, so a secondary outcome reported there as a benefit can be carried forward as one. The paper has been cited about 88 times. Meta-analysts who extract from Table 2 would get the correct estimate; readers who rely on the abstract would not.

Two further features of the data argue for caution even about the point estimate. Blood pressure was analysed in 139 of 204 randomized participants (68.1%). Baseline diastolic pressure was 3.0 mm Hg higher in the control group (P = .023), an imbalance the paper reports but does not adjust for in any presented analysis.

The error is limited in scope. The title, Key Points and abstract Conclusions mention only weight, glycemic and medication outcomes. The primary outcome (weight change, −2.4 kg at 6 months, P < .001), HbA1c (−0.4%, P = .01) and fasting glucose stand on Table 2. The main-text Conclusions refer more broadly to "significant improvements in weight and multiple metabolic profiles"; of the metabolic outcomes in Table 2, only HbA1c and fasting glucose have between-group P < .05 at 6 months.

Explanations considered

  1. A different time point. Table 2 and eTable 1 give diastolic results at 3 and 6 months only, with P from .08 to .17. No time point reaches P < .05. This does not account for the claim.

  2. Within-group rather than between-group change. Footnote b marks the intervention arm's diastolic decrease as significant at both times, so "favoring the intervention" could describe a within-arm improvement. But the abstract says "Intergroup differences", and the control arm's 6-month decrease is also marked significant, so a within-group reading does not distinguish the arms. This does not account for the wording.

  3. "Favoring" meaning only the direction of the point estimate. The other outcomes in the sentence (fasting glucose, dietary changes) are significant in Table 2. If direction alone were the criterion, systolic blood pressure (−4.2 and −2.8 mm Hg) and total and LDL cholesterol at 6 months would also qualify, and they are not listed. The paper's own Results sentence also treats the blood pressure result as no difference. This does not account for the claim.

  4. A one-sided test. The Methods state 2-sided tests. One-sided P values implied by the printed CIs are .076 and .080 for the main analysis. Only the 6-month complete-case result would fall below .05 one-sided (.042), and the paper presents the complete-case analysis as a supplementary check. This does not account for the claim.

  5. The paper's Benjamini-Hochberg procedure. BH adjustment can only raise a P value, so it cannot turn P = .15 or .17 into P < .05. For completeness: if all 21 between-group P values in Table 2 at a time point are treated as one family and q ≤ 0.20 is used as the criterion (the paper applies FDR 0.20 to within-group changes only), diastolic BP has q = .175 at 3 months and would pass, and q = .255 at 6 months and would not. On the same rule, systolic BP (q = .092), total cholesterol (q = .056) and LDL cholesterol (q = .173) would also pass at 3 months, and none of them appears in the abstract. The paper reports no between-group FDR analysis and sets significance at P < .05. This does not plausibly account for the claim.

  6. A systolic/diastolic slip. The only blood pressure comparison in the paper with P < .05 is complete-case systolic BP at 3 months (P = .04, eTable 1). If the abstract meant systolic, it would still rest on a supplementary analysis at the earlier time point, with P = .07 in the main analysis. This would explain the word but not support the claim.

  7. A baseline-adjusted analysis. Because baseline diastolic pressure was higher in the control group, an ANCOVA could give a larger estimate than the change-score analysis. As an illustration only, using the full-sample baseline gap from Table 1 (the 139-person subgroup's gap is not reported) and an assumed baseline coefficient between 0.3 and 1.0, the 6-month adjusted estimate would lie between −4.5 and −2.4 mm Hg. Its standard error cannot be derived from the published summaries. No such analysis is reported in the paper or supplement, so it cannot be the basis of the abstract statement.

  8. Conversion or transcription artifact. The abstract sentence, the Results sentence and the Table 2 rows read identically in the publisher's page, the live PMC page and the PMC archive. The printed P values agree with their own CIs and with recomputation. No artifact is indicated.

Status

Suggested correction

In the Abstract, Results, replace

Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.

with, for example,

Intergroup differences favoring the intervention were also noted for fasting blood glucose and dietary changes; between-group differences in blood pressure were not statistically significant (diastolic blood pressure at 6 months, −2.4 mm Hg [95% CI, −5.7 to 1.0]; P = .17).

If the authors intended a baseline-adjusted or other analysis, that analysis and its result should be reported in the Results and the supplement before it is summarized in the abstract. The editors may also wish to check the abstract's P = .001 for the HbA1c ≥ 8% subgroup against the P = .02 in the Results and Table 3, and the 6-month fasting glucose CI in Table 2.

Recomputation script (python recompute.py.txt)
"""Recompute the diastolic blood pressure (DBP) between-group results of PMC8176331
(JAMA Netw Open 2021, doi:10.1001/jamanetworkopen.2021.12417).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC and from Supplement 1:
  Table 1  baseline blood pressure, control n = 105, intervention n = 99
  Table 2  multiple-imputation (MI) results; blood pressure block: control 72, intervention 67
  eTable 1 (Supplement 1) complete-case (CC) results; total n = 132 (3 mo) and 131 (6 mo),
           split between groups not printed
The paper fitted generalized linear mixed models on 5 imputed data sets. Those models
cannot be refitted from published summaries, so two approximations are used:
  (a) a two-sample t test (pooled and Welch) from the printed mean (SD) changes and group n;
  (b) the 2-sided P value implied by the printed difference and its 95% CI
      (normal approximation, SE = CI width / (2 x 1.96)).
Both only check the printed numbers against each other; neither is a reanalysis.
"""
import os

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

# ------------------------------------------------------------------ inputs (from the paper)
# label: (control mean, SD, n, intervention mean, SD, n, reported diff, reported CI, reported P, reported d, location)
ROWS = {
    "DBP 3 mo (MI)": (-1.8, 11.0, 72, -4.4, 9.9, 67, -2.6, (-6.2, 0.9), ".15", 0.25, "Table 2"),
    "DBP 6 mo (MI)": (-3.1, 9.2, 72, -5.4, 11.1, 67, -2.4, (-5.7, 1.0), ".17", 0.23, "Table 2"),
    "SBP 3 mo (MI)": (-2.2, 14.7, 72, -6.5, 13.1, 67, -4.2, (-8.8, 0.3), ".07", 0.31, "Table 2"),
    "SBP 6 mo (MI)": (-5.0, 13.8, 72, -7.8, 15.2, 67, -2.8, (-7.7, 2.0), ".25", 0.19, "Table 2"),
    # complete case: group split not printed; 66/66 and 66/65 assumed (sensitivity below)
    "DBP 3 mo (CC)": (-1.6, 10.6, 66, -4.1, 8.8, 66, -2.6, (-5.9, 0.8), ".13", 0.26, "eTable 1"),
    "DBP 6 mo (CC)": (-2.5, 7.9, 66, -5.4, 10.9, 65, -2.9, (-6.2, 0.4), ".08", 0.30, "eTable 1"),
    "SBP 3 mo (CC)": (-1.4, 13.8, 66, -6.1, 12.6, 66, -4.7, (-9.3, -0.2), ".04", 0.36, "eTable 1"),
    "SBP 6 mo (CC)": (-4.4, 13.2, 66, -7.9, 14.2, 65, -3.5, (-8.2, 1.3), ".14", 0.26, "eTable 1"),
    # comparators named in the same abstract sentence (Table 2, n = 105 / 99)
    "FBG 3 mo (MI)": (-3.6, 32.4, 105, -18.0, 37.8, 99, -14.4, (-23.4, -3.6), ".005", 0.41, "Table 2"),
    "FBG 6 mo (MI)": (-1.8, 25.2, 105, -14.4, 37.8, 99, -12.6, (-23.4, -3.6), ".01", 0.39, "Table 2"),
    "Calories 6 mo (MI)": (-245.8, 466.7, 105, -551.3, 515.4, 99, -305.8, (-441.0, -170.7), "<.001", 0.62, "Table 2"),
}
BASELINE_DBP = (85.7, 9.8, 105, 82.7, 8.8, 99)   # Table 1, control vs intervention
BASELINE_SBP = (135.3, 13.0, 105, 134.7, 13.5, 99)
BP_N, TOTAL_N = 139, 204                        # Table 2 blood pressure block vs randomized


def recompute(mc, sc, nc, mi, si, ni, rep_diff, rep_ci):
    d = mi - mc
    sp = np.sqrt(((nc - 1) * sc**2 + (ni - 1) * si**2) / (nc + ni - 2))
    se_p = sp * np.sqrt(1 / nc + 1 / ni)
    df = nc + ni - 2
    q = stats.t.ppf(0.975, df)
    p_pool = 2 * stats.t.sf(abs(d / se_p), df)
    se_w = np.sqrt(sc**2 / nc + si**2 / ni)
    dfw = se_w**4 / ((sc**2 / nc) ** 2 / (nc - 1) + (si**2 / ni) ** 2 / (ni - 1))
    p_welch = 2 * stats.t.sf(abs(d / se_w), dfw)
    se_ci = (rep_ci[1] - rep_ci[0]) / (2 * Z)
    p_ci = 2 * stats.norm.sf(abs(rep_diff) / se_ci)
    return dict(diff=d, lo=d - q * se_p, hi=d + q * se_p, p_pool=p_pool, p_welch=p_welch,
                p_ci=p_ci, d=abs(d) / sp, p_one=p_pool / 2, mid_ci=(rep_ci[0] + rep_ci[1]) / 2)


R = {}
print("=== 1. Between-group differences (intervention minus control) ===")
print("  diff/CI from rounded mean (SD) changes, pooled-variance t; P also by Welch and from the printed CI\n")
for k, v in ROWS.items():
    mc, sc, nc, mi, si, ni, rd, rci, rp, rdd, loc = v
    r = recompute(mc, sc, nc, mi, si, ni, rd, rci)
    R[k] = r
    print(f"{k:20s} [{loc}] reported {rd:+.1f} ({rci[0]} to {rci[1]}), P={rp}, d={rdd}")
    print(f"{'':20s} recomputed {r['diff']:+.2f} ({r['lo']:+.2f} to {r['hi']:+.2f}); "
          f"P pooled={r['p_pool']:.3f}, Welch={r['p_welch']:.3f}, from printed CI={r['p_ci']:.3f}; "
          f"d={r['d']:.2f}; one-sided P={r['p_one']:.3f}; midpoint of printed CI={r['mid_ci']:+.2f}")

print("\n=== 2. Complete-case rows: sensitivity to the unknown group split ===")
for k, tot in (("DBP 3 mo (CC)", 132), ("DBP 6 mo (CC)", 131)):
    mc, sc, _, mi, si, _, rd, rci, *_ = ROWS[k]
    ps = []
    for nc in range(tot - 72, 73):          # control at most 72, intervention at most 67
        ni = tot - nc
        if ni > 67 or ni < 1:
            continue
        ps.append(recompute(mc, sc, nc, mi, si, ni, rd, rci)["p_pool"])
    print(f"{k}: over every split with control <= 72 and intervention <= 67, "
          f"2-sided P ranges {min(ps):.3f} to {max(ps):.3f}")

print("\n=== 3. Does any DBP between-group test reach 2-sided P < .05? ===")
dbp = [k for k in ROWS if k.startswith("DBP")]
for k in dbp:
    r = R[k]
    print(f"  {k:16s} printed CI includes 0: {ROWS[k][7][0] < 0 < ROWS[k][7][1]}; "
          f"recomputed CI includes 0: {r['lo'] < 0 < r['hi']}; max P (any method) "
          f"{max(r['p_pool'], r['p_welch'], r['p_ci']):.3f}, min P {min(r['p_pool'], r['p_welch'], r['p_ci']):.3f}")
print(f"  any DBP row with recomputed 2-sided P < .05: {any(min(R[k]['p_pool'], R[k]['p_welch'], R[k]['p_ci']) < .05 for k in dbp)}")

print("\n=== 4. Baseline imbalance and coverage ===")
for lab, b in (("DBP", BASELINE_DBP), ("SBP", BASELINE_SBP)):
    t, p = stats.ttest_ind_from_stats(*b)
    print(f"  baseline {lab}: control {b[0]} ({b[1]}) vs intervention {b[3]} ({b[4]}); "
          f"difference {b[0] - b[3]:.1f} mm Hg; t={t:.2f}, P={p:.3f}")
print(f"  participants in the blood pressure analysis: {BP_N}/{TOTAL_N} = {100 * BP_N / TOTAL_N:.1f}%")

print("\n=== 5. One-sided P implied by the printed CIs (the paper states 2-sided tests) ===")
for k in dbp:
    print(f"  {k:16s} one-sided P from printed CI = {R[k]['p_ci'] / 2:.3f}")

print("\n=== 6. Benjamini-Hochberg q over all 21 between-group P values printed in Table 2 ===")
# printed P values, Table 2, in row order: weight kg, weight %, BMI, HbA1c, FBG, SBP, DBP, cost,
# total chol, LDL, HDL, TG, creatinine, calories, carbohydrate, sugar, protein, total fat, sat fat, fiber, PA.
# "<.001" is entered as .001 (conservative: larger than the true value).
NAMES = ["weight kg", "weight %", "BMI", "HbA1c", "FBG", "SBP", "DBP", "cost", "total chol", "LDL",
         "HDL", "TG", "creatinine", "calories", "carbohydrate", "sugar", "protein", "total fat",
         "sat fat", "fiber", "physical activity"]
P_T2 = {
    "3 mo": [.001, .001, .001, .006, .005, .07, .15, .04, .04, .14, .90, .48, .91, .001, .001, .009, .006, .001, .001, .02, .007],
    "6 mo": [.001, .001, .001, .01, .01, .25, .17, .01, .65, .53, .76, .49, .63, .001, .001, .01, .09, .001, .001, .54, .009],
}


def bh(p):
    p = np.asarray(p, float); m = len(p); o = np.argsort(p)
    q = np.empty(m); run = 1.0
    for rank in range(m, 0, -1):
        i = o[rank - 1]; run = min(run, p[i] * m / rank); q[i] = run
    return q


for tp, ps in P_T2.items():
    q = bh(ps)
    print(f"  {tp}: q(DBP)={q[NAMES.index('DBP')]:.3f}  q(SBP)={q[NAMES.index('SBP')]:.3f}  "
          f"q(LDL)={q[NAMES.index('LDL')]:.3f}  q(total chol)={q[NAMES.index('total chol')]:.3f}")
    print(f"        outcomes with q <= 0.20: {[n for n, qq in zip(NAMES, q) if qq <= 0.20]}")

print("\n=== 7. Illustration only: what a baseline-adjusted (ANCOVA) estimate could be ===")
# change-score difference = (F_I - F_C) - (B_I - B_C); ANCOVA = (F_I - F_C) - b (B_I - B_C)
# => ANCOVA = change difference + (1 - b)(B_I - B_C). Baseline gap from Table 1 (all 204, not the 139).
gap = BASELINE_DBP[3] - BASELINE_DBP[0]
for b in (0.3, 0.5, 0.7, 1.0):
    print(f"  6 mo, assumed b={b:.1f}: {-2.4 + (1 - b) * gap:+.1f} mm Hg (standard error not derivable)")

print("\n=== 8. Side observations (not the subject of the note) ===")
r = R["FBG 6 mo (MI)"]
print(f"  FBG 6 mo: printed CI (-23.4 to -3.6) is identical to the 3-mo row; its midpoint is {r['mid_ci']:+.1f}, "
      f"not the printed -12.6; recomputed {r['diff']:+.1f} ({r['lo']:.1f} to {r['hi']:.1f}), P={r['p_pool']:.3f}")
# HbA1c >= 8% subgroup, 6 mo, Table 3: control -1.0 (1.4) n=29, intervention -1.8 (1.4) n=26; -0.9 (-1.6 to -0.1), P=.02
hb = recompute(-1.0, 1.4, 29, -1.8, 1.4, 26, -0.9, (-1.6, -0.1))
print(f"  HbA1c>=8% subgroup 6 mo: abstract P=.001; Results text and Table 3 P=.02; "
      f"P from Table 3 CI={hb['p_ci']:.3f}; Welch from Table 3 summaries={hb['p_welch']:.3f}")

# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
                     "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
                     "ytick.color": INK2, "svg.fonttype": "none"})


def save(fig, name):
    for ext in ("svg", "png"):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
    plt.close(fig)


# ---- Figure 1: abstract vs Results / Table 2
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")


def panel(x, y, w, h, title, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, weight="bold", va="top", color=INK)


panel(0.15, 1.05, 2.85, 3.25, "Abstract, Results", INK2)
ax.text(0.29, 3.68,
        "“Intergroup differences favoring\nthe intervention were also noted\n"
        "for fasting blood glucose,\ndiastolic blood pressure, and\ndietary changes.”",
        fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(0.29, 1.62, "Claim: a between-group\ndifference in diastolic BP", fontsize=10,
        va="top", color=INK2, style="italic", linespacing=1.35)

panel(3.75, 1.05, 3.3, 3.25, "Results text and Table 2", INK2)
ax.text(3.89, 3.68, "“Between-group differences in blood\npressure and lipids were not\nobserved at 6 months.”",
        fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(3.89, 2.55, "Table 2, change in diastolic BP\n(intervention − control, mm Hg)",
        fontsize=10, va="top", color=INK2, linespacing=1.35)
for i, (k, lab) in enumerate((("DBP 3 mo (MI)", "3 mo"), ("DBP 6 mo (MI)", "6 mo"))):
    v = ROWS[k]
    yy = 1.88 - i * 0.28
    ax.text(3.89, yy, f"{lab}: −{abs(v[6]):.1f} ({v[7][0]:.1f} to {v[7][1]:.1f}), P = {v[8]}".replace("-", "−"),
            fontsize=10.5, va="top", color=ORANGE, weight="bold")
ax.text(3.89, 1.27, "printed in the paper", fontsize=10, va="top", color=ORANGE)

ax.annotate("", xy=(3.72, 2.7), xytext=(3.03, 2.7),
            arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.375, 2.82, "conflict", ha="center", fontsize=10, color=INK, weight="bold")

r3, r6 = R["DBP 3 mo (MI)"], R["DBP 6 mo (MI)"]
ax.text(0.15, 0.78,
        f"Recomputed from Table 2's own mean (SD) changes: P = {r3['p_pool']:.3f} (3 mo), {r6['p_pool']:.3f} (6 mo);",
        fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.50,
        f"P implied by the printed 95% CIs: {r3['p_ci']:.3f} and {r6['p_ci']:.3f}. Every CI, printed or recomputed, includes 0.",
        fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.22, "Orange: printed in the paper. Blue: recomputed from the paper's numbers.",
        fontsize=10, color=INK2, va="top")
save(fig, "fig1")

# ---- Figure 2: dot-and-interval, DBP between-group differences, reported vs recomputed
order = ["DBP 3 mo (MI)", "DBP 6 mo (MI)", "DBP 3 mo (CC)", "DBP 6 mo (CC)"]
names = ["3 mo, multiple imputation\n(Table 2)", "6 mo, multiple imputation\n(Table 2)",
         "3 mo, complete case\n(eTable 1)", "6 mo, complete case\n(eTable 1)"]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
off = 0.13
for i, k in enumerate(order):
    y = len(order) - 1 - i
    v, r = ROWS[k], R[k]
    ax.plot(v[7], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
    ax.plot(v[6], y + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5,
            label="Reported (paper)" if i == 0 else None)
    ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
    ax.plot(r["diff"], y - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5,
            label="Recomputed from the paper's mean (SD) changes" if i == 0 else None)
    ax.text(3.0, y + off, f"P = {v[8]}", va="center", fontsize=10, color=INK)
    ax.text(3.0, y - off, f"P = {r['p_pool']:.3f}".replace("0.", ".", 1), va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.08, 3.55, "no difference", fontsize=10, color=INK2, va="center")
ax.set_yticks(range(len(order)))
ax.set_yticklabels(names[::-1], fontsize=10, color=INK)
ax.set_xlim(-7, 4.2); ax.set_ylim(-0.6, 3.75)
ax.set_xticks(range(-7, 3))
ax.set_xlabel("Diastolic BP change, intervention − control (mm Hg), 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.legend(loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)

Hospital-stay estimate of 8.4 days lies outside its own reported 95% CI of 4.4–6.7

Kidney Int (2021) · doi:10.1016/j.kint.2021.02.026 · PMC7927648

Summary

This retrospective cohort of 152 children hospitalized with acute COVID-19 or MIS-C reports, in the Abstract, the Results and Table 4, that acute kidney injury (AKI, stages 1–3) was associated with an 8.4-day longer hospital stay, with a 95% CI of 4.4 to 6.7 and P < 0.0001, from unadjusted simple linear regression. The point estimate lies 1.7 days above the upper bound of its own interval. A regression CI is centred on its coefficient, and the stated interval is centred on 5.55. Every other row of Table 4 except one is centred on its estimate. At least one of the three printed numbers is therefore wrong, and the paper does not contain enough information to say which. The direction and statistical significance of the association survive every reconstruction considered, but the precision is overstated or the size is misstated. A second Table 4 row, for length of mechanical ventilation, is also not centred on its estimate.

What the paper reports

Abstract:

In unadjusted models, patients with acute kidney injury had 8.4 days longer hospitalizations compared to patients without acute kidney injury (95% confidence interval, 4.4-6.7).

Results, "Clinical course and outcomes", third paragraph:

In addition, patients with AKI had a significantly greater length of hospital stay of 8.4 days compared with patients without AKI (95% CI, 4.4–6.7; P < 0.0001).

Table 4, "Association of AKI with hospital outcomes", under the column header "β (95% CI)" (columns: outcome by AKI stage | β (95% CI) | P value):

Length of hospital stay, d AKI stage 1 + 2 + 3 | 8.4 (4.4 to 6.7) | <0.0001 AKI stage 2 + 3 | 5.2 (0.07 to 10.2) | 0.047 Length of PICU stay, d AKI stage 1 + 2 + 3 | 5 (0.86 to 9.1) | 0.02 AKI stage 2 + 3 | 2.3 (–3.7 to 8.3) | 0.45 Length of mechanical ventilation, d AKI stage 1 + 2 + 3 | –3.9 (–9.4 to 12.7) | 0.15 AKI stage 2 + 3 | –1.08 (–7.4 to 5.3) | 0.71

The same table gives mortality as a hazard ratio: "AKI stage 1 + 2 + 3 | 0.57 (0.03 to 11.8) | 0.72".

Methods, "Statistical analysis":

Multiple imputation was used for missing values in regression analysis. Subsequently, simple linear regression analysis was performed to assess the relationship between AKI and continuous outcomes, including PICU and hospital length of stay and length of time on mechanical ventilation.

The conclusions that draw on this result. Discussion, first paragraph:

AKI, in unadjusted models, was significantly associated with longer PICU and hospital length of stay.

Discussion, "Conclusions":

Pediatric COVID-19–related AKI, similar to reports in larger AKI studies, was associated with poor outcomes, such as increased PICU and hospital length of stay.

What the paper's own numbers show

Method. For simple linear regression the 95% CI of a coefficient is β ± t(0.975, df) × SE. It always contains β and is centred on it. A CI pooled across multiply imputed data sets with Rubin's rules has the same form, so the same holds. For each Table 4 row I checked whether the estimate lies inside its interval and computed the centre of the interval, the SE implied by its width (df = 150, from N = 152), and the P value implied by β/SE. For the hazard ratio I did the same on the log scale with a normal critical value. I then rebuilt the hospital-stay row under each assumption about which number is correct. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Hospital stay, AKI 1–3: is β inside its CI? Abstract; Results; Table 4 8.4 (4.4 to 6.7) No. 8.4 is 1.7 d above the upper bound 6.7
Hospital stay, AKI 1–3: centre of the CI Table 4 8.4 5.55 (half-width 1.15; SE from width 0.582)
Same row, if 8.4 and the lower bound 4.4 are right derived upper bound 6.7; P < 0.0001 upper bound 12.4; SE 2.024; t = 4.15; P = 5.6 × 10⁻⁵ (consistent with P < 0.0001)
Same row, if 8.4 and the upper bound 6.7 are right derived — lower bound 2 × 8.4 − 6.7 = 10.1, above the estimate: impossible
Same row, if the CI 4.4–6.7 is right derived β = 8.4 β = 5.55; t = 9.54; P = 3.8 × 10⁻¹⁷ (also < 0.0001)
SE under the 4.4–12.4 reconstruction vs SE implied by the printed width derived — 2.024 / 0.582 = 3.48 times larger
Hospital stay, AKI 2–3 Table 4 5.2 (0.07 to 10.2), P = 0.047 centre 5.135; P from β/SE 0.044: consistent
PICU stay, AKI 1–3 Table 4 5 (0.86 to 9.1), P = 0.02 centre 4.98; P 0.018: consistent
PICU stay, AKI 2–3 Table 4 2.3 (–3.7 to 8.3), P = 0.45 centre 2.30; P 0.45: consistent
Mechanical ventilation, AKI 1–3 Table 4 –3.9 (–9.4 to 12.7), P = 0.15 inside, but centre +1.65; arms 5.5 below and 16.6 above; P from β/SE 0.49 (0.45 if only the 11 ventilated patients in Table 3 were analysed, df = 9): inconsistent
Mechanical ventilation, AKI 2–3 Table 4 –1.08 (–7.4 to 5.3), P = 0.71 centre –1.05; P 0.74: consistent
Mortality hazard ratio, AKI 1–3 Table 4 0.57 (0.03 to 11.8), P = 0.72 geometric centre 0.595; P 0.71: consistent
Median hospital stay, without vs with AKI Table 3 acute COVID-19 2.8 vs 22.2 d; MIS-C 4.6 vs 8.2 d differences 19.4 and 3.6 d (medians by subgroup, so they cannot reproduce the pooled β)

Five of the seven Table 4 rows are centred on their estimates to within rounding, and their printed P values agree with the P implied by their intervals. The hospital-stay row for AKI stages 1–3 is the only one whose estimate falls outside its interval. The mechanical-ventilation row for AKI stages 1–3 is the only other row that is not centred on its estimate, and its printed P = 0.15 does not match the P implied by its interval.

A few smaller inconsistencies turned up during the comparison. None of them affects a conclusion:

Quantity Location Reported Recomputed
PICU stay, AKI 1–3 Results text vs Table 4 text "95% CI, 0.86–9; P = 0.025"; table "0.86 to 9.1", 0.02 same estimate, different rounding of the bound and P
Acute COVID-19 group size Table 3 vs Table 1 and Abstract Table 3 "No AKI (N = 88)", "AKI (N = 8)" 88 + 8 = 96; Table 1 gives 97 overall and No AKI N = 89
Mechanical ventilation, acute COVID-19 with AKI Table 3 2 (33.3) of N = 8 2/8 = 25.0%
Vasopressors, acute COVID-19 with AKI Table 3 5 (32) of N = 8 5/8 = 62.5%

Figure

The Table 4 day-scale estimates and intervals, with the centre of each reported interval

Figure 1. The six day-scale rows of Table 4 on one axis, with no effect (0 days) as the thin vertical line. Orange dots and bars are the reported β and 95% CI. Blue rings mark the centre of each reported interval, computed from its printed bounds; where the orange dot sits inside the blue ring the estimate is centred, as it must be for a linear-regression CI. For hospital stay with AKI stages 1–3 the estimate 8.4 lies outside the interval, 1.7 days above its upper bound, and the interval's centre is 5.55. The dashed blue row below it is one possible reconstruction, not a correction: the interval centred on 8.4 that keeps the printed lower bound 4.4 (4.4 to 12.4, P = 6 × 10⁻⁵). The mechanical-ventilation row for AKI stages 1–3 contains its estimate, but its centre (+1.65) is 5.55 days from the estimate.

Why it matters

The result appears in the Abstract as one of the study's main findings, in the Results text and in Table 4, and it supports the Discussion and Conclusions statements that AKI was associated with longer hospital stays. As printed, the interval is impossible. A reader cannot tell whether the effect is about 8.4 days with a wide interval (for example 4.4 to 12.4) or about 5.6 days with the narrow interval printed. The two readings differ in effect size by almost 3 days and in standard error by a factor of about 3.5.

What does not change: every reconstruction leaves the lower bound well above zero and gives P < 0.0001, so the qualitative statement that AKI was associated with a significantly longer unadjusted hospital stay is not overturned. Table 3 medians (2.8 vs 22.2 days in acute COVID-19, 4.6 vs 8.2 days in MIS-C) and the PICU-stay result (5 days, 0.86 to 9.1) point the same way. Length of stay is a secondary outcome. The paper states that "The primary outcome of this study was the incidence rate of AKI" (11.8% overall), and that result, the albumin and white-cell odds ratios, the MIS-C systolic-dysfunction comparison and the mortality result do not depend on this row.

The mechanical-ventilation row for AKI stages 1–3 is a second, smaller problem: the printed estimate, interval and P value cannot all come from one symmetric interval. It is reported as non-significant, and every value implied by its interval is non-significant too, so the paper's statement that there was no significant association with length of mechanical ventilation is not affected.

Explanations considered

Status

Suggested correction

  1. Re-run or re-extract the unadjusted linear regression of hospital length of stay on AKI (stages 1–3) and correct the estimate, the 95% CI or both in the Abstract, the Results text ("Clinical course and outcomes") and Table 4, together with the P value if it changes.
  2. Check the mechanical-ventilation row for AKI stages 1–3 in Table 4, whose estimate (–3.9), interval (–9.4 to 12.7) and P value (0.15) are mutually inconsistent, and state how many patients each length-of-ventilation model included.
  3. Minor: reconcile the PICU-stay interval and P value between the Results text and Table 4; correct the acute COVID-19 No AKI column size in Table 3 (88 vs 89); and check the percentages for mechanical ventilation (2 of 8) and vasopressors (5 of 8) in the acute COVID-19 AKI column of Table 3.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 4 regression intervals of PMC7927648
(Kidney Int 2021, doi:10.1016/j.kint.2021.02.026).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png next to this file, and writes five candidate figure designs to
candidates/ (c1..c5 .svg/.png plus contact_sheet.png). fig1 is a copy of the candidate
named in CHOSEN below.

Inputs are copied from the paper as archived in PMC:
  Table 4 (beta or hazard ratio, 95% CI, P), also quoted in the Abstract and Results
  Methods: unadjusted simple linear regression (SPSS 26), multiple imputation; cohort N = 152
  Table 3: medians (IQR) by AKI status, and counts used in the minor checks
"""
import math
import os
import shutil

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402
from scipy import stats  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
CHOSEN = "c1"

# ---------------------------------------------------------------- inputs (from the paper)
N = 152                       # Abstract / Results: "The cohort included 152 patients"
DF = N - 2                    # simple linear regression, one predictor
TCRIT = stats.t.ppf(0.975, DF)
# label: (estimate, lower, upper, printed P, scale)  -- Table 4
ROWS = [
    ("Hospital stay, AKI 1-3", 8.4, 4.4, 6.7, "<0.0001", "lin"),
    ("Hospital stay, AKI 2-3", 5.2, 0.07, 10.2, "0.047", "lin"),
    ("PICU stay, AKI 1-3", 5.0, 0.86, 9.1, "0.02", "lin"),
    ("PICU stay, AKI 2-3", 2.3, -3.7, 8.3, "0.45", "lin"),
    ("Mechanical ventilation, AKI 1-3", -3.9, -9.4, 12.7, "0.15", "lin"),
    ("Mechanical ventilation, AKI 2-3", -1.08, -7.4, 5.3, "0.71", "lin"),
    ("Mortality (hazard ratio), AKI 1-3", 0.57, 0.03, 11.8, "0.72", "log"),
]
N_VENTILATED = 4 + 2 + 1 + 4  # Table 3, "Mechanical ventilation, n": 4, 2, 1, 4

R = {}
print("=== 1. Is each Table 4 estimate centred in its own 95% CI? ===")
print(f"t critical value, df = {DF}: {TCRIT:.4f}")
for lab, b, lo, hi, p, sc in ROWS:
    if sc == "log":
        mid = math.exp((math.log(lo) + math.log(hi)) / 2)
        half = (math.log(hi) - math.log(lo)) / 2
        se = half / 1.959964
        pos = (math.log(b) - math.log(mid)) / half
        pp = 2 * stats.norm.sf(abs(math.log(b) / se))
    else:
        mid = (lo + hi) / 2
        half = (hi - lo) / 2
        se = half / TCRIT
        pos = (b - mid) / half
        pp = 2 * stats.t.sf(abs(b / se), DF)
    inside = lo <= b <= hi
    R[lab] = dict(b=b, lo=lo, hi=hi, p=p, sc=sc, mid=mid, half=half, se=se, pos=pos, pp=pp, inside=inside)
    print(f"{lab:36s} est {b:6.2f}  CI {lo:6.2f} to {hi:6.2f}  inside={inside!s:5s}  "
          f"CI centre {mid:6.3f}  est - centre {b - mid if sc == 'lin' else b / mid:+.3f}"
          f"{'' if sc == 'lin' else ' (ratio)'}  position in CI {pos:+.2f}  "
          f"SE from CI {se:.3f}  P from est/SE {pp:.2g}  (printed {p})")

print("\n=== 2. The hospital-stay row, AKI stages 1-3 ===")
b, lo, hi = 8.4, 4.4, 6.7
print(f"8.4 within [4.4, 6.7]: {lo <= b <= hi}; 8.4 - 6.7 = {b - hi:.1f} days above the upper bound")
print(f"centre of printed CI = {(lo + hi) / 2:.2f}; half-width = {(hi - lo) / 2:.2f}")
se_ci = (hi - lo) / 2 / TCRIT
print(f"SE implied by the printed CI width = {se_ci:.3f}")
up_rec = 2 * b - lo
se_rec = (b - lo) / TCRIT
p_rec = 2 * stats.t.sf(b / se_rec, DF)
print(f"(a) if 8.4 and the lower bound 4.4 are right: upper bound = 2*8.4 - 4.4 = {up_rec:.1f}; "
      f"SE = {se_rec:.3f}; t = {b / se_rec:.2f}; P = {p_rec:.1e}  (printed P < 0.0001: {p_rec < 1e-4})")
print(f"    SE under (a) / SE implied by printed width = {se_rec / se_ci:.2f}")
lo_b = 2 * b - hi
print(f"(b) if 8.4 and the upper bound 6.7 are right: lower bound = 2*8.4 - 6.7 = {lo_b:.1f} > 8.4, impossible")
mid = (lo + hi) / 2
p_c = 2 * stats.t.sf(mid / se_ci, DF)
print(f"(c) if the CI 4.4-6.7 is right: estimate = {mid:.2f}; t = {mid / se_ci:.2f}; P = {p_c:.1e} "
      f"(also < 0.0001: {p_c < 1e-4})")
print("Every reconstruction keeps the lower bound well above 0, so the direction and significance survive.")

print("\n=== 3. Side finding: mechanical-ventilation row, AKI stages 1-3 ===")
b, lo, hi = -3.9, -9.4, 12.7
for df, lab in ((DF, f"df = {DF} (whole cohort)"), (N_VENTILATED - 2, f"df = {N_VENTILATED - 2} (only the {N_VENTILATED} ventilated patients, Table 3)")):
    tc = stats.t.ppf(0.975, df)
    se = (hi - lo) / 2 / tc
    print(f"  {lab}: CI centre {(lo + hi) / 2:+.2f}; SE {se:.3f}; P from -3.9/SE {2 * stats.t.sf(abs(b) / se, df):.2f} (printed 0.15)")
print(f"  estimate - lower = {b - lo:.1f}; upper - estimate = {hi - b:.1f}")

print("\n=== 4. Plausibility of an ~8-day difference (Table 3 medians, not a regression) ===")
print(f"acute COVID-19: 22.2 - 2.8 = {22.2 - 2.8:.1f} d; MIS-C: 8.2 - 4.6 = {8.2 - 4.6:.1f} d")

print("\n=== 5. Smaller inconsistencies noted in passing ===")
print("PICU stay, AKI 1-3: Results text '95% CI, 0.86-9; P = 0.025'; Table 4 '0.86 to 9.1', P 0.02")
print(f"Table 3 acute COVID-19 columns: 88 + 8 = {88 + 8}; Table 1 and Abstract: 97 (No AKI N = 89 in Table 1)")
print(f"Table 3 mechanical ventilation, acute COVID-19 with AKI: 2/8 = {200 / 8:.1f}% (printed 33.3)")
print(f"Table 3 vasopressors, acute COVID-19 with AKI: 5/8 = {500 / 8:.1f}% (printed 32)")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
    "xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
    "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5   # 2 px at 96 px/in is 1.5 pt
MS = 8.5


def save(fig, stem, folder=CAND):
    fig.savefig(os.path.join(folder, stem + ".svg"))
    fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
    plt.close(fig)


def tidy(ax, xgrid=True, ygrid=False):
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    if xgrid:
        ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
    if ygrid:
        ax.yaxis.grid(True, color=GRID, lw=0.8, zorder=0)
    ax.set_axisbelow(True)


LIN = [r for r in ROWS if r[5] == "lin"]
FLAG = "Hospital stay, AKI 1-3"

# --- c1: all day-scale Table 4 rows, reported intervals with CI centres, plus the reconstruction
fig, ax = plt.subplots(figsize=(7.2, 4.9))
labels, ys = [], []
y = 0
for lab, b, lo, hi, p, sc in LIN:
    r = R[lab]
    ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="butt", zorder=2)
    for e in (lo, hi):
        ax.plot([e, e], [y - 0.13, y + 0.13], color=ORANGE, lw=LW, zorder=2)
    ax.plot([b], [y], "o", ms=MS, color=ORANGE, zorder=4)
    ax.plot([r["mid"]], [y], "o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=5)
    labels.append(f"{lab}\n{b:g} ({lo:g} to {hi:g}), P {p if p.startswith('<') else '= ' + p}")
    ys.append(y)
    if lab == FLAG:
        y += 0.9
        ax.plot([4.4, up_rec], [y, y], color=BLUE, lw=LW, ls=(0, (4, 2)), zorder=2)
        for e in (4.4, up_rec):
            ax.plot([e, e], [y - 0.13, y + 0.13], color=BLUE, lw=LW, zorder=2)
        ax.plot([8.4], [y], "D", ms=MS - 1, color=BLUE, zorder=4)
        labels.append(f"Same row, rebuilt from 8.4 and 4.4\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e-"))
        ys.append(y)
        ax.annotate("8.4 lies 1.7 d\nabove its own\nupper bound", xy=(8.4, 0), xytext=(10.6, -0.05),
                    fontsize=10, color=TXT, va="center", ha="left",
                    arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
    y += 1.15
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.invert_yaxis()
ax.set_xlim(-11, 16.5)
ax.set_xticks(range(-10, 16, 5))
ax.set_xlabel("β: difference with vs without AKI, days")
tidy(ax)
fig.legend(handles=[
    Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported β and 95% CI (Table 4)"),
    Line2D([], [], marker="o", ls="none", ms=MS + 5, mfc="none", mec=BLUE, mew=2,
           label="Centre of the reported CI (recomputed); a ring around a dot means the estimate is centred"),
    Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS - 1, color=BLUE,
           label="Interval centred on 8.4 with lower bound 4.4 (recomputed)")],
    loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "c1")

# --- c2: the flagged row alone, three readings on one axis
fig, ax = plt.subplots(figsize=(7.2, 3.3))
items = [
    ("Reported (Abstract, Results, Table 4)\n8.4 (4.4 to 6.7), P < 0.0001", 8.4, 4.4, 6.7, ORANGE, "o", "-"),
    (f"If the CI is right: estimate = its centre\n{(4.4 + 6.7) / 2:.2f} (4.4 to 6.7), P = {p_c:.0e}".replace("e-", "e−"),
     (4.4 + 6.7) / 2, 4.4, 6.7, BLUE, "D", (0, (4, 2))),
    (f"If 8.4 and 4.4 are right: symmetric CI\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e−"),
     8.4, 4.4, up_rec, BLUE, "D", (0, (4, 2))),
]
for i, (lab, b, lo, hi, col, mk, ls) in enumerate(items):
    ax.plot([lo, hi], [i, i], color=col, lw=LW, ls=ls, zorder=2)
    for e in (lo, hi):
        ax.plot([e, e], [i - 0.12, i + 0.12], color=col, lw=LW, zorder=2)
    ax.plot([b], [i], mk, ms=MS, color=col, zorder=4)
ax.annotate("", xy=(6.7, -0.32), xytext=(8.4, -0.32),
            arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d outside", ha="center", va="bottom", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(range(len(items)))
ax.set_yticklabels([it[0] for it in items], fontsize=10)
ax.set_ylim(len(items) - 0.5, -0.8)
ax.set_xlim(-0.5, 14)
ax.set_xticks(range(0, 15, 2))
ax.set_xlabel("Longer hospital stay with AKI (stages 1–3), days")
tidy(ax)
fig.legend(handles=[
    Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported"),
    Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS, color=BLUE, label="Recomputed from the paper's numbers")],
    loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout(rect=(0, 0.09, 1, 1))
save(fig, "c2")

# --- c3: where each estimate sits inside its own CI (-1 = lower bound, +1 = upper bound)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
order = [r[0] for r in ROWS]
ax.axvspan(-1, 1, color="#efeeea", zorder=0, lw=0)
ax.text(0, -0.85, "inside the CI", ha="center", va="center", fontsize=10, color=SEC)
for x in (-1, 1):
    ax.axvline(x, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
for i, lab in enumerate(order):
    r = R[lab]
    ax.plot([0, r["pos"]], [i, i], color=BLUE, lw=LW, zorder=2)
    ax.plot([r["pos"]], [i], "o", ms=MS, color=BLUE, zorder=3)
    ax.text(r["pos"] + (0.1 if r["pos"] >= 0 else -0.1), i - 0.3, f"{r['pos'] + 0.0:+.2f}".replace("-0.00", "+0.00").replace("-", "−"),
            ha="left" if r["pos"] >= 0 else "right", va="center", fontsize=10, color=TXT)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n{R[lab]['b']:g} ({R[lab]['lo']:g} to {R[lab]['hi']:g})" for lab in order], fontsize=10)
ax.set_ylim(len(order) - 0.5, -1.2)
ax.set_xlim(-1.6, 3.0)
ax.set_xticks([-1, 0, 1, 2, 3])
ax.set_xticklabels(["−1\nlower bound", "0\ncentre", "+1\nupper bound", "+2", "+3"])
ax.set_xlabel("Where the reported estimate sits in its reported CI (recomputed;\nhazard ratio on the log scale)")
tidy(ax)
fig.tight_layout()
save(fig, "c3")

# --- c4: symmetry check, lower arm vs upper arm of each interval
fig, ax = plt.subplots(figsize=(7.2, 5.0))
lim = (-3, 18)
ax.plot(lim, lim, color=SEC, lw=0.8, zorder=1)
ax.text(17.6, 13.6, "symmetric CI\n(arms equal)", fontsize=10, color=SEC, ha="right", va="top")
ax.axhline(0, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.text(17.8, 0.25, "below 0: estimate above its own upper bound", fontsize=10, color=SEC,
        ha="right", va="bottom")
TPOS = {"Hospital stay, AKI 1-3": (5.2, -2.2), "Hospital stay, AKI 2-3": (9.5, 4.4),
        "PICU stay, AKI 1-3": (9.5, 1.9), "PICU stay, AKI 2-3": (-2.5, 9.6),
        "Mechanical ventilation, AKI 1-3": (6.4, 16.6), "Mechanical ventilation, AKI 2-3": (-2.5, 12.6)}
for lab, b, lo, hi, p, sc in LIN:
    la, ua = b - lo, hi - b
    ax.plot([la], [ua], "o", ms=MS, color=BLUE, zorder=3)
    tx, ty = TPOS[lab]
    ax.annotate(f"{lab}\n({la:.1f}, {ua:.1f})".replace("-", "−").replace("AKI 1−3", "AKI 1-3").replace("AKI 2−3", "AKI 2-3"),
                xy=(la, ua), xytext=(tx, ty), fontsize=10, ha="left", va="center", color=TXT,
                arrowprops=dict(arrowstyle="-", color=SEC, lw=0.7, shrinkA=2, shrinkB=5))
ax.set_xlim(*lim)
ax.set_ylim(*lim)
ax.set_aspect("equal")
ax.set_xticks(range(0, 19, 5))
ax.set_yticks(range(0, 19, 5))
ax.set_xlabel("Estimate − lower bound, days (recomputed from Table 4)")
ax.set_ylabel("Upper bound − estimate, days")
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
ax.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
fig.tight_layout()
save(fig, "c4")

# --- c5: the three printed locations as text, above one number line
fig = plt.figure(figsize=(7.2, 5.0))
quotes = [
    ("Abstract", "“… 8.4 days longer hospitalizations compared to patients without acute kidney injury\n(95% confidence interval, 4.4-6.7).”"),
    ("Results, Clinical course and outcomes", "“… greater length of hospital stay of 8.4 days compared with patients without AKI\n(95% CI, 4.4–6.7; P < 0.0001).”"),
    ("Table 4, Length of hospital stay, AKI stage 1 + 2 + 3", "β (95% CI) “8.4 (4.4 to 6.7)”    P value “<0.0001”"),
]
for i, (where, q) in enumerate(quotes):
    y0 = 0.80 - i * 0.165
    fig.patches.append(matplotlib.patches.FancyBboxPatch(
        (0.02, y0), 0.96, 0.15, boxstyle="round,pad=0.004,rounding_size=0.01", transform=fig.transFigure,
        facecolor="#ffffff", edgecolor="#d6d5d0", lw=1))
    fig.text(0.035, y0 + 0.13, where, fontsize=10, fontweight="bold", va="top", color=TXT)
    fig.text(0.035, y0 + 0.088, q, fontsize=10, va="top", color=TXT, linespacing=1.3)
fig.text(0.5, 0.425, "the same estimate and interval in all three places, drawn on one axis below",
         ha="center", fontsize=10, color=SEC)
ax = fig.add_axes((0.05, 0.13, 0.9, 0.25))
ax.plot([4.4, 6.7], [0, 0], color=ORANGE, lw=LW * 2, solid_capstyle="butt", zorder=2)
for e in (4.4, 6.7):
    ax.plot([e, e], [-0.18, 0.18], color=ORANGE, lw=LW, zorder=2)
ax.plot([8.4], [0], "o", ms=MS + 1, color=ORANGE, zorder=4)
ax.plot([5.55], [0], "o", ms=MS, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(5.55, 0.32, "reported 95% CI 4.4 to 6.7", ha="center", fontsize=10, color=TXT)
ax.text(8.4, 0.32, "reported β = 8.4", ha="center", fontsize=10, color=TXT)
ax.text(5.55, -0.42, "CI centre 5.55\n(recomputed)", ha="center", va="top", fontsize=10, color=TXT)
ax.annotate("", xy=(6.7, -0.25), xytext=(8.4, -0.25),
            arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d", ha="center", va="top", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_xlim(-0.3, 10)
ax.set_ylim(-1, 0.7)
ax.set_yticks([])
ax.set_xticks(range(0, 11, 2))
ax.set_xlabel("Longer hospital stay with AKI, days")
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
save(fig, "c5")

# contact sheet (PNG only, for choosing a design)
import matplotlib.image as mpimg  # noqa: E402
fig, axs = plt.subplots(3, 2, figsize=(14, 16))
for ax, stem in zip(axs.flat, ("c1", "c2", "c3", "c4", "c5")):
    ax.imshow(mpimg.imread(os.path.join(CAND, stem + ".png")))
    ax.set_title(stem, fontsize=16, fontweight="bold", loc="left")
    ax.axis("off")
axs.flat[-1].axis("off")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=100)
plt.close(fig)

for ext in ("svg", "png"):
    shutil.copyfile(os.path.join(CAND, f"{CHOSEN}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"\nwrote candidates c1-c5 and contact_sheet.png in {CAND}; fig1 = {CHOSEN}")

Abstract reports 58 (53%) of 113 resistant sites; Results' own counts give 57 (51%) of 112

Lancet Infect Dis (2014) · doi:10.1016/S1473-3099(14)70855-2 · PMC4178238

Summary

This systematic review reports its headline result in the abstract as chloroquine resistance in "58 (53%) of 113 assessable study sites". The Results instead give 57 (51%) of 112 site estimates, and the body's own sub-counts support that figure: 177 site estimates minus 65 that could not be categorised (37 + 15 + 4 + 4 + 5) leaves 112, and resistance categories 1, 2 and 3 (35 + 8 + 14) add up to 57. The abstract figure is also wrong on its own terms, because 58/113 = 51.3%, which rounds to 51%, not 53%. The qualitative finding, that about half of assessable sites showed resistance, is unchanged. But the abstract's one headline prevalence figure, the number most likely to be quoted, is about 2 percentage points too high, and its count and denominator match nothing in the paper.

What the paper reports

Abstract, Findings:

We identified 129 eligible clinical trials involving 21 694 patients at 179 study sites and 26 case reports describing 54 patients. Chloroquine resistance was present in 58 (53%) of 113 assessable study sites, spread across most countries that are endemic for P vivax.

Results, fifth paragraph:

Estimates of chloroquine efficacy could be derived for 177 sites from 144 treatment groups. In 65 (37%) of these sites chloroquine resistance could not be categorised for the following reasons: early administration of primaquine (37 sites); no documentation of parasitological data at day 28 (15 sites); sample size of fewer than ten patients (four sites); good response documented in asymptomatic patients (four sites); and administration of only a single dose of chloroquine (five sites). Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates, 35 (61%) of these sites fulfilled the predefined criteria for category 1 resistance, eight (14%) for category 2 resistance, and 14 (25%) for category 3 resistance.

Table 1, "Site estimates of chloroquine efficacy" column, "Overall" row:

Overall | 129 | 230 | 177

Discussion, ninth paragraph:

Our definitions were conservative, with data excluded from almost 40% of studies that we thought could be confounded. Almost half of the informative studies suggested evidence of reduced chloroquine efficacy.

What the paper's own numbers show

Method. I checked each printed percentage against its own fraction. I rebuilt the assessable denominator from the Table 1 site total and the five listed reasons for exclusion, and rebuilt the resistant count from the three resistance categories. I checked the Table 1 site total by each of its three breakdowns (location, year, origin of patients). Finally, I searched for fractions that would round to 53%. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Resistant sites, abstract Abstract, Findings 58 (53%) of 113 58/113 = 51.3%, which rounds to 51%, not 53%
Resistant sites, Results Results, para 5 57 (51%) of 112 57/112 = 50.9% (51%); consistent
Chloroquine site estimates Table 1; Results, para 5 177 134 + 13 + 22 + 8 = 177 by location; 8 + 41 + 80 + 48 = 177 by year; 22 + 21 + 134 = 177 by origin
Uncategorisable sites Results, para 5 65 (37%) 37 + 15 + 4 + 4 + 5 = 65; 65/177 = 36.7% (37%)
Assessable sites Results 112; Abstract 113 112 / 113 177 − 65 = 112
Resistant sites = categories 1 + 2 + 3 Results 57; Abstract 58 57 / 58 35 + 8 + 14 = 57
Category shares Results, para 5 61% / 14% / 25% of 57: 61.4% / 14.0% / 24.6% (match); of 58: 60.3% / 13.8% / 24.1% (35/58 would print as 60%, not 61%)
Implied sensitive sites derived — 112 − 57 = 55 (Results); 113 − 58 = 55 (Abstract)
Fractions that round to 53% derived 53% 58/109, 58/110, 60/113 or 59/112; none of these counts appears in the paper
Size of the discrepancy derived — printed 53% is 1.67 points above 58/113 and 2.11 points above 57/112; the two fractions differ by 0.43 points

Figure

Abstract and Results give different resistant-site counts, and the abstract percentage does not match its own fraction

Figure 1. The two passages that report the overall prevalence of chloroquine-resistant P vivax sites. Values printed in the abstract are in orange; values printed in the Results, which agree with the body's sub-counts, and values recomputed from the paper's numbers are in blue. The abstract's count (58) and denominator (113) are each one site higher than the Results' (57 and 112), and its percentage (53%) does not follow from its own fraction (58/113 = 51.3%). The lower panel rebuilds the Results' figures from the Table 1 site total, the five listed reasons for exclusion and the three resistance categories. All numbers are from the paper or from recompute.py.txt.

Why it matters

The abstract's sentence is the paper's single summary figure for how widespread chloroquine-resistant P vivax is, and it is the version most likely to be quoted in later reviews, guideline background documents and resistance summaries. The error is small: the figure the body supports is 51% (57/112), and the abstract's own fraction also gives 51%. Its direction and the qualitative reading ("almost half of the informative studies", in the Discussion's words) are unchanged, and so are the conclusions that resistance is geographically widespread and that monitoring needs to improve. The practical consequences are that readers who quote the abstract carry a percentage about 2 points too high, and that the counts 58 and 113 cannot be reconciled with the paper's own tables and text. The abstract and the body disagree in three places at once (count, denominator and percentage), which will puzzle anyone who tries to check or update the analysis.

Explanations considered

  1. A different unit of analysis. The abstract says "assessable study sites" and the Results say "site estimates". The paper gives only two site totals: 179 study sites for all drugs (Abstract) and 177 site estimates of chloroquine efficacy (Results; Table 1, whose three breakdowns each sum to 177). The only assessable denominator the paper derives is 177 − 65 = 112. No figure in the text, tables or legends gives 113. Applying the 65 exclusions to the 179 study sites would give 114, not 113. This does not account for the numbers.
  2. A different set of resistance categories. Resistance is defined as categories 1, 2 or 3, and these sum to 35 + 8 + 14 = 57. No further resistant subgroup is described that could add a 58th site. Both versions imply 55 sensitive sites (112 − 57 = 55 and 113 − 58 = 55). The abstract therefore has one extra site that is counted as resistant. The printed category shares also fit a base of 57 rather than 58 (35/58 = 60.3% would print as 60%, not 61%). This does not account for the numbers.
  3. Rounding convention. 58/113 = 51.33%. No rounding rule turns that into 53%. A printed 53% would need 58/109, 58/110 or 60/113 (or 59/112), and none of these counts appears in the paper. This does not account for the percentage.
  4. A conversion artefact in the archived text. The same sentences appear in the Lancet's own typeset PDF (the online-first version), in the PMC HTML and in the Europe PMC XML. This is not a conversion artefact.
  5. A data update between versions, such as an earlier count left in the abstract. The body is internally consistent throughout (65 = 37 + 15 + 4 + 4 + 5; 65/177 = 37%; 57/112 = 51%; the category shares of 57). That points to the abstract's figure as the inconsistent one, but the paper gives no information on how it arose, and even such a history would not explain why 53% does not equal 58/113.
  6. The supplement. The single supplementary appendix contains reference lists and a reference table only. It has no site-level categories or counts, so it cannot reconcile 58/113 with 57/112.

Status

Suggested correction

  1. In the abstract's Findings, replace "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" with "Chloroquine resistance was present in 57 (51%) of 112 assessable study sites". This matches the Results, the five listed exclusions from the 177 site estimates in Table 1, and the 35 + 8 + 14 resistance categories.
  2. If the authors' dataset shows that 58 of 113 is the correct count, then the Results paragraph (the 112 denominator, the 65 exclusions and their breakdown, the 57 total and the category counts and shares) needs correcting instead. The abstract percentage would then still need to change from 53% to 51%, because 58/113 = 51.3%.
Recomputation script (python recompute.py.txt)
# Recompute the site counts behind the chloroquine-resistance prevalence in
# PMC4178238 (Lancet Infect Dis 2014; doi:10.1016/S1473-3099(14)70855-2)
# and regenerate fig1.svg / fig1.png in this file's folder.
# Run: python recompute.py.txt
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
pct = lambda a, b: 100.0 * a / b

# ---- Numbers printed in the paper -------------------------------------------
abs_res, abs_den, abs_pct = 58, 113, 53          # Abstract, Findings
res_res, res_den, res_pct = 57, 112, 51          # Results, para 5
sites_all_drugs = 179                            # Abstract: "179 study sites"
cq_sites = 177                                   # Results / Table 1
uncat = {"early primaquine": 37, "no day-28 data": 15, "n < 10": 4,
         "asymptomatic, good response": 4, "single CQ dose": 5}
uncat_reported, uncat_pct_reported = 65, 37
cats = {1: 35, 2: 8, 3: 14}
cats_pct_reported = {1: 61, 2: 14, 3: 25}
table1 = {"location": [134, 13, 22, 8], "year": [8, 41, 80, 48],
          "origin": [22, 21, 134]}

print("== Abstract figure checked against its own fraction")
print("58/113 = %.2f%% -> rounds to %d%% (printed %d%%)"
      % (pct(abs_res, abs_den), round(pct(abs_res, abs_den)), abs_pct))
print("== Results figure checked against its own fraction")
print("57/112 = %.2f%% -> rounds to %d%% (printed %d%%)"
      % (pct(res_res, res_den), round(pct(res_res, res_den)), res_pct))

print("== Table 1 site-estimate totals")
for k, v in table1.items():
    print("  by %-8s: %s = %d" % (k, "+".join(map(str, v)), sum(v)))

print("== Uncategorisable sites (Results para 5)")
u = sum(uncat.values())
print("  37+15+4+4+5 = %d (printed %d); %d/177 = %.1f%% (printed %d%%)"
      % (u, uncat_reported, u, pct(u, cq_sites), uncat_pct_reported))
assess = cq_sites - u
print("  assessable = 177 - %d = %d (Results %d; Abstract %d)"
      % (u, assess, res_den, abs_den))

print("== Resistant sites = categories 1+2+3")
r = sum(cats.values())
print("  35+8+14 = %d (Results %d; Abstract %d)" % (r, res_res, abs_res))
for c, n in cats.items():
    print("  category %d: %d/57 = %.1f%% (printed %d%%)"
          % (c, n, pct(n, r), cats_pct_reported[c]))
print("  same shares with 58 as base: " + ", ".join(
    "%d/58 = %.1f%%" % (n, pct(n, 58)) for n in cats.values()))

print("  179 study sites - 65 = %d (not 113)" % (sites_all_drugs - u))
print("== Implied sensitive (category 4) sites")
print("  Results: 112-57 = %d; Abstract: 113-58 = %d"
      % (res_den - res_res, abs_den - abs_res))

print("== Differences")
print("  printed 53%% minus 58/113: %.2f points" % (abs_pct - pct(58, 113)))
print("  printed 53%% minus 57/112: %.2f points" % (abs_pct - pct(57, 112)))
print("  58/113 minus 57/112: %.2f points" % (pct(58, 113) - pct(57, 112)))

print("== Fractions that would round to 53%")
print("  d with round(58/d) = 53%%: %s"
      % [d for d in range(90, 131) if round(pct(58, d)) == 53])
print("  n with round(n/113) = 53%%: %s"
      % [n for n in range(40, 81) if round(pct(n, 113)) == 53])
print("  n with round(n/112) = 53%%: %s"
      % [n for n in range(40, 81) if round(pct(n, 112)) == 53])
print("  n with round(n/177) = 53%%: %s"
      % [n for n in range(60, 120) if round(pct(n, 177)) == 53])
print("  n with round(n/179) = 53%%: %s"
      % [n for n in range(60, 120) if round(pct(n, 179)) == 53])
print("  58/179 = %.1f%%, 58/177 = %.1f%%, 57/177 = %.1f%%"
      % (pct(58, 179), pct(58, 177), pct(57, 177)))

# ---- Figure 1: abstract vs Results, with the body's arithmetic -------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 72); ax.set_ylim(0, 50)
ax.axis("off"); ax.set_facecolor(BG)

def panel(x, y, w, h, edge, title):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
                                fc="white", ec=edge, lw=1.5))
    ax.text(x + 1, y + h - 1.6, title, fontsize=11, fontweight="bold",
            color=TXT, va="top")

# Left: Abstract
panel(1.5, 21, 28.5, 25, ORANGE, "Abstract, Findings")
ax.text(2.5, 40.5, "“Chloroquine resistance was present\nin 58 (53%) of 113 "
        "assessable\nstudy sites”", fontsize=10, color=TXT, va="top",
        style="italic", linespacing=1.35)
ax.text(2.5, 31.0, "Count  58", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 28.0, "Denominator  113", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 25.0, "Percentage  53%", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 22.0, "but 58/113 = %.1f%% → 51%%" % pct(58, 113),
        fontsize=10, color=BLUE, fontweight="bold")

# Right: Results
panel(42, 21, 28.5, 25, BLUE, "Results, paragraph 5")
ax.text(43, 40.5, "“Overall P vivax was defined as\nchloroquine resistant "
        "for 57 (51%)\nof 112 site estimates”", fontsize=10, color=TXT,
        va="top", style="italic", linespacing=1.35)
ax.text(43, 31.0, "Count  57", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 28.0, "Denominator  112", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 25.0, "Percentage  51%", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 22.0, "57/112 = %.1f%% → 51%%" % pct(57, 112),
        fontsize=10, color=BLUE, fontweight="bold")

# Connector
ax.add_patch(FancyArrowPatch((31.0, 33.5), (41.3, 33.5), arrowstyle="<->",
                             mutation_scale=14, lw=2, color=SEC))
ax.text(36, 35.2, "conflict", ha="center", fontsize=10, color=SEC, fontweight="bold")
ax.text(36, 31.6, "one site\nhigher in\nboth count\nand\ndenominator", ha="center",
        va="top", fontsize=10, color=SEC, linespacing=1.2)

# Bottom: the body's own arithmetic
panel(1.5, 1.5, 69, 15.5, GRID, "What the Results' own sub-counts give (recomputed)")
ax.text(2.5, 11.6, "Site estimates (Table 1)  177", fontsize=10, color=TXT)
ax.text(2.5, 8.6, "− uncategorisable  37+15+4+4+5 = %d" % u, fontsize=10, color=TXT)
ax.text(2.5, 5.6, "= assessable  %d" % assess, fontsize=11, color=BLUE, fontweight="bold")
ax.text(37, 11.6, "Resistant = category 1 + 2 + 3", fontsize=10, color=TXT)
ax.text(37, 8.6, "35 + 8 + 14 = %d" % r, fontsize=10, color=TXT)
ax.text(37, 5.6, "= resistant  %d  →  %d/%d = %.1f%%" % (r, r, assess, pct(r, assess)),
        fontsize=11, color=BLUE, fontweight="bold")
ax.text(2.5, 2.6, "No count or subgroup in the paper gives 113 or 58; 53% needs 58/109, 58/110 or 60/113.",
        fontsize=10, color=SEC)

# Legend (names the colours)
ax.text(1.8, 48.3, "■", color=ORANGE, fontsize=12, va="center")
ax.text(3.6, 48.3, "printed in the abstract", color=TXT, fontsize=10, va="center")
ax.text(26, 48.3, "■", color=BLUE, fontsize=12, va="center")
ax.text(27.8, 48.3, "recomputed from / consistent with the body's counts",
        color=TXT, fontsize=10, va="center")

for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
    fig.savefig(os.path.join(HERE, "fig1." + ext), facecolor=BG, **kw)
print("wrote fig1.svg and fig1.png in", HERE)

Abstract reports lower Ct values in vaccinated participants; the paper's data and text show higher

Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101309 · PMC12703870

Summary

The Findings section of the Abstract states that vaccinated participants had "significantly lower RT-qPCR Ct values" than placebo recipients. The Results report the opposite: mean Ct was higher in the vaccinated group for both serotypes, by 5.0 cycles for DENV-1 (27.2 vs 22.2) and 2.9 cycles for DENV-2 (28.2 vs 25.3), and the Research in context panel, the Discussion, Fig. 2a/2i and Supplementary Table 6 all agree with the Results. Because the paper defines Ct as inversely related to the amount of viral nucleic acid, the Abstract's wording implies more viral RNA in vaccinated participants, which contradicts both the data and the conclusion drawn in the same sentence ("the vaccine could be reducing viral replication"). The error is confined to one word in the Abstract, but it reverses the direction of the paper's main virological finding in the part of the paper most often read on its own.

What the paper reports

Abstract, Findings:

Despite the limited statistical power due to the low number of samples, we observed significantly lower RT-qPCR Ct values in the vaccinated group, particularly for the DENV-1 cases, suggesting that the vaccine could be reducing viral replication.

Research in context, "Added value of this study":

Additionally, we observed higher RT-qPCR Ct values in the vaccinated group compared to the placebo group, indicating a lower viral load among vaccinated participants and suggesting that the Butantan-DV is initially effective in reducing viral loads.

Results, "Characterization of RT-qPCR cycle threshold values":

RT-qPCR cycle threshold (Ct) values indirectly measure DENV genomic load in samples, with an inverse relationship to the input target nucleic acid. Our analysis included a total of 365 DENV-positive samples, with an overall mean Ct of 23.7 (SD: 4.85) for DENV-1 (n = 161) and 26.5 (SD: 4.67) for DENV-2 (n = 205). When comparing RT-qPCR Ct values between the Pb and Vx groups for each serotype, a statistically significant difference was observed (p < 0.001). The Vx group exhibited a higher mean Ct value than the Pb group. Specifically, the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group. This trend was even stronger for DENV-1, with the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) in the Pb group (Fig. 2a and i) (Supplementary Table S6).

Discussion, fourth paragraph:

The higher Ct values observed in the vaccinated group, therefore, suggest that Butantan-DV-induced immunity may confer protection not only by preventing infection, but also by mitigating disease severity through the suppression of viral replication.

Results, "Sample characterization":

There were 234 (64.1%) participants included in the Pb group and 131 (35.8%) participants in the Vx group confirmed for DENV.

Supplementary Table 6 ("Comparative analysis of cycle threshold (Ct) values for DENV-1 and DENV-2 between vaccinated and placebo groups"; columns: N positive Placebo | Butantan-DV | mean Ct Placebo | Butantan-DV | Δ mean | t-statistic | p-value | Cohen's d):

DENV-1 | 113 | 48 | 22.2 | 27.2 | 5.00 | 10.87 | < 0.001 | 1.16

DENV-2 | 122 | 83 | 25.3 | 28.2 | 2.90 | 6.21 | < 0.001 | 0.86

Its footnote reads:

The p-value and t-statistic were calculated using the Welch's t-test for unequal variances.

What the paper's own numbers show

Method. I took the group means and SDs from the Results text and the group sizes from Supplementary Table 6. I computed the vaccinated-minus-placebo difference in mean Ct for each serotype and checked whether rounding of the printed means (to 0.1) could change its sign. To rule out swapped group labels in the body, I checked that the n-weighted combination of the two group means reproduces the overall mean the Results report for each serotype, and compared that with the combination obtained if the labels were swapped. I then recomputed a Welch t-test, its 95% CI and Cohen's d from the summary statistics. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Direction, all serotypes Abstract, Findings "significantly lower RT-qPCR Ct values in the vaccinated group" Opposite to every number below
DENV-1 mean Ct, Vx vs Pb Results; Suppl. Table 6 27.2 (SD 3.98) vs 22.2 (SD 4.40); Δ 5.00 Vx − Pb = +5.0 cycles (+4.9 to +5.1 allowing for rounding): vaccinated higher
DENV-2 mean Ct, Vx vs Pb Results; Suppl. Table 6 28.2 (SD 4.18) vs 25.3 (SD 4.65); Δ 2.90 Vx − Pb = +2.9 cycles (+2.8 to +3.0): vaccinated higher
DENV-1 overall mean and SD Results 23.7 (SD 4.85), n = 161 With Table S6 n (48 Vx, 113 Pb): 23.69 (SD 4.84). Labels swapped: 25.71
DENV-2 overall mean and SD Results 26.5 (SD 4.67), n = 205 With Table S6 n (83 Vx, 122 Pb): 26.47 (SD 4.68). Labels swapped: 27.03
Vaccinated share implied by overall means Results 131 Vx of 365 0.300 × 161 + 0.414 × 205 ≈ 133.1 Vx, close to the 131 in Table S6 and the Results
DENV-1 difference, 95% CI and test Results "p < 0.001"; Table S6 p < 0.001 Welch 95% CI +3.6 to +6.4; t = 7.06 (df 97.4), p = 2.4 × 10⁻¹⁰
DENV-2 difference, 95% CI and test Results "p < 0.001"; Table S6 p < 0.001 Welch 95% CI +1.7 to +4.1; t = 4.66 (df 187.9), p = 6.0 × 10⁻⁶
Meaning of the direction Results Ct has "an inverse relationship to the input target nucleic acid" Higher Ct in Vx = less viral RNA in Vx, as stated in Research in context and the Discussion

The body of the paper is internally consistent: the group means, overall means, group sizes, Fig. 2a and 2i (boxplots in which the vaccinated group sits above placebo for both serotypes) and Supplementary Table 6 all describe higher Ct in vaccinated participants. Neither CI comes near zero, so the reversed direction in the Abstract cannot be a matter of rounding or of a borderline result.

Four smaller inconsistencies turned up during the check. None of them affects the direction or the significance of the difference:

Quantity Location Reported Recomputed
Welch t-statistic Suppl. Table 6 10.87 (DENV-1), 6.21 (DENV-2) 7.06 and 4.66 from the printed means, SDs and n; p stays below 0.001
Cohen's d, DENV-2 Suppl. Table 6 0.86 0.65 with the pooled SD (4.47); DENV-1 reproduces (1.17 vs 1.16)
Group counts Suppl. Table 6 vs Results 113 + 122 = 235 placebo; 161 + 205 = 366 serotype results Results give 234 placebo and 365 samples (difference of one)
Test used for the Ct comparison Methods vs Suppl. Table 6 Methods and Fig. 2 legend: Wilcoxon rank-sum; Table S6: Welch t-test Both tests are reported as p < 0.001

Figure

The Abstract and the Results state opposite directions for the same Ct comparison

Figure 1. The two statements of the vaccinated-vs-placebo Ct comparison, quoted from the Abstract (Findings) and from the Results ("Characterization of RT-qPCR cycle threshold values"). Orange outlines mark text and values as reported in the paper; blue text is the difference recomputed from the Results means. The Abstract states that Ct was lower in the vaccinated group; the Results, which describe the same 365 samples, state and show that it was higher, by 5.0 cycles for DENV-1 and 2.9 cycles for DENV-2. The other places in the paper that give a direction (Research in context, Discussion, Fig. 2a/2i, Supplementary Table 6) agree with the Results.

Vaccinated-minus-placebo difference in mean Ct with recomputed 95% confidence intervals

Figure 2. Difference in mean RT-qPCR Ct, vaccinated minus placebo, on one axis of Ct cycles. Orange diamonds are the differences printed in Supplementary Table 6 (5.00 and 2.90). Blue circles and lines are the differences recomputed from the Results means and SDs with the Table S6 group sizes, with Welch 95% confidence intervals (+3.6 to +6.4 and +1.7 to +4.1). The vertical line marks no difference. The shaded half is the direction the Abstract states (vaccinated Ct lower); both intervals lie entirely in the other half.

Why it matters

The Ct comparison is the only viral-load finding in the Abstract; the other findings there describe the sample, lineages and selection. Many readers, indexing services and secondary summaries rely on the Abstract alone. A reader who takes the Abstract's "significantly lower RT-qPCR Ct values in the vaccinated group" at face value would conclude that vaccinated participants with breakthrough infections carried more viral RNA than unvaccinated ones, which is the opposite of what the paper found and the opposite of the protective interpretation the paper draws.

The sentence also contradicts itself. Under the paper's own definition, lower Ct means more viral nucleic acid, so "lower Ct" cannot support "the vaccine could be reducing viral replication". A reader who knows the Ct scale will notice the conflict; a reader who does not may repeat either half. Press coverage of the study (for example Agência FAPESP) reported reduced viral load in vaccinated participants, which matches the body of the paper, so I found no sign that the reversed wording has been repeated, but the Abstract is the text that remains attached to the article in PubMed, Europe PMC and the publisher's site.

The error does not affect the size, significance or interpretation of the result in the body of the paper, nor its main phylogenetic conclusions (no lineage linked to breakthrough infections, no positive selection, similar intra-host mutation rates), which do not depend on Ct.

Explanations considered

The remaining explanation consistent with all the evidence is that the word "lower" in the Abstract should read "higher".

Status

Suggested correction

  1. In the Abstract (Findings), change "we observed significantly lower RT-qPCR Ct values in the vaccinated group" to "we observed significantly higher RT-qPCR Ct values (indicating lower viral load) in the vaccinated group". No other text, table or figure needs to change for this point.
  2. Minor, for Supplementary Table 6: check the Welch t-statistics (7.06 and 4.66 follow from the printed means, SDs and n, against 10.87 and 6.21 printed) and the DENV-2 Cohen's d (0.65 vs 0.86); reconcile the placebo count (113 + 122 = 235) with the 234 placebo participants and 365 samples in the Results; and state which test (Wilcoxon rank-sum, per the Methods, or Welch t, per the table) gives the reported p < 0.001.
Recomputation script (python recompute.py.txt)
"""Recompute the vaccinated-vs-placebo RT-qPCR Ct comparison of PMC12703870
(Lancet Reg Health Am, doi:10.1016/j.lana.2025.101309).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
  fig1.svg/png, fig2.svg/png           (the figures used in writeup.md)
  candidates/cand_A..E.svg/png         (five alternative figure designs for review)

Inputs are copied from the paper as archived in PMC and from its only supplement
(mmc1.docx, Supplementary Table 6):
  Results, "Characterization of RT-qPCR cycle threshold values": group means/SDs,
      overall means/SDs and serotype n
  Results, "Sample characterization": 234 Pb and 131 Vx participants (365 in all)
  Supplementary Table 6: group n, mean difference, Welch t, Cohen's d
"""
import math
import os
import textwrap

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Patch  # noqa: E402
from matplotlib.lines import Line2D  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ------------------------------------------------------------------ inputs (from the paper)
# Results text: mean (SD)
REP = {
    "DENV-1": dict(n=161, overall=(23.7, 4.85), vx=(27.2, 3.98), pb=(22.2, 4.40)),
    "DENV-2": dict(n=205, overall=(26.5, 4.67), vx=(28.2, 4.18), pb=(25.3, 4.65)),
}
# Supplementary Table 6: N positive (Pb, Vx), delta mean, t (Welch), Cohen's d
S6 = {
    "DENV-1": dict(n_pb=113, n_vx=48, delta=5.00, t=10.87, d=1.16),
    "DENV-2": dict(n_pb=122, n_vx=83, delta=2.90, t=6.21, d=0.86),
}
N_PB_TOTAL, N_VX_TOTAL, N_TOTAL = 234, 131, 365   # Results, "Sample characterization"

# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5,
    "text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
    fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


# ------------------------------------------------------------------ 1. direction and size
print("1. Direction of the group difference (Results text)")
R = {}
for s, r in REP.items():
    mv, sv = r["vx"]
    mp, sp = r["pb"]
    diff = mv - mp
    # rounding to one decimal moves each mean by at most 0.05
    lo, hi = diff - 0.1, diff + 0.1
    R[s] = dict(diff=diff)
    print(f"   {s}: Vx {mv} - Pb {mp} = {diff:+.1f} Ct cycles "
          f"(range allowed by rounding {lo:+.1f} to {hi:+.1f}) -> vaccinated "
          f"{'HIGHER' if lo > 0 else 'LOWER/unclear'}")

# ------------------------------------------------------------------ 2. group sizes and consistency
print("\n2. Group sizes: implied by the overall mean vs Supplementary Table 6")
implied_vx_total = 0.0
for s, r in REP.items():
    mo, so = r["overall"]
    mv, sv = r["vx"]
    mp, sp = r["pb"]
    w = (mo - mp) / (mv - mp)
    implied_vx_total += w * r["n"]
    nv, npb = S6[s]["n_vx"], S6[s]["n_pb"]
    n = nv + npb
    m = (nv * mv + npb * mp) / n
    ss = (nv - 1) * sv ** 2 + (npb - 1) * sp ** 2 + nv * (mv - m) ** 2 + npb * (mp - m) ** 2
    sd = math.sqrt(ss / (n - 1))
    R[s].update(n_vx=nv, n_pb=npb, pooled_mean=m, pooled_sd=sd)
    print(f"   {s}: overall mean {mo} implies Vx share {w:.3f} -> n_Vx ~ {w * r['n']:.1f}, "
          f"n_Pb ~ {r['n'] - w * r['n']:.1f}")
    print(f"      Table S6 n: Vx {nv}, Pb {npb} (sum {n}; Results n = {r['n']}); "
          f"n-weighted mean {m:.2f} (reported {mo}), pooled SD {sd:.2f} (reported {so})")
    print(f"      if the group labels were swapped: n-weighted mean "
          f"{(nv * mp + npb * mv) / n:.2f} (reported {mo})")
vx6 = sum(S6[s]["n_vx"] for s in S6)
pb6 = sum(S6[s]["n_pb"] for s in S6)
print(f"   Vx implied by overall means ~ {implied_vx_total:.1f}; Table S6 Vx total {vx6} "
      f"(Results {N_VX_TOTAL}); Table S6 Pb total {pb6} (Results {N_PB_TOTAL}); "
      f"serotype n {REP['DENV-1']['n']} + {REP['DENV-2']['n']} = "
      f"{REP['DENV-1']['n'] + REP['DENV-2']['n']} (Results {N_TOTAL} samples)")

# ------------------------------------------------------------------ 3. tests from summary stats
print("\n3. Welch t-test from the reported means, SDs and Table S6 n (two-sided)")
for s, r in REP.items():
    mv, sv = r["vx"]
    mp, sp = r["pb"]
    nv, npb = R[s]["n_vx"], R[s]["n_pb"]
    t, p = stats.ttest_ind_from_stats(mv, sv, nv, mp, sp, npb, equal_var=False)
    se = math.sqrt(sv ** 2 / nv + sp ** 2 / npb)
    df = se ** 4 / ((sv ** 2 / nv) ** 2 / (nv - 1) + (sp ** 2 / npb) ** 2 / (npb - 1))
    q = stats.t.ppf(0.975, df)
    ci = (R[s]["diff"] - q * se, R[s]["diff"] + q * se)
    sp_pool = math.sqrt(((nv - 1) * sv ** 2 + (npb - 1) * sp ** 2) / (nv + npb - 2))
    d = R[s]["diff"] / sp_pool
    R[s].update(t=t, p=p, df=df, ci=ci, d=d)
    print(f"   {s}: diff {R[s]['diff']:+.2f}, SE {se:.3f}, Welch df {df:.1f}, "
          f"95% CI {ci[0]:+.2f} to {ci[1]:+.2f}; t = {t:.2f}, p = {p:.1e}")
    print(f"      Table S6: delta {S6[s]['delta']:.2f}, t {S6[s]['t']:.2f}, d {S6[s]['d']:.2f}; "
          f"recomputed Cohen's d (pooled SD {sp_pool:.2f}) = {d:.2f}")
    print(f"      t needed to reproduce Table S6 t with this difference: SE would be "
          f"{R[s]['diff'] / S6[s]['t']:.3f} (vs {se:.3f})")

print("\n4. What the direction means on the paper's own scale")
print("   Results: Ct values have 'an inverse relationship to the input target nucleic acid'.")
print("   Higher Ct in Vx -> less viral RNA in Vx -> consistent with 'reducing viral replication'.")
print("   'Lower Ct in Vx' (Abstract) would mean more viral RNA in Vx, contradicting the same")
print("   sentence's conclusion, the Results numbers, Research in context and the Discussion.")

# ------------------------------------------------------------------ shared text (verbatim)
Q_ABS = ("we observed significantly lower RT-qPCR Ct values in the vaccinated group, "
         "particularly for the DENV-1 cases, suggesting that the vaccine could be "
         "reducing viral replication")
Q_ADD = ("we observed higher RT-qPCR Ct values in the vaccinated group compared to the "
         "placebo group, indicating a lower viral load among vaccinated participants")
Q_RES = "The Vx group exhibited a higher mean Ct value than the Pb group."
Q_RES1 = ("the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) "
          "in the Pb group")
Q_RES2 = "the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group"
Q_DIS = "The higher Ct values observed in the vaccinated group"
Q_INV = "with an inverse relationship to the input target nucleic acid"
d1, d2 = R["DENV-1"]["diff"], R["DENV-2"]["diff"]
RECOMP = f"Vx − Pb mean Ct: {d1:+.1f} cycles (DENV-1), {d2:+.1f} cycles (DENV-2)"


def wrap(s, n):
    return "\n".join(textwrap.wrap(s, n, break_on_hyphens=False))


def panel(ax, x, y, w, h, edge, lw=1.6, face="white"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                lw=lw, ec=edge, fc=face, transform=ax.transAxes))


def legend_text_kinds(ax, loc="lower center", y=-0.02):
    h = [Patch(fc="white", ec=ORANGE, lw=2, label="Text or values as reported in the paper"),
         Patch(fc="white", ec=BLUE, lw=2, label="Recomputed from the paper's own numbers")]
    ax.legend(handles=h, loc=loc, bbox_to_anchor=(0.5, y), ncol=2, frameon=False, fontsize=10)


# ------------------------------------------------------------------ design A: two text panels
def design_a(path):
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.axis("off")
    # left: Abstract
    panel(ax, 0.02, 0.25, 0.42, 0.67, ORANGE)
    ax.text(0.04, 0.88, "Abstract, Findings", fontsize=11.5, weight="bold", va="top")
    ax.text(0.04, 0.80, "States: vaccinated Ct LOWER", fontsize=10.5, color=INK2, va="top")
    ax.text(0.04, 0.72, "\u201c\u2026" + wrap(Q_ABS, 38) + "\u201d", fontsize=10, va="top",
            linespacing=1.35)
    ax.text(0.04, 0.30, "(The Abstract gives no Ct values.)", fontsize=10, color=INK2)
    # right: Results
    panel(ax, 0.56, 0.25, 0.42, 0.67, ORANGE)
    ax.text(0.58, 0.88, "Results, Ct paragraph", fontsize=11.5, weight="bold", va="top")
    ax.text(0.58, 0.80, "States: vaccinated Ct HIGHER", fontsize=10.5, color=INK2, va="top")
    ax.text(0.58, 0.72, "\u201c" + wrap(Q_RES, 38) + "\u201d", fontsize=10, va="top",
            linespacing=1.35)
    ax.text(0.58, 0.60, "Mean Ct, Vx vs Pb:\nDENV-1  27.2 vs 22.2\nDENV-2  28.2 vs 25.3\n"
            "p < 0.001 for each serotype", fontsize=10, va="top", linespacing=1.45)
    ax.text(0.58, 0.395, f"Recomputed Vx \u2212 Pb:\n{d1:+.1f} cycles (DENV-1)\n"
            f"{d2:+.1f} cycles (DENV-2)", fontsize=10, color=BLUE, weight="bold", va="top",
            linespacing=1.35)
    # connector
    ax.annotate("", xy=(0.555, 0.61), xytext=(0.445, 0.61),
                arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
    ax.text(0.50, 0.66, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
            weight="bold")
    ax.text(0.50, 0.56, "same 365\nsamples", ha="center", va="top", fontsize=10, color=INK2)
    ax.text(0.02, 0.205, "Research in context (Added value), Discussion, Fig. 2a/2i and "
            "Supplementary Table 6 also give the higher-Ct\ndirection. Results: Ct has "
            "\u201can inverse relationship to the input target nucleic acid\u201d, so higher "
            "Ct = less viral RNA.", fontsize=10, color=INK2, va="top", linespacing=1.4)
    legend_text_kinds(ax, y=-0.01)
    save(fig, path)


# ------------------------------------------------------------------ design B: four-location tally
def design_b(path):
    rows = [
        ("Abstract, Findings", "“…significantly lower RT-qPCR Ct values in the vaccinated group…”", "LOWER", ORANGE),
        ("Research in context,\nAdded value", "“…higher RT-qPCR Ct values in the vaccinated group compared to the placebo group…”", "HIGHER", ORANGE),
        ("Results, Ct paragraph", "“" + Q_RES + "”", "HIGHER", ORANGE),
        ("Discussion, para. 4", "“" + Q_DIS + "…”", "HIGHER", ORANGE),
        ("Paper's means,\nrecomputed", RECOMP, "HIGHER", BLUE),
    ]
    fig = plt.figure(figsize=(W, 4.4))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.axis("off")
    ax.text(0.02, 0.965, "Where the paper states the direction", fontsize=10.5, color=INK2, va="top")
    ax.text(0.97, 0.965, "Vaccinated Ct vs placebo", fontsize=10.5, color=INK2, va="top", ha="right")
    top, hgt = 0.89, 0.148
    for i, (loc, txt, dirn, col) in enumerate(rows):
        y = top - i * hgt
        is_abs = i == 0
        panel(ax, 0.02, y - hgt + 0.02, 0.96, hgt - 0.025, col, lw=2.6 if is_abs else 1.4)
        ax.text(0.04, y - hgt / 2 + 0.007, loc, fontsize=10.5, weight="bold", va="center",
                linespacing=1.2)
        ax.text(0.305, y - hgt / 2 + 0.007, wrap(txt, 46), fontsize=10, va="center",
                color=BLUE if col == BLUE else INK, linespacing=1.3)
        arrow = "▼" if dirn == "LOWER" else "▲"
        ax.text(0.955, y - hgt / 2 + 0.007, f"{arrow} {dirn}", fontsize=11, weight="bold",
                ha="right", va="center")
    ax.text(0.02, 0.135, "Only the Abstract gives “lower”; every other location, "
                         "and the numbers, give “higher” (higher Ct = less viral RNA).",
            fontsize=10, color=INK2, va="top")
    legend_text_kinds(ax, y=-0.005)
    save(fig, path)


# ------------------------------------------------------------------ design C: means +/- SD on the Ct scale
def design_c(path):
    fig, ax = plt.subplots(figsize=(W, 4.2))
    xs = {"DENV-1": 0, "DENV-2": 1}
    off = 0.16
    for s, r in REP.items():
        x0 = xs[s]
        for k, dx, mk in (("pb", -off, "o"), ("vx", off, "s")):
            m, sd = r[k]
            ax.errorbar(x0 + dx, m, yerr=sd, fmt=mk, ms=9, color=ORANGE, mfc=ORANGE,
                        mec=BG, mew=1.5, elinewidth=2, capsize=0, zorder=3)
            ax.text(x0 + dx + 0.05, m, f"{'Pb' if k == 'pb' else 'Vx'} {m}", va="center",
                    fontsize=10, color=INK)
        top = r["vx"][0] + r["vx"][1] + 1.2
        ax.plot([x0 - off, x0 - off, x0 + off, x0 + off],
                [r["pb"][0] + r["pb"][1] + 0.6, top, top, r["vx"][0] + r["vx"][1] + 0.6],
                color=BLUE, lw=2)
        ax.text(x0, top + 0.5, f"Vx − Pb = {R[s]['diff']:+.1f}", ha="center",
                va="bottom", color=BLUE, weight="bold", fontsize=10.5)
    ax.set_xticks([0, 1], ["DENV-1 (n = 161)", "DENV-2 (n = 205)"], fontsize=10.5, color=INK)
    ax.set_xlim(-0.6, 1.6)
    ax.set_ylim(0, 40)
    ax.set_ylabel("RT-qPCR Ct (mean ± SD)\nhigher Ct = less viral RNA")
    ax.yaxis.grid(True, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for sp in ("top", "right"):
        ax.spines[sp].set_visible(False)
    ax.text(-0.55, 4.2, "Abstract: “significantly lower RT-qPCR Ct values in the "
                        "vaccinated group”\n— the reported means show the opposite",
            fontsize=10, color=INK, va="center")
    h = [Line2D([], [], marker="o", ls="none", color=ORANGE, ms=9, label="Placebo (Pb), reported"),
         Line2D([], [], marker="s", ls="none", color=ORANGE, ms=9, label="Vaccinated (Vx), reported"),
         Line2D([], [], color=BLUE, lw=2, label="Difference recomputed")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.1), ncol=3,
              frameon=False, fontsize=10)
    save(fig, path)


# ------------------------------------------------------------------ design D: difference with CI, null at 0
def design_d(path):
    fig, ax = plt.subplots(figsize=(W, 3.3))
    ys = {"DENV-1": 1, "DENV-2": 0}
    ax.axvspan(-7, 0, color="#f1f0ec", zorder=0)
    ax.axvline(0, color=INK2, lw=1, zorder=1)
    ax.text(-3.5, 1.62, "Direction stated in the Abstract:\nvaccinated Ct lower",
            ha="center", va="top", fontsize=10, color=INK2)
    ax.text(3.5, 1.62, "Direction in Results, Added value,\nDiscussion: vaccinated Ct higher",
            ha="center", va="top", fontsize=10, color=INK2)
    for s in REP:
        y = ys[s]
        lo, hi = R[s]["ci"]
        ax.plot([lo, hi], [y - 0.1] * 2, color=BLUE, lw=2, zorder=2)
        ax.plot(R[s]["diff"], y - 0.1, "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
        ax.text(hi + 0.15, y - 0.1, f"{R[s]['diff']:+.1f} ({lo:+.1f} to {hi:+.1f})",
                va="center", fontsize=10, color=INK)
        ax.plot(S6[s]["delta"], y + 0.12, "D", color=ORANGE, ms=8, mec=BG, mew=1.5, zorder=3)
        ax.text(S6[s]["delta"] + 0.25, y + 0.12, f"{S6[s]['delta']:+.2f}", va="center",
                fontsize=10, color=INK)
    ax.set_yticks([1, 0], ["DENV-1\n(Vx 48, Pb 113)", "DENV-2\n(Vx 83, Pb 122)"],
                  fontsize=10.5, color=INK)
    ax.set_xlim(-7, 7)
    ax.set_ylim(-0.5, 1.7)
    ax.set_xlabel("Vaccinated − placebo mean RT-qPCR Ct (cycles)")
    ax.xaxis.grid(True, color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for sp in ("top", "right", "left"):
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h = [Line2D([], [], marker="D", ls="none", color=ORANGE, ms=8,
                label="Reported difference (Suppl. Table 6)"),
         Line2D([], [], marker="o", color=BLUE, lw=2, ms=9,
                label="Recomputed: Results means, Welch 95% CI")]
    ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2,
              frameon=False, fontsize=10)
    save(fig, path)


# ------------------------------------------------------------------ design E: logic chain
def design_e(path):
    fig = plt.figure(figsize=(W, 3.8))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.axis("off")
    xs = [0.02, 0.355, 0.69]
    bw = 0.29

    def row(y, label, boxes, cols, links):
        ax.text(0.02, y + 0.255, label, fontsize=11, weight="bold", va="bottom")
        for x, txt, c in zip(xs, boxes, cols):
            panel(ax, x, y, bw, 0.24, c)
            ax.text(x + 0.012, y + 0.12, txt, fontsize=10, va="center", linespacing=1.3,
                    color=BLUE if c == BLUE else INK)
        for i, (sym, ok) in enumerate(links):
            xa, xb = xs[i] + bw, xs[i + 1]
            ax.annotate("", xy=(xb - 0.004, y + 0.12), xytext=(xa + 0.004, y + 0.12),
                        arrowprops=dict(arrowstyle="->", color=INK, lw=2))
            ax.text((xa + xb) / 2, y + 0.155, sym, ha="center", va="bottom", fontsize=12,
                    weight="bold")

    row(0.60, "Abstract, Findings",
        ["Premise:\n“significantly lower\nRT-qPCR Ct values in\nthe vaccinated group”",
         "Paper's definition\n(Results): Ct inverse to\nnucleic acid, so lower\nCt = more viral RNA",
         "Conclusion:\n“the vaccine could be\nreducing viral\nreplication”"],
        [ORANGE, ORANGE, ORANGE], [("", True), ("≠", False)])
    ax.text(0.6675, 0.585, "does not follow", ha="center", va="top", fontsize=10,
            color=INK2)
    row(0.16, "Results, Added value, Discussion; recomputed from the Results means",
        ["Recomputed Vx \u2212 Pb\nmean Ct: " + f"{d1:+.1f} (DENV-1),\n{d2:+.1f} (DENV-2), so\nvaccinated Ct higher",
         "Same definition:\nhigher Ct =\nless viral RNA",
         "Same conclusion:\nsuppression of viral\nreplication"],
        [BLUE, ORANGE, ORANGE], [("", True), ("=", True)])
    ax.text(0.6675, 0.145, "follows", ha="center", va="top", fontsize=10, color=INK2)
    legend_text_kinds(ax, y=-0.05)
    save(fig, path)


design_a(os.path.join(CAND, "cand_A_text_panels"))
design_b(os.path.join(CAND, "cand_B_location_tally"))
design_c(os.path.join(CAND, "cand_C_means_sd"))
design_d(os.path.join(CAND, "cand_D_difference_ci"))
design_e(os.path.join(CAND, "cand_E_logic_chain"))
# provisional choice for writeup.md, pending review of the candidates
design_a(os.path.join(HERE, "fig1"))
design_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)

Abstract reverses the direction of the cerebellar IDH3A protein difference reported in Table 3B

Mol Psychiatry (2016) · doi:10.1038/mp.2015.217 · PMC5078854

Summary

The abstract states that protein expression of IDH3A in the cerebellum of patients with bipolar disorder (BD) "was higher than that of controls". The paper's own Table 3B gives the opposite: 0.804 ± 0.390 in BD against 1.227 ± 0.543 in controls (n = 15 per group). That is 0.423 units, or 34.5%, lower in BD, and the Results and Discussion both describe it as lower. Recomputing from the table's summary statistics reproduces its reported tests to rounding (ANOVA F(3, 56) = 5.73 against the printed 5.74, P = 0.0017; Dunnett P = 0.0172 for BD vs controls, printed 0.017), so the table and Results are internally consistent and only the abstract's direction word is wrong. The abstract is the most widely read and indexed part of the paper, and as printed it makes the protein result appear to contradict the lower IDH3A and IDH3B mRNA reported in the preceding sentence of the same abstract.

What the paper reports

Abstract:

Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.

Abstract, preceding sentence (the mRNA result the protein finding is presented alongside):

Furthermore, gene expression of two subtypes (IDH3A and IDH3B) of isocitrate dehydrogenase (IDH) in the dorsolateral prefrontal cortex from BD patients was significantly lower than that of controls

Results, "Protein expression of IDH3A and IDH3B in the cerebellum and parietal cortex":

Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), major depressive disorder (P=0.006) and schizophrenia (P=0.001) were significantly lower than in controls (Table 3B).

Table 3B, cerebellum row (columns: Controls n=15 | BD n=15 | MDD n=15 | Schizophrenia n=15 | One-way ANOVA):

IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002

Table 3B footer:

The data are the mean±s.d.*P<0.05, **P<0.01 (vs control, Dunnett's test).

Discussion, first paragraph:

Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients were lower than those in controls.

Discussion, second paragraph:

We did find reduced expression of IDH3A protein in the cerebellum in BD and other psychiatric disorders such as major depressive disorder and schizophrenia.

Taken all together, it is likely that decreased activity of IDH3A in the brain may play a crucial role in the pathogenesis of BD

The conclusions that draw on the finding. Abstract:

These findings suggest that abnormality in the metabolism of isocitrate by IDH3A in the mitochondria plays a key role in the pathogenesis of BD, supporting the mitochondrial dysfunction hypothesis of BD. Therefore, IDH3 in the citric acid cycle could potentially be a novel therapeutic target for BD.

Conclusions:

Our study provides evidence for abnormality in the metabolism of isocitrate by IDH3A in the pathogenesis of BD.

What the paper's own numbers show

Method. I took the group means and standard deviations printed in Table 3B (n = 15 per group, four groups) and Table 3A (dorsolateral prefrontal cortex, DLPFC, mRNA; controls n = 34, BD n = 35). For each Table 3B row I rebuilt the one-way ANOVA from the summary statistics (between-group and pooled within-group sums of squares, df 3 and 56). For the cerebellar IDH3A row I computed each patient group's difference from controls, the pooled-variance t statistic on 56 df, and the exact two-sided Dunnett P for three comparisons against a common control with equal n (correlation 0.5), by numerical integration rather than simulation. For all seven rows of Table 3A I ran Student and Welch two-sample t tests from the summary statistics, and found the range of P values each test can give when every printed mean and SD is allowed to be off by up to 0.0005 (their 3-decimal rounding). I also applied the Benjamini–Hochberg adjustment over the seven genes to the printed P values, to check the table's false discovery rate (FDR) column. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Direction of cerebellar IDH3A protein, BD vs controls Abstract "higher than that of controls" lower: BD − controls = 0.804 − 1.227 = −0.423 (−34.5%; ratio 0.655)
Direction of cerebellar IDH3A protein, BD vs controls Results; Discussion ¶1 and ¶2 "significantly lower"; "lower"; "reduced" lower (agrees with Table 3B)
Cerebellar IDH3A protein, controls vs BD Table 3B 1.227 ± 0.543 vs 0.804 ± 0.390* as printed; pooled t(56) = −2.84
One-way ANOVA, cerebellar IDH3A Table 3B F(3, 56) = 5.74, P = 0.002 F(3, 56) = 5.73, P = 0.0017 (rounding of the printed means and SDs allows F = 5.70 to 5.77)
Dunnett, BD vs controls, cerebellar IDH3A Results; Table 3B (*) P = 0.017 t = −2.840, P = 0.0172
Dunnett, MDD vs controls, cerebellar IDH3A Results; Table 3B (**) P = 0.006 t = −3.229, P = 0.0058
Dunnett, schizophrenia vs controls, cerebellar IDH3A Results; Table 3B (**) P = 0.001 t = −3.807, P = 0.0010
One-way ANOVA, cerebellar IDH3B Table 3B F(3, 56) = 2.30, P = 0.087 F = 2.30, P = 0.0869; BD − controls = −0.181
One-way ANOVA, parietal IDH3A Table 3B F(3, 56) = 2.58, P = 0.063 F = 2.58, P = 0.0628; BD − controls = −0.279
One-way ANOVA, parietal IDH3B Table 3B F(3, 56) = 0.63, P = 0.597 F = 0.63, P = 0.5959; BD − controls = −0.117
DLPFC IDH3A mRNA, controls vs BD Table 3A; Abstract ("lower") 1.466 ± 0.832 vs 1.017 ± 0.416, P = 0.007 BD − controls = −0.449 (−30.6%); Student P = 0.0058, Welch P = 0.0069
DLPFC IDH3B mRNA, controls vs BD Table 3A; Abstract ("lower") 1.198 ± 0.420 vs 0.957 ± 0.287, P = 0.007 BD − controls = −0.241 (−20.1%); Student P = 0.0069, Welch P = 0.0074
DLPFC mRNA, other Table 3A genes (unpaired t-test) Table 3A ACO1 0.178; ACO2 0.487; IDH1 0.755; IDH2 0.279; IDH3G 0.422 Student P: ACO1 0.175; ACO2 0.478; IDH1 0.758; IDH2 0.277; IDH3G 0.419. Each printed P lies inside the range that 3-decimal rounding of the means and SDs allows (for example ACO2 0.459 to 0.497)
FDR column, Table 3A Table 3A 0.414; 0.568; 0.755; 0.488; 0.026; 0.026; 0.568 (ACO1 … IDH3G) Benjamini–Hochberg of the printed P values: 0.415; 0.568; 0.755; 0.488; 0.025; 0.025; 0.568 (printed P ± 0.0005 gives 0.023 to 0.026 for IDH3A and IDH3B)
Cerebellar IDH3A protein, BD vs the other patient groups Table 3B — BD − MDD = +0.058; BD − schizophrenia = +0.144 (BD is higher only relative to these groups, not to controls)

All four one-way ANOVAs and the three Dunnett P values in Table 3B are reproduced to rounding from the printed summary statistics. In Table 3A, the printed t-test P values for IDH3A and IDH3B match the Welch test (0.0069 and 0.0074); the Student test gives 0.0058 for IDH3A, which does not round to the printed 0.007 even allowing for rounding of the inputs. For the other five genes the recomputed P values differ from the printed ones by up to 0.009 (ACO2: 0.478 against 0.487), but each printed value lies within the range that the 3-decimal rounding of the printed means and SDs allows, so these differences say nothing either way. The FDR column matches a Benjamini–Hochberg adjustment of the printed P values to rounding. None of this bears on the error: the only statement that disagrees with the numbers is the direction word in the abstract.

Figure

The abstract and the Results/Table 3B describe the same cerebellar IDH3A protein comparison in opposite directions

Figure 1. The two places in the paper that describe the same quantity: cerebellar IDH3A protein, BD vs controls, Stanley post-mortem samples, n = 15 per group. Left: the abstract sentence, with its direction word in orange (reported). Right: the Results sentence (black) and the Table 3B means (orange, reported), with the difference recomputed from those means in blue (BD − controls = −0.423, 34.5% lower). The connector marks the conflict. The Discussion, not shown in a panel, agrees with the Results. All text is quoted verbatim; all numbers come from Table 3B or recompute.py.txt.

BD/control ratios of means for IDH3A and IDH3B, recomputed from Tables 3A and 3B: all point estimates below 1

Figure 2. BD/control ratio of means for every IDH3A and IDH3B measurement in Tables 3A and 3B, on a log scale with 1 (no difference) as the reference line. Points and intervals (blue) are recomputed from the printed means and standard deviations; intervals are approximate 95% confidence intervals for the ratio by the delta method on the log scale and are shown only to convey precision, not as the paper's own analysis. The right-hand labels give the direction the abstract states for the three measurements it mentions; only the cerebellar IDH3A protein label (orange) conflicts with its point, which lies at 0.655 (95% CI 0.470 to 0.914). The figure shows that, read correctly, the protein result points the same way as the two mRNA results the abstract cites beside it.

Why it matters

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In the abstract, replace

Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.

with

Moreover, protein expression of IDH3A in the cerebellum from BD patients was lower than that of controls.

No change is needed to Table 3B, the Results, the Discussion or the Conclusions, which already report the direction correctly. Optionally, the corrected sentence could add the effect and test (0.804 ± 0.390 vs 1.227 ± 0.543; Dunnett P = 0.017) and note that the reduction was also seen in major depressive disorder and schizophrenia, as the Results state.

Recomputation script (python recompute.py.txt)
# Recomputation for PMC5078854 (Mol Psychiatry 2016, doi:10.1038/mp.2015.217)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG at 200 dpi) in this file's own folder. Inputs are the summary
# statistics printed in Table 3A and Table 3B of the paper; nothing else is
# assumed.
#   python recompute.py.txt --candidates DIR
# additionally renders the five candidate figure designs (A to E) into DIR, for
# choosing a design; they are not part of the write-up.

import os
import sys
import itertools
import numpy as np
from scipy import stats, integrate
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = sys.argv[sys.argv.index("--candidates") + 1] if "--candidates" in sys.argv else None

# ----------------------------------------------------------------------------
# 1. Inputs from the paper
# ----------------------------------------------------------------------------
GROUPS = ["Controls", "BD", "MDD", "Schizophrenia"]
N_PER_GROUP = 15
TABLE3B = {  # (mean, s.d.) per group, n=15 each; reported one-way ANOVA (F, P)
    "Cerebellum IDH3A": ([(1.227, 0.543), (0.804, 0.390), (0.746, 0.356), (0.660, 0.303)], (5.74, 0.002)),
    "Cerebellum IDH3B": ([(1.020, 0.883), (0.839, 0.373), (0.605, 0.243), (0.610, 0.255)], (2.30, 0.087)),
    "Parietal IDH3A":   ([(0.951, 0.527), (0.672, 0.310), (0.648, 0.435), (0.564, 0.307)], (2.58, 0.063)),
    "Parietal IDH3B":   ([(0.738, 0.311), (0.621, 0.330), (0.760, 0.560), (0.602, 0.300)], (0.63, 0.597)),
}
DUNNETT_REPORTED = {"BD": 0.017, "MDD": 0.006, "Schizophrenia": 0.001}  # Results text
TABLE3A = {  # DLPFC mRNA: control mean, sd, n, BD mean, sd, n, reported unpaired t-test P
    "DLPFC IDH3A mRNA": (1.466, 0.832, 34, 1.017, 0.416, 35, 0.007),
    "DLPFC IDH3B mRNA": (1.198, 0.420, 34, 0.957, 0.287, 35, 0.007),
}
# All seven rows of Table 3A: (control mean, sd, BD mean, sd, reported t-test P, reported FDR P)
TABLE3A_ALL = {
    "ACO1":  (1.083, 0.205, 1.171, 0.315, 0.178, 0.414),
    "ACO2":  (0.922, 0.149, 0.897, 0.142, 0.487, 0.568),
    "IDH1":  (1.208, 0.337, 1.233, 0.333, 0.755, 0.755),
    "IDH2":  (1.316, 0.544, 1.186, 0.437, 0.279, 0.488),
    "IDH3A": (1.466, 0.832, 1.017, 0.416, 0.007, 0.026),
    "IDH3B": (1.198, 0.420, 0.957, 0.287, 0.007, 0.026),
    "IDH3G": (1.084, 0.174, 1.049, 0.183, 0.422, 0.568),
}


def dunnett_two_sided_p(t, k, df):
    """Exact two-sided Dunnett P for k treatment-vs-control contrasts with equal n
    (correlation 0.5). With U0, Ui iid N(0,1) and S^2 ~ chi2_df/df, the contrast
    statistics are (Ui-U0)/(sqrt2*S); conditioning on U0=u and S=s makes them
    independent. P(max|T| < c) = E[ prod_i (Phi(u + sqrt2*c*s) - Phi(u - sqrt2*c*s)) ]."""
    c = abs(t)

    def inner(s):
        f = lambda u: stats.norm.pdf(u) * (stats.norm.cdf(u + np.sqrt(2) * c * s)
                                           - stats.norm.cdf(u - np.sqrt(2) * c * s)) ** k
        return integrate.quad(f, -10, 10, epsabs=1e-12)[0]

    # density of S = sqrt(chi2_df / df)
    dens = lambda s: stats.chi.pdf(s * np.sqrt(df), df) * np.sqrt(df)
    lo, hi = stats.chi.ppf([1e-12, 1 - 1e-12], df) / np.sqrt(df)
    cover = integrate.quad(lambda s: dens(s) * inner(s), lo, hi, epsabs=1e-12, limit=200)[0]
    return 1 - cover


print("=" * 78)
print("Table 3B: one-way ANOVA from summary statistics (n=15 per group, df 3, 56)")
print("=" * 78)
RES = {}
for name, (rows, (F_rep, P_rep)) in TABLE3B.items():
    m = np.array([a for a, b in rows]); s = np.array([b for a, b in rows])
    K = len(m); N = N_PER_GROUP * K
    gm = m.mean()
    ssb = N_PER_GROUP * ((m - gm) ** 2).sum()
    ssw = ((N_PER_GROUP - 1) * s ** 2).sum()
    msw = ssw / (N - K)
    F = (ssb / (K - 1)) / msw
    P = stats.f.sf(F, K - 1, N - K)
    RES[name] = dict(m=m, s=s, msw=msw, F=F, P=P)
    print(f"{name:18s} F(3,56) = {F:.2f}, P = {P:.4f}   [reported F(3, 56)={F_rep}, P={P_rep}]")
    Fs = []
    for dm in itertools.product([-5e-4, 5e-4], repeat=K):
        for ds in itertools.product([-5e-4, 5e-4], repeat=K):
            mm = m + np.array(dm); ss = s + np.array(ds)
            Fs.append((N_PER_GROUP * ((mm - mm.mean()) ** 2).sum() / (K - 1))
                      / (((N_PER_GROUP - 1) * ss ** 2).sum() / (N - K)))
    print(f"    F range under 3-decimal rounding of means and SDs: {min(Fs):.3f} to {max(Fs):.3f}")
    for i in range(1, K):
        d = m[i] - m[0]
        t = d / np.sqrt(msw * 2 / N_PER_GROUP)
        p_unadj = 2 * stats.t.sf(abs(t), N - K)
        print(f"    {GROUPS[i]:13s} - Controls = {d:+.3f} ({100 * (m[i] / m[0] - 1):+.1f}%; "
              f"ratio {m[i] / m[0]:.3f}), pooled t(56) = {t:+.2f}, unadjusted two-sided P = {p_unadj:.4f}")

print()
print("Cerebellar IDH3A: Dunnett two-sided P (3 comparisons vs controls, df 56)")
r = RES["Cerebellum IDH3A"]
DUN = {}
for i, g in enumerate(GROUPS[1:], start=1):
    t = (r["m"][i] - r["m"][0]) / np.sqrt(r["msw"] * 2 / N_PER_GROUP)
    p = dunnett_two_sided_p(t, 3, 56)
    DUN[g] = (t, p)
    print(f"    {g:13s} t = {t:+.3f}, Dunnett P = {p:.4f}   [reported P={DUNNETT_REPORTED[g]}]")

bd_ctrl = r["m"][1] - r["m"][0]
bd_ratio = r["m"][1] / r["m"][0]
print()
print(f"Direction, cerebellar IDH3A protein, BD vs controls: BD - controls = {bd_ctrl:+.3f} "
      f"({100 * (bd_ratio - 1):+.1f}%), ratio {bd_ratio:.3f} -> BD is "
      f"{'LOWER' if bd_ctrl < 0 else 'HIGHER'} than controls")
print(f"For contrast, BD vs the other patient groups: BD - MDD = {r['m'][1] - r['m'][2]:+.3f}, "
      f"BD - schizophrenia = {r['m'][1] - r['m'][3]:+.3f}")

print()
print("=" * 78)
print("Table 3A: DLPFC mRNA, controls (n=34) vs BD (n=35)")
print("=" * 78)
MRNA = {}
for name, (m1, s1, n1, m2, s2, n2, prep) in TABLE3A.items():
    st = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2)
    we = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=False)
    MRNA[name] = (m1, s1, n1, m2, s2, n2)
    print(f"{name:18s} BD - controls = {m2 - m1:+.3f} ({100 * (m2 / m1 - 1):+.1f}%; ratio {m2 / m1:.3f}); "
          f"Student P = {st.pvalue:.4f}, Welch P = {we.pvalue:.4f}   [reported P={prep}]")

print()
print("All seven Table 3A rows (n=34 vs 35). P recomputed from the printed means/SDs;")
print("'range' = smallest to largest P obtainable when each printed mean and SD may be")
print("off by up to 0.0005 from 3-decimal rounding (extremes over the 16 corners);")
print("'fits' = the range overlaps the interval the printed 3-decimal P stands for.")
def p_t(m1, s1, m2, s2, welch):
    return stats.ttest_ind_from_stats(m1, s1, 34, m2, s2, 35, equal_var=not welch).pvalue
def p_range(m1, s1, m2, s2, welch):
    ps = [p_t(m1 + a, s1 + b, m2 + c, s2 + d, welch)
          for a, b, c, d in itertools.product([-5e-4, 5e-4], repeat=4)]
    return min(ps), max(ps)
P_REP = []
for g, (m1, s1, m2, s2, prep, frep) in TABLE3A_ALL.items():
    ps, pw = p_t(m1, s1, m2, s2, False), p_t(m1, s1, m2, s2, True)
    (sl, sh), (wl, wh) = p_range(m1, s1, m2, s2, False), p_range(m1, s1, m2, s2, True)
    fits = lambda lo, hi: "fits" if (lo < prep + 5e-4 and prep - 5e-4 <= hi) else "does NOT fit"
    P_REP.append(prep)
    print(f"    {g:6s} Student P = {ps:.4f} (range {sl:.4f}-{sh:.4f}, {fits(sl, sh)}); "
          f"Welch P = {pw:.4f} (range {wl:.4f}-{wh:.4f}, {fits(wl, wh)})   [reported P={prep}]")

def bh(p):
    p = np.asarray(p, float); n = len(p); o = np.argsort(p)
    adj = np.minimum.accumulate((p[o] * n / np.arange(1, n + 1))[::-1])[::-1]
    out = np.empty(n); out[o] = np.minimum(adj, 1); return out
print()
print("Table 3A FDR column: Benjamini-Hochberg over the 7 genes")
fdr_rep = bh(P_REP)
fdr_lo = bh(np.maximum(np.array(P_REP) - 5e-4, 0)); fdr_hi = bh(np.array(P_REP) + 5e-4)
for i, (g, row) in enumerate(TABLE3A_ALL.items()):
    print(f"    {g:6s} BH of printed P = {fdr_rep[i]:.3f} (printed P +/- 0.0005 gives {fdr_lo[i]:.4f} to "
          f"{fdr_hi[i]:.4f})   [reported FDR P={row[5]}]")

# BD/control ratios with approximate 95% CIs (delta method on the log ratio),
# used only for the ratio figure (candidate D / fig2).
print()
print("BD/control ratios, approximate 95% CI (delta method on log ratio, normal approx.)")
RATIOS = []
def ratio_ci(m1, s1, n1, m2, s2, n2):
    lr = np.log(m2 / m1)
    se = np.sqrt(s1 ** 2 / (n1 * m1 ** 2) + s2 ** 2 / (n2 * m2 ** 2))
    return m2 / m1, np.exp(lr - 1.96 * se), np.exp(lr + 1.96 * se)
for name, (m1, s1, n1, m2, s2, n2) in MRNA.items():
    RATIOS.append((name, *ratio_ci(m1, s1, n1, m2, s2, n2)))
for name in ["Cerebellum IDH3A", "Cerebellum IDH3B", "Parietal IDH3A", "Parietal IDH3B"]:
    m = RES[name]["m"]; s = RES[name]["s"]
    RATIOS.append((name + " protein", *ratio_ci(m[0], s[0], 15, m[1], s[1], 15)))
for name, est, lo, hi in RATIOS:
    print(f"    {name:26s} ratio {est:.3f}  (95% CI {lo:.3f} to {hi:.3f})")

# ----------------------------------------------------------------------------
# 2. Figures
# ----------------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; INK2 = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3df"; PANEL = "#f3f2ef"; EDGE = "#c9c7c1"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})
W = 7.2

ABSTRACT_Q = ("Moreover, protein expression of IDH3A in the cerebellum from BD patients "
              "was higher than that of controls.")
RESULTS_Q = ("Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), "
             "major depressive disorder (P=0.006) and schizophrenia (P=0.001) were "
             "significantly lower than in controls (Table 3B).")
DISC_Q = ("Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients "
          "were lower than those in controls.")
TABLE_ROW = "IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002"


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def rich_text(ax, x, y, runs, width, size=10.5, line=1.45):
    """Draw word-wrapped text made of (text, color, weight) runs in axes coords,
    starting at the top-left (x, y). Returns the y of the next line."""
    fig = ax.figure
    rend = fig.canvas.get_renderer()
    inv = ax.transAxes.inverted()
    probe = ax.text(0, 0, "Ag", fontsize=size)
    bb = probe.get_window_extent(rend); probe.remove()
    lh = (inv.transform((0, bb.height)) - inv.transform((0, 0)))[1] * line
    sp_t = ax.text(0, 0, "a a", fontsize=size); sp_a = ax.text(0, 0, "aa", fontsize=size)
    sw = sp_t.get_window_extent(rend).width - sp_a.get_window_extent(rend).width
    sp_t.remove(); sp_a.remove()
    sw = 0.8 * (inv.transform((sw, 0)) - inv.transform((0, 0)))[0]
    cx, cy = x, y
    for text, color, weight in runs:
        for word in text.split(" "):
            if word == "":
                continue
            t = ax.text(cx, cy, word, color=color, fontweight=weight, fontsize=size,
                        va="top", ha="left", transform=ax.transAxes)
            wb = t.get_window_extent(rend)
            ww = (inv.transform((wb.width, 0)) - inv.transform((0, 0)))[0]
            if cx + ww > x + width and cx > x:
                cx, cy = x, cy - lh
                t.set_position((cx, cy))
            cx += ww + sw
    return cy - lh


def panel(ax, x, y, w, h, fc=PANEL, ec=EDGE, lw=1.0):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                transform=ax.transAxes, fc=fc, ec=ec, lw=lw))


def tag(ax, x, y, text, color):
    ax.text(x, y, text, transform=ax.transAxes, fontsize=10, color="white", fontweight="bold",
            va="center", ha="left",
            bbox=dict(boxstyle="round,pad=0.3", fc=color, ec="none"))


# ---- Candidate A (fig1): two text panels joined by a connector -------------
def design_A(path):
    fig = plt.figure(figsize=(W, 4.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.text(0.03, 0.955, "Same quantity, opposite direction:\ncerebellar IDH3A protein, BD vs controls (n=15 per group)",
            fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
    # left panel: abstract
    panel(ax, 0.03, 0.30, 0.38, 0.55)
    ax.text(0.05, 0.81, "Abstract", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
    rich_text(ax, 0.05, 0.73, [("“Moreover, protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
                               ("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.34)
    tag(ax, 0.05, 0.36, "reported: BD higher", ORANGE)
    # right panel: results + table
    panel(ax, 0.52, 0.30, 0.45, 0.55)
    ax.text(0.54, 0.81, "Results and Table 3B", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
    yy = rich_text(ax, 0.54, 0.73, [("“protein levels of IDH3A in the cerebellum from BD (P=0.017) [...] were significantly", INK, "normal"),
                                     ("lower", INK, "bold"), ("than in controls (Table 3B).”", INK, "normal")], 0.41)
    ax.text(0.54, yy - 0.015, "Table 3B, cerebellum IDH3A (mean ± s.d.):", fontsize=10, color=INK2,
            transform=ax.transAxes, va="top")
    ax.text(0.54, yy - 0.075, "controls 1.227 ± 0.543", fontsize=10, color=ORANGE, fontweight="normal",
            transform=ax.transAxes, va="top")
    ax.text(0.775, yy - 0.075, "BD 0.804 ± 0.390*", fontsize=10, color=ORANGE, fontweight="normal",
            transform=ax.transAxes, va="top")
    tag(ax, 0.54, 0.36, f"recomputed: BD − controls = {bd_ctrl:+.3f}", BLUE)
    # connector
    arr = FancyArrowPatch((0.415, 0.575), (0.515, 0.575), transform=ax.transAxes,
                          arrowstyle="<|-|>", mutation_scale=14, lw=2, color=INK2)
    ax.add_patch(arr)
    ax.text(0.465, 0.61, "conflict", ha="center", fontsize=10, color=INK2, transform=ax.transAxes,
            fontweight="bold")
    # footer
    rich_text(ax, 0.03, 0.22, [("The Discussion also states the protein levels “were lower than those in controls”. "
                                f"Recomputed from Table 3B: BD is {100 * (1 - bd_ratio):.1f}% below controls, pooled t(56) = {DUN['BD'][0]:.2f}, Dunnett P = {DUN['BD'][1]:.3f} "
                                "(reported P=0.017). Only the abstract gives the direction as higher.", INK2, "normal")],
              0.94, size=10)
    save(fig, path)


# ---- Candidate B: statement map across the paper ---------------------------
def design_B(path):
    rows = [
        ("Abstract", "“...was higher than that of controls.”", "higher", ORANGE),
        ("Results", "“...BD (P=0.017) ... significantly lower than in controls”", "lower", ORANGE),
        ("Table 3B", "controls 1.227±0.543   BD 0.804±0.390*", "lower", ORANGE),
        ("Discussion ¶1", "“...from BD patients were lower than those in controls.”", "lower", ORANGE),
        ("Discussion ¶2", "“We did find reduced expression of IDH3A protein ...”", "lower", ORANGE),
        ("Recomputed", f"BD − controls = {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%), Dunnett P = {DUN['BD'][1]:.3f}", "lower", BLUE),
    ]
    fig = plt.figure(figsize=(W, 4.3))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    ax.text(0.03, 0.955, "Direction of cerebellar IDH3A protein, BD vs controls, in each part of the paper",
            fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
    ax.text(0.03, 0.885, "orange = stated in the paper; blue = recomputed from Table 3B", fontsize=10,
            color=INK2, transform=ax.transAxes, va="top")
    top = 0.80; step = 0.118
    for i, (loc, txt, d, col) in enumerate(rows):
        y = top - i * step
        if loc == "Abstract":
            panel(ax, 0.02, y - 0.05, 0.96, 0.10, fc="#fbe9e1", ec=ORANGE, lw=1.5)
        elif loc == "Recomputed":
            ax.plot([0.03, 0.97], [y + 0.062, y + 0.062], color=EDGE, lw=1, transform=ax.transAxes)
        ax.text(0.04, y, loc, fontsize=10.5, fontweight="bold", va="center", transform=ax.transAxes)
        ax.text(0.20, y, txt, fontsize=10, va="center", transform=ax.transAxes, color=INK)
        mk = "^" if d == "higher" else "v"
        ax.plot([0.85], [y], marker=mk, ms=11, color=col, transform=ax.transAxes, ls="none")
        ax.text(0.875, y, d, fontsize=10.5, va="center", transform=ax.transAxes, color=INK, fontweight="bold")
    ax.text(0.04, 0.06, "The abstract is the only place that states the direction as higher.", fontsize=10,
            color=INK2, transform=ax.transAxes)
    save(fig, path)


# ---- Candidate C: group means ± s.d. with the abstract claim annotated ------
def design_C(path):
    m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
    fig, ax = plt.subplots(figsize=(W, 4.4))
    fig.subplots_adjust(left=0.11, right=0.97, top=0.86, bottom=0.12)
    x = np.arange(4)
    ax.errorbar(x, m, yerr=s, fmt="o", ms=9, color=ORANGE, ecolor=ORANGE, elinewidth=2, capsize=0,
                label="Table 3B: mean ± s.d. (reported)", zorder=3)
    ax.axhline(m[0], color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
    ax.text(3.45, m[0] + 0.03, "control mean", fontsize=10, color=INK2, ha="right")
    # recomputed difference bracket
    ax.annotate("", xy=(1.18, m[1]), xytext=(1.18, m[0]),
                arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
    ax.text(1.25, (m[0] + m[1]) / 2, f"recomputed:\n{bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)\n"
            f"Dunnett P = {DUN['BD'][1]:.3f}", color=INK, fontsize=10, va="center")
    ax.plot([], [], color=BLUE, lw=2, label="BD − controls, recomputed")
    # abstract claim
    ax.annotate("", xy=(0.82, m[0] + 0.42), xytext=(0.82, m[0] + 0.05),
                arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--", mutation_scale=14))
    ax.text(0.90, m[0] + 0.30, "abstract (reported):\nBD “higher than\nthat of controls”", color=INK,
            fontsize=10, ha="left", va="center")
    ax.set_xticks(x, ["Controls", "BD", "MDD", "Schizophrenia"])
    ax.set_xlim(-0.5, 3.5); ax.set_ylim(0, 2.0)
    ax.set_ylabel("IDH3A protein, cerebellum\n(relative expression)")
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ["top", "right"]:
        ax.spines[sp].set_visible(False)
    ax.legend(loc="upper right", bbox_to_anchor=(1.0, 1.02), frameon=False, fontsize=10)
    fig.suptitle("Cerebellar IDH3A protein is lower in BD than in controls (n=15 per group)",
                 x=0.03, ha="left", fontsize=11, fontweight="bold")
    save(fig, path)


# ---- Candidate D (fig2): BD/control ratios on a log axis --------------------
def design_D(path):
    order = ["DLPFC IDH3A mRNA", "DLPFC IDH3B mRNA", "Cerebellum IDH3A protein",
             "Cerebellum IDH3B protein", "Parietal IDH3A protein", "Parietal IDH3B protein"]
    abstract_note = {"DLPFC IDH3A mRNA": "abstract: lower", "DLPFC IDH3B mRNA": "abstract: lower",
                     "Cerebellum IDH3A protein": "abstract: higher"}
    d = {n: (e, l, h) for n, e, l, h in RATIOS}
    fig, ax = plt.subplots(figsize=(W, 4.3))
    fig.subplots_adjust(left=0.30, right=0.76, top=0.80, bottom=0.14)
    ys = np.arange(len(order))[::-1]
    ax.axvline(1, color=INK2, lw=1)
    for y, n in zip(ys, order):
        e, l, h = d[n]
        ax.plot([l, h], [y, y], color=BLUE, lw=2, solid_capstyle="round")
        ax.plot([e], [y], "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
        if n in abstract_note:
            note = abstract_note[n]
            ax.text(1.62, y, note, va="center", fontsize=10,
                    color=ORANGE if "higher" in note else INK2, fontweight="bold" if "higher" in note else "normal",
                    transform=ax.get_yaxis_transform() if False else ax.transData, clip_on=False)
    ax.set_xscale("log")
    ticks = [0.4, 0.5, 0.6, 0.8, 1.0, 1.25, 1.5]
    ax.set_xticks(ticks, [f"{t:g}" for t in ticks]); ax.minorticks_off()
    ax.set_xlim(0.38, 1.55)
    ax.set_yticks(ys, order)
    ax.set_xlabel("BD / control ratio of means (log scale); 1 = no difference")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for sp in ["top", "right", "left"]:
        ax.spines[sp].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.text(0.40, len(order) - 0.35, "← lower in BD", fontsize=10, color=INK2)
    ax.text(1.04, len(order) - 0.35, "higher in BD →", fontsize=10, color=INK2)
    ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="recomputed from\nTables 3A/3B\n(approx. 95% CI)")
    ax.legend(loc="lower left", bbox_to_anchor=(1.0, -0.02), frameon=False, fontsize=10,
              handlelength=1.6, borderaxespad=0)
    fig.suptitle("BD/control ratios for IDH3A and IDH3B, Tables 3A and 3B:\nall point estimates below 1", x=0.03, ha="left", y=0.985,
                 fontsize=11, fontweight="bold")
    save(fig, path)


# ---- Candidate E: printed vs supported sentence + number line --------------
def design_E(path):
    m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
    fig = plt.figure(figsize=(W, 4.3))
    ax = fig.add_axes([0, 0.50, 1, 0.50]); ax.axis("off")
    ax.text(0.03, 0.92, "The abstract sentence as printed, and as Table 3B supports it",
            fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
    ax.text(0.03, 0.66, "Printed", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
    rich_text(ax, 0.17, 0.66, [("“protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
                               ("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.80)
    ax.text(0.03, 0.28, "Supported", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
    rich_text(ax, 0.17, 0.28, [("protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
                               ("lower", BLUE, "bold"), ("than that of controls.", INK, "normal")], 0.80)
    bx = fig.add_axes([0.17, 0.20, 0.55, 0.25])
    for y, (mm, ss, lab) in zip([2, 0], [(m[0], s[0], "Controls"), (m[1], s[1], "BD")]):
        bx.plot([mm - ss, mm + ss], [y, y], color=ORANGE, lw=2)
        bx.plot([mm], [y], "o", ms=9, color=ORANGE)
        bx.text(1.02, y, f"{lab} {mm:.3f} ± {ss:.3f}", va="center", fontsize=10,
                transform=bx.get_yaxis_transform(), clip_on=False)
    bx.annotate("", xy=(m[1], 1), xytext=(m[0], 1),
                arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
    bx.text(m[0] + 0.04, 1, f"recomputed {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)", va="center",
            fontsize=10, color=INK)
    bx.set_ylim(-0.6, 2.6); bx.set_xlim(0, 2.0); bx.set_yticks([])
    bx.set_xlabel("Table 3B cerebellar IDH3A protein,\nmean ± s.d. (reported), n=15 per group", fontsize=10)
    bx.grid(axis="x", color=GRID, lw=0.8); bx.set_axisbelow(True)
    for sp in ["top", "right", "left"]:
        bx.spines[sp].set_visible(False)
    save(fig, path)


design_A(os.path.join(HERE, "fig1"))
design_D(os.path.join(HERE, "fig2"))
print()
print("Figures written: fig1.svg/png (design A), fig2.svg/png (design D)")
if CAND:
    os.makedirs(CAND, exist_ok=True)
    for nm, fn in zip("ABCDE", [design_A, design_B, design_C, design_D, design_E]):
        fn(os.path.join(CAND, f"candidate_{nm}"))
    print(f"Candidate designs written to {CAND}: candidate_A..E.svg/png")

Re-positive rate's 95% confidence interval excludes its own point estimate in abstract and Results

Nat Commun (2020) · doi:10.1038/s41467-020-19802-w · PMC7679396

Summary

The paper reports that 107 of 34,424 recovered COVID-19 patients tested positive again in Wuhan's city-wide screening, a re-positive rate of 0.31%, with a 95% confidence interval of 0.423–0.574%. The interval lies entirely above the point estimate, which none of the standard two-sided intervals (exact, score, Wald, Agresti–Coull, Jeffreys) does for 107/34,424. Applied to 107/34,424, the paper's stated method (Clopper–Pearson exact) gives 0.255–0.375%. The same method reproduces exactly all 27 other intervals in Tables 1–2 and the antibody text, and also the two single- and mixed-testing intervals once their unprinted counts are inferred. The point estimate and the paper's conclusions are unaffected, but the wrong interval is printed in the abstract. Anyone who extracts it, for example for a meta-analysis of re-positivity after recovery, gets a range whose lower bound lies above the correct upper bound, centred near 0.50% instead of 0.31%.

What the paper reports

Abstract:

107 of 34,424 previously recovered COVID-19 patients tested positive again (re-positive rate 0.31%, 95% CI 0.423–0.574%).

Results, second paragraph (screening results):

Of the 34,424 participants with a history of COVID-19, 107 tested positive again, giving a repositive rate of 0.310% (95% CI 0.423–0.574%).

Results, first paragraph (the denominators):

Of the 9899,828 participants, 9,865,404 had no previous diagnosis of COVID-19, and 34,424 were recovered COVID-19 patients.

Methods, "Statistical analysis" (the stated method, which covers both the asymptomatic and the re-positive rates):

Detection rate of asymptomatic positive or repositive cases was calculated by dividing the number of individuals with a positive result of nucleic acid testing by the number of participants tested. Because of extremely low detection rates, we calculated 95% confidence intervals of estimated proportions using Pearson–Klopper exact method, implemented through R package “binom” version 1.1-1…

Discussion, third paragraph (the statement that rests on this estimate):

There was a low repositive rate in recovered COVID-19 patients in Wuhan.

What the paper's own numbers show

Method. I took the count (107) and denominator (34,424) printed in the abstract and the Results and computed the proportion and its two-sided 95% Clopper–Pearson interval from beta quantiles. This is the interval that R's binom package returns for method = "exact". For comparison I also computed the Wilson, Wald, Agresti–Coull, Jeffreys and Poisson-exact intervals. To confirm that I was applying the paper's method the way the authors did, I applied the same calculation to every other interval in the paper whose count and denominator are printed: the 22 rows of Table 1 (asymptomatic positives per 10,000 by sex, age and district), the four rows of Table 2, the IgG-positive proportion in the Results (antibody results, % of 300) and the asymptomatic rate in the second paragraph of the Results. The paper has two further intervals, the asymptomatic rates for single and mixed testing in the fourth paragraph of the Results, for which only the share of samples (76.7% and 23.3%) is printed, not the counts. For these I took the denominators as 76.7% and 23.3% of 9,899,828 and searched for the counts whose exact interval matches the printed one. Finally I searched for the count, denominator and confidence level that would produce the published interval, and compared the counts in the publisher's Source Data workbook with the printed Tables 1 and 2. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Re-positive rate, 107/34,424 Abstract; Results para 2 0.31% (abstract), 0.310% (Results) 0.3108%, which rounds to 0.311%; the Results' 0.310% is the truncated value, a negligible difference
95% CI, Clopper–Pearson (stated method) Abstract; Results para 2 0.423–0.574% 0.255–0.375% (0.2548–0.3755%)
95% CI, other standard methods — — Wilson 0.257–0.375%; Wald 0.252–0.370%; Agresti–Coull 0.257–0.376%; Jeffreys 0.256–0.374%; Poisson exact 0.255–0.376%
Does the reported CI contain the point estimate? — — No; 0.311% is below the reported lower bound of 0.423%
Gap between the intervals — — Reported lower bound 0.423% exceeds the recomputed upper bound 0.375% by 0.048 points; the intervals do not overlap
Midpoint of the interval — 0.4985% (midpoint of 0.423–0.574%) 0.311%; the reported midpoint is 1.60 times the point estimate
Width / implied standard error — 0.151 points; implied SE 0.0385 points 0.121 points; binomial SE 0.0300 points
Reported bound ÷ recomputed bound — — lower 1.66, upper 1.53
Count x out of 34,424 whose exact CI rounds to 0.423–0.574% reverse search — only x = 170 (0.4225–0.5737%; rate 0.494%)
Denominator n for which 107/n gives 0.423–0.574% reverse search, n = 10,000–60,000 — none; the closest (n = 22,514) gives 0.390–0.574%
Other confidence levels for 107/34,424 reverse search — 90%: 0.263–0.365%; 99%: 0.239–0.397%; 99.9%: 0.221–0.422%. Even the 99.9% upper bound is below 0.423%
Table 1, 22 rates per 10,000 with CIs Table 1 e.g. Total 0.303 (0.270–0.339); Male 0.256 (0.214–0.303); Wuchang 0.807 (0.633–1.015); East Lake Scenic 0.000 (0.000–0.430) all 22 reproduce exactly with Clopper–Pearson
Antibody proportions, % of 300 Table 2; Results para 9 53.7 (47.8–59.4); 36.7 (31.2–42.4); 9.7 (6.6–13.6); 0.0 (0.0–1.2); IgG+ 63.3 (57.6–68.8) all 5 reproduce exactly from the printed counts (161, 110, 29, 0; IgG+ 190)
Asymptomatic rates, single and mixed testing (counts not printed) Results para 4 single 0.321 (0.282–0.364)/10,000; mixed 0.243 (0.183–0.315)/10,000 with denominators of about 7,593,168 (76.7% of 9,899,828) and 2,306,660 (23.3%), only the counts 244 and 56 reproduce the printed rate and interval, and 244 + 56 = 300, the total of asymptomatic positives. Both intervals are consistent with Clopper–Pearson
Asymptomatic rate, denominator 9,865,404 Results para 2 ("participants without a history of COVID-19") 0.303 (0.270–0.339)/10,000 0.304 (0.271–0.341)/10,000; the printed figures match 300/9,899,828 (all participants), as in Table 1
Denominators Results para 1; Table 1; Table 2 9,865,404 + 34,424; total 9,899,828 sum 9,899,828; sex, age and district totals also sum to 9,899,828 and district cases to 300; printed antibody counts sum to 300, with 161 + 29 = 190 IgG-positive
Antibody counts in the publisher's Source Data workbook Source Data, sheet "Raw data from Table 2" printed Table 2: 161, 110, 29, 0; Results: 190 IgG-positive the sheet gives 157, 110, 28, 5 (also summing to 300), hence 157 + 28 = 185 IgG-positive, 61.7% (55.9–67.2%) instead of 63.3% (57.6–68.8%). The sheet "Raw data from Table 1" matches the printed Table 1 in all 21 rows

The stated method reproduces every other interval in Tables 1–2 and the antibody text to the last printed digit, and the two single/mixed intervals are reproduced by inferred counts that add up to the paper's 300 asymptomatic positives, so the method itself is not in doubt. The count, the denominator and the point estimate are mutually consistent. Only the re-positive interval cannot be obtained from the printed inputs, under the stated method or any other standard one.

There is one minor, separate observation. The Results attach the asymptomatic rate to the 9,865,404 participants without a COVID-19 history, but the printed rate and interval are those for all 9,899,828 participants (Table 1, Total). The difference is in the third decimal and changes nothing. A second separate observation concerns the antibody counts: the publisher's Source Data workbook gives different Table 2 counts from the printed table (row above). This does not bear on the re-positive interval.

Figure

Reported and recomputed 95% confidence intervals for the re-positive rate 107/34,424

Figure 1. The re-positive rate among recovered patients on one percentage axis, starting at zero. The orange row is the reported point estimate and 95% CI (identical in the Abstract and the Results). The blue rows are intervals recomputed from the paper's own count and denominator, 107/34,424: Clopper–Pearson (the method the Methods name), Wilson score and Wald. The dashed vertical line marks the point estimate, 0.311%. Every recomputed interval contains it and spans about 0.25–0.38%. The reported interval, 0.423–0.574%, lies wholly to the right of its own point estimate and does not overlap any of the recomputed intervals.

Ratio of reported to recomputed confidence bounds for the paper's 31 intervals, two of them using inferred counts

Figure 2. Each reported confidence bound in the paper, divided by the Clopper–Pearson bound recomputed from that row's count and denominator and rounded to the same number of decimals (log scale; 1 means identical). The counts are printed in the paper except for the single- and mixed-testing rates, where they are inferred (244 and 56; see the table). Within each row, hollow circles (lower bounds) are drawn just above the row and filled diamonds (upper bounds) just below it, so both stay visible when their ratios are equal; lower bounds of zero are omitted. All 22 Table 1 intervals, all 5 antibody intervals (from the printed Table 2 counts) and both single/mixed intervals give a ratio of exactly 1.000. The Results-text asymptomatic rate, checked against the 9,865,404 denominator the text names, gives 0.996 and 0.994 (it matches the 9,899,828 total exactly). The re-positive interval (orange) gives 1.66 for the lower bound and 1.53 for the upper bound. It is the only interval that the stated method does not reproduce with any of the paper's denominators.

Why it matters

Explanations considered

Status

Suggested correction

  1. Replace the re-positive interval in the Abstract and in the second paragraph of the Results. If 107 of 34,424 is correct, the text would read "re-positive rate 0.31%, 95% CI 0.25–0.38%" in the Abstract and "repositive rate of 0.311% (95% CI 0.255–0.375%)" in the Results, using the Clopper–Pearson method stated in the Methods. (107/34,424 = 0.3108%; the printed 0.310% is the truncated value, and 0.311% is the rounded one.) If the count or denominator is the figure in error instead, correct it and recompute both the rate and the interval.
  2. Minor: in the second paragraph of the Results, either give the asymptomatic rate for the 9,865,404 participants without a COVID-19 history (0.304 per 10,000, 95% CI 0.271–0.341), or state that 0.303 (0.270–0.339) refers to all 9,899,828 participants, as in Table 1. Separately, "9899,828" in the first paragraph of the Results should read "9,899,828".
  3. Minor and separate: the antibody counts in the Source Data workbook (157, 110, 28, 5) differ from the printed Table 2 (161, 110, 29, 0) and from the 190 IgG-positive cases in the Results. Whichever set is in error should be corrected so that the Source Data and the printed table agree; if the Source Data counts are the correct ones, Table 2 and the 63.3% (57.6–68.8%) IgG-positive figure in the Results would change, the latter to 61.7% (55.9–67.2%).
Recomputation script (python recompute.py.txt)
"""Recompute the confidence intervals of PMC7679396
(Nat Commun 2020, doi:10.1038/s41467-020-19802-w).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

Inputs are copied from the paper as archived in PMC:
  Abstract and Results paragraph 2: 107 re-positive of 34,424 recovered patients,
      "0.31%" / "0.310% (95% CI 0.423-0.574%)"; 300 asymptomatic positives,
      "0.303 (95% CI 0.270-0.339)/10,000"
  Results paragraph 1: 9,899,828 tested = 9,865,404 without and 34,424 with prior COVID-19
  Table 1: counts, cases and detection rate per 10,000 (95% CI) by sex, age and district
  Table 2 and Results paragraph 9: antibody results in the 300 asymptomatic cases
  Results paragraph 4: single and mixed testing for 76.7% and 23.3% of samples, asymptomatic
      rates "0.321 (95% CI 0.282-0.364)/10,000 and 0.243 (95% CI 0.183-0.315)/10,000".
      The counts behind these two rates are not printed; they are inferred below.
  Methods: 95% CIs by the "Pearson-Klopper exact method" (R package binom), i.e. the
      Clopper-Pearson interval, computed here with the beta quantile function.
"""
import math
import os

import numpy as np
from scipy.stats import beta, chi2, norm

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)

# ---------------------------------------------------------------- inputs (from the paper)
X_RE, N_RE = 107, 34424                      # re-positive cases / recovered patients
REP_RE_PT_ABS, REP_RE_PT_RES = 0.31, 0.310   # % (abstract, Results)
REP_RE_CI = (0.423, 0.574)                   # % (abstract and Results)

N_TOTAL, N_NOHIST, N_HIST = 9899828, 9865404, 34424
X_ASYM = 300
REP_ASYM = (0.303, 0.270, 0.339)             # per 10,000 (abstract, Results, Table 1 Total)

# Table 1: label, participants, asymptomatic positives, reported rate (lo, hi) per 10,000
TABLE1 = [
    ("Total", 9899828, 300, (0.303, 0.270, 0.339)),
    ("Male", 5162960, 132, (0.256, 0.214, 0.303)),
    ("Female", 4736868, 168, (0.355, 0.303, 0.413)),
    ("Age <=17", 969014, 12, (0.124, 0.064, 0.216)),
    ("Age 18-44", 4448230, 104, (0.234, 0.191, 0.283)),
    ("Age 45-59", 2492943, 96, (0.385, 0.312, 0.470)),
    ("Age >=60", 1989641, 88, (0.442, 0.355, 0.545)),
    ("Wuchang", 904636, 73, (0.807, 0.633, 1.015)),
    ("Qingshan", 414312, 23, (0.555, 0.352, 0.833)),
    ("Qiaokou", 583440, 32, (0.548, 0.375, 0.774)),
    ("Hanyang", 717429, 29, (0.404, 0.271, 0.581)),
    ("Jianghan", 524224, 19, (0.362, 0.218, 0.566)),
    ("Hongshan", 1103079, 35, (0.317, 0.221, 0.441)),
    ("East Lake High-tech", 782987, 19, (0.243, 0.146, 0.379)),
    ("Jiangan", 800440, 19, (0.237, 0.143, 0.371)),
    ("Caidian", 503595, 11, (0.218, 0.109, 0.391)),
    ("Jiangxia", 671248, 14, (0.209, 0.114, 0.350)),
    ("Huangpi", 979920, 14, (0.143, 0.078, 0.240)),
    ("Hannan", 417022, 4, (0.096, 0.026, 0.246)),
    ("Dongxihu", 777204, 5, (0.064, 0.021, 0.150)),
    ("Xinzhou", 634408, 3, (0.047, 0.010, 0.138)),
    ("East Lake Scenic", 85884, 0, (0.000, 0.000, 0.430)),
]
# Table 2 and Results paragraph 9: label, count of 300, reported % (lo, hi)
TABLE2 = [
    ("IgM- IgG+", 161, (53.7, 47.8, 59.4)),
    ("IgM- IgG-", 110, (36.7, 31.2, 42.4)),
    ("IgM+ IgG+", 29, (9.7, 6.6, 13.6)),
    ("IgM+ IgG-", 0, (0.0, 0.0, 1.2)),
    ("IgG+ (text)", 190, (63.3, 57.6, 68.8)),
]
# Results paragraph 4: label, share of samples, reported rate (lo, hi) per 10,000
SINGLE_MIXED = [
    ("Single testing", 0.767, (0.321, 0.282, 0.364)),
    ("Mixed testing", 0.233, (0.243, 0.183, 0.315)),
]


# ---------------------------------------------------------------- interval methods
def clopper_pearson(x, n, a=0.05):
    lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
    hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
    return lo, hi


def wilson(x, n, z=Z):
    p = x / n
    d = 1 + z * z / n
    c = (p + z * z / (2 * n)) / d
    h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return c - h, c + h


def wald(x, n, z=Z):
    p = x / n
    se = math.sqrt(p * (1 - p) / n)
    return p - z * se, p + z * se


def agresti_coull(x, n, z=Z):
    nt = n + z * z
    pt = (x + z * z / 2) / nt
    se = math.sqrt(pt * (1 - pt) / nt)
    return pt - z * se, pt + z * se


def jeffreys(x, n, a=0.05):
    return beta.ppf(a / 2, x + 0.5, n - x + 0.5), beta.ppf(1 - a / 2, x + 0.5, n - x + 0.5)


def poisson_exact(x, n, a=0.05):
    lo = 0.0 if x == 0 else chi2.ppf(a / 2, 2 * x) / 2
    hi = chi2.ppf(1 - a / 2, 2 * x + 2) / 2
    return lo / n, hi / n


def pct(v):
    return 100.0 * v


# ---------------------------------------------------------------- 1. the re-positive rate
print("=" * 78)
print("1. Re-positive rate among recovered patients (Abstract; Results paragraph 2)")
print("=" * 78)
p_re = X_RE / N_RE
print(f"Point estimate 107/34,424 = {pct(p_re):.4f}%  (reported {REP_RE_PT_ABS}% abstract, "
      f"{REP_RE_PT_RES:.3f}% Results)")
print(f"Reported 95% CI: {REP_RE_CI[0]:.3f}-{REP_RE_CI[1]:.3f}%")
print(f"Does the reported CI contain the point estimate? {REP_RE_CI[0] <= pct(p_re) <= REP_RE_CI[1]}")
methods = [
    ("Clopper-Pearson exact (stated method)", clopper_pearson),
    ("Wilson score", wilson),
    ("Wald (normal approximation)", wald),
    ("Agresti-Coull", agresti_coull),
    ("Jeffreys", jeffreys),
    ("Poisson exact (count 107 as Poisson)", poisson_exact),
]
RE_CI = {}
for name, f in methods:
    lo, hi = f(X_RE, N_RE)
    RE_CI[name] = (lo, hi)
    print(f"  {name:40s} {pct(lo):.4f}-{pct(hi):.4f}%  -> {pct(lo):.3f}-{pct(hi):.3f}%")
cp_lo, cp_hi = RE_CI["Clopper-Pearson exact (stated method)"]
print(f"Width: reported {REP_RE_CI[1] - REP_RE_CI[0]:.3f} points; Clopper-Pearson "
      f"{pct(cp_hi - cp_lo):.3f} points")
print(f"Reported lower bound {REP_RE_CI[0]}% minus recomputed upper bound {pct(cp_hi):.3f}% = "
      f"{REP_RE_CI[0] - pct(cp_hi):.3f} points (no overlap)")
mid = (REP_RE_CI[0] + REP_RE_CI[1]) / 2
print(f"Midpoint of reported CI: {mid:.4f}%; ratio to point estimate {mid / pct(p_re):.2f}")
print(f"Reported bounds / Clopper-Pearson bounds: lower {REP_RE_CI[0] / pct(cp_lo):.2f}, "
      f"upper {REP_RE_CI[1] / pct(cp_hi):.2f}")
se_from_reported = (REP_RE_CI[1] - REP_RE_CI[0]) / (2 * Z)
se_true = pct(math.sqrt(p_re * (1 - p_re) / N_RE))
print(f"SE back-calculated from reported CI: {se_from_reported:.4f} points; binomial SE "
      f"{se_true:.4f} points (ratio {se_from_reported / se_true:.2f})")
print(f"Inverse-variance weight with the back-calculated SE, relative to the correct weight: "
      f"{(se_true / se_from_reported) ** 2:.2f}")

print("\nReverse search 1: which count x out of 34,424 gives a Clopper-Pearson CI that rounds "
      "to the reported one?")
match_x = []
for x in range(1, 400):
    lo, hi = clopper_pearson(x, N_RE)
    if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
        match_x.append(x)
        print(f"  x = {x}: {pct(lo):.4f}-{pct(hi):.4f}% (rate {pct(x / N_RE):.3f}%)")
X_MATCH = match_x[0] if match_x else None
for x in (169, 171):
    lo, hi = clopper_pearson(x, N_RE)
    print(f"  neighbour x = {x}: {pct(lo):.3f}-{pct(hi):.3f}%")

print("\nReverse search 2: which denominator n gives 107/n a Clopper-Pearson CI that rounds "
      "to the reported one?")
hits = []
for n in range(10000, 60001):
    lo, hi = clopper_pearson(X_RE, n)
    if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
        hits.append(n)
print(f"  denominators matching both bounds: {len(hits)}")
best = min(range(10000, 60001),
           key=lambda n: sum(abs(pct(b) - r) for b, r in zip(clopper_pearson(X_RE, n), REP_RE_CI)))
lo, hi = clopper_pearson(X_RE, best)
print(f"  closest: n = {best}: {pct(lo):.3f}-{pct(hi):.3f}% (lower bound cannot be matched)")

print("\nReverse search 3: other alpha levels for 107/34,424 (Clopper-Pearson)")
for conf in (0.90, 0.99, 0.999):
    lo, hi = clopper_pearson(X_RE, N_RE, a=1 - conf)
    print(f"  {int(conf * 1000) / 10:g}% CI: {pct(lo):.3f}-{pct(hi):.3f}%")
print("  Every standard two-sided interval for 107/34,424 contains 0.311%; none lies above it.")

# ---------------------------------------------------------------- 2. the method elsewhere
print()
print("=" * 78)
print("2. The stated method applied to the paper's other intervals")
print("=" * 78)
ROWS = []   # (group, label, rep_lo, rep_hi, cp_lo, cp_hi) on the paper's own scale


def show(group, label, x, n, rep, scale, nd):
    lo, hi = clopper_pearson(x, n)
    est, lo, hi = scale * x / n, scale * lo, scale * hi
    ok = (round(est, nd), round(lo, nd), round(hi, nd)) == tuple(round(v, nd) for v in rep)
    fmt = f"{{:.{nd}f}}"
    s = f"{fmt.format(est)} ({fmt.format(lo)}-{fmt.format(hi)})"
    r = f"{fmt.format(rep[0])} ({fmt.format(rep[1])}-{fmt.format(rep[2])})"
    print(f"  {label:22s} {x:>4d}/{n:<9,d} recomputed {s:24s} reported {r:24s} "
          f"{'match' if ok else 'DIFFERS'}")
    ROWS.append((group, label, rep[1], rep[2], lo, hi))
    return ok


print("Table 1, per 10,000:")
n_ok1 = sum(show("Table 1", lab, x, n, rep, 1e4, 3) for lab, n, x, rep in TABLE1)
print(f"  -> {n_ok1} of {len(TABLE1)} Table 1 rows reproduce exactly")
print("Table 2 and Results paragraph 9, % of 300:")
n_ok2 = sum(show("Table 2", lab, x, 300, rep, 100, 1) for lab, x, rep in TABLE2)
print(f"  -> {n_ok2} of {len(TABLE2)} antibody intervals reproduce exactly")
print("Results paragraph 2, asymptomatic rate with the 'no history' denominator 9,865,404:")
show("Results", "300/9,865,404", X_ASYM, N_NOHIST, REP_ASYM, 1e4, 3)
print("Results paragraph 4, single and mixed testing, per 10,000 (counts not printed; inferred):")
print("  Denominator taken as share x 9,899,828; the count is the one whose exact CI matches.")
SM_X = []
for lab, share, rp in SINGLE_MIXED:
    n_sm = round(share * N_TOTAL)
    xs = [x for x in range(0, X_ASYM + 1)
          if (round(1e4 * x / n_sm, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n_sm)))
          == rp]
    print(f"  {lab}: n ~ {n_sm:,}; counts reproducing {rp[0]:.3f} ({rp[1]:.3f}-{rp[2]:.3f}): {xs}")
    # robustness: shares that round to the printed percentage
    for sh in (share - 0.0003, share + 0.0002):
        n2 = round(sh * N_TOTAL)
        xs2 = [x for x in range(0, X_ASYM + 1)
               if (round(1e4 * x / n2, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n2)))
               == rp]
        print(f"      share {100 * sh:.2f}% (n = {n2:,}): matching counts {xs2}")
    SM_X.append(xs[0])
    show("Single/mixed", f"{lab} {xs[0]}/~{n_sm:,}", xs[0], n_sm, rp, 1e4, 3)
print(f"  Inferred counts {SM_X[0]} + {SM_X[1]} = {sum(SM_X)} (total asymptomatic positives "
      f"{X_ASYM})")
print("Re-positive rate, % (for comparison):")
show("Re-positive", "107/34,424", X_RE, N_RE, (REP_RE_PT_RES, *REP_RE_CI), 100, 3)

print("\nDenominator checks:")
print(f"  9,865,404 + 34,424 = {N_NOHIST + N_HIST:,} (reported total 9,899,828)")
print(f"  Male + Female = {5162960 + 4736868:,}")
print(f"  Sum of age groups = {969014 + 4448230 + 2492943 + 1989641:,}")
print(f"  Sum of districts = {sum(r[1] for r in TABLE1[7:]):,}; "
      f"cases = {sum(r[2] for r in TABLE1[7:])}")
print(f"  Antibody counts 161+110+29+0 = {161 + 110 + 29 + 0}; IgG+ 161+29 = {161 + 29}")

# ---------------------------------------------------------------- 3. Source Data workbook
# Counts copied from the publisher's Source Data workbook (41467_2020_19802_MOESM4_ESM.xlsx),
# sheets "Raw data from Table 1" and "Raw data from Table 2".
SD_TABLE1 = {
    "Male": (5162960, 132), "Female": (4736868, 168),
    "Age <=17": (969014, 12), "Age 18-44": (4448230, 104), "Age 45-59": (2492943, 96),
    "Age >=60": (1989641, 88), "Wuchang": (904636, 73), "Qingshan": (414312, 23),
    "Qiaokou": (583440, 32), "Hanyang": (717429, 29), "Jianghan": (524224, 19),
    "Hongshan": (1103079, 35), "East Lake High-tech": (782987, 19), "Jiangan": (800440, 19),
    "Caidian": (503595, 11), "Jiangxia": (671248, 14), "Huangpi": (979920, 14),
    "Hannan": (417022, 4), "Dongxihu": (777204, 5), "Xinzhou": (634408, 3),
    "East Lake Scenic": (85884, 0),
}
SD_TABLE2 = [("IgM- IgG+", 157), ("IgM- IgG-", 110), ("IgM+ IgG+", 28), ("IgM+ IgG-", 5)]
print()
print("=" * 78)
print("3. Publisher's Source Data workbook compared with the printed tables")
print("=" * 78)
t1 = {lab: (n, x) for lab, n, x, _ in TABLE1}
same1 = sum(t1[k] == v for k, v in SD_TABLE1.items())
print(f"Table 1 sheet: {same1} of {len(SD_TABLE1)} rows identical to the printed Table 1 "
      f"(the sheet has no Total row)")
print("Table 2 sheet vs printed Table 2 (counts of 300; % with Clopper-Pearson 95% CI):")
for (lab, xsd), (_, xp, rep) in zip(SD_TABLE2, TABLE2):
    lo, hi = clopper_pearson(xsd, 300)
    print(f"  {lab:10s} Source Data {xsd:>3d} -> {100 * xsd / 300:.1f} ({100 * lo:.1f}-{100 * hi:.1f})"
          f"   printed {xp:>3d} -> {rep[0]:.1f} ({rep[1]:.1f}-{rep[2]:.1f})"
          f"   {'same' if xsd == xp else 'DIFFERS'}")
sd_sum = sum(x for _, x in SD_TABLE2)
sd_igg = SD_TABLE2[0][1] + SD_TABLE2[2][1]
lo, hi = clopper_pearson(sd_igg, 300)
print(f"  Source Data sum = {sd_sum}; IgG+ = {SD_TABLE2[0][1]}+{SD_TABLE2[2][1]} = {sd_igg} -> "
      f"{100 * sd_igg / 300:.1f}% ({100 * lo:.1f}-{100 * hi:.1f}); printed IgG+ 190 -> 63.3% "
      f"(57.6-68.8)")
sd_igm = SD_TABLE2[2][1] + SD_TABLE2[3][1]
print(f"  IgM+ = {sd_igm} in the Source Data vs {29 + 0} in the printed table")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
    "axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
    "axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "axes.spines.top": False, "axes.spines.right": False,
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# Figure 1: reported vs recomputed interval for 107/34,424 on one % axis
rows = [
    ("Reported\n(Abstract; Results)", REP_RE_PT_RES, REP_RE_CI, ORANGE, "o"),
    ("Recomputed: Clopper–Pearson\n(the stated method)", pct(p_re),
     tuple(pct(v) for v in RE_CI["Clopper-Pearson exact (stated method)"]), BLUE, "o"),
    ("Recomputed: Wilson score", pct(p_re), tuple(pct(v) for v in RE_CI["Wilson score"]), BLUE, "s"),
    ("Recomputed: Wald", pct(p_re),
     tuple(pct(v) for v in RE_CI["Wald (normal approximation)"]), BLUE, "D"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, est, (lo, hi), col, mk), y in zip(rows, ys):
    ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
    ax.plot([lo, lo], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
    ax.plot([hi, hi], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
    ax.plot(est, y, marker=mk, ms=9, color=col, mec=col, zorder=4)
    ax.text(hi + 0.012, y, f"{lo:.3f}–{hi:.3f}%", va="center", color=col, fontsize=10.5)
    if col == ORANGE:
        ax.annotate(f"reported point estimate {est:.3f}%\nlies outside its own interval",
                    xy=(est + 0.008, y - 0.06), xytext=(0.395, y - 0.47), va="center",
                    color=ORANGE, fontsize=10, arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1))
ax.axvline(pct(p_re), color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(pct(p_re), len(rows) - 0.45, f"107/34,424 = {pct(p_re):.3f}%", ha="center",
        va="bottom", color=SEC, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
for t, r in zip(ax.get_yticklabels(), rows):
    t.set_color(r[3])
ax.set_xlim(0, 0.70)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Re-positive rate, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig1")

# Figure 2: reported bound / recomputed Clopper-Pearson bound for every interval in the paper
# (the two single/mixed-testing rates use the inferred counts above)
fig, ax = plt.subplots(figsize=(7.2, 4.2))
groups = [("Table 1 (22 rates per 10,000)", "Table 1"),
          ("Table 2 and Results\n(5 antibody %, printed counts)", "Table 2"),
          ("Results: asymptomatic rate,\ndenominator 9,865,404", "Results"),
          ("Results: single / mixed testing\n(2 rates, counts inferred)", "Single/mixed"),
          ("Re-positive rate, 107/34,424", "Re-positive")]
ys = np.arange(len(groups))[::-1].astype(float)
rng = np.random.default_rng(1)
for (glab, gkey), y in zip(groups, ys):
    sub = [r for r in ROWS if r[0] == gkey]
    col = ORANGE if gkey == "Re-positive" else BLUE
    for _, _, rlo, rhi, clo, chi in sub:
        jit = 0.0   # identical ratios within a group would overlap anyway
        if clo > 0:   # lower bounds drawn above the row centre, upper bounds below it
            ax.plot(rlo / round(clo, 3 if gkey != "Table 2" else 1), y + 0.14 + jit, "o", ms=8,
                    mfc="none", mec=col, mew=1.6, zorder=3)
        ax.plot(rhi / round(chi, 3 if gkey != "Table 2" else 1), y - 0.14 + jit, "D", ms=8,
                color=col, alpha=0.85, zorder=3)
re = [r for r in ROWS if r[0] == "Re-positive"][0]
ax.text(re[3] / round(re[5], 3), ys[-1] - 0.3,
        f"upper {REP_RE_CI[1] / round(pct(cp_hi), 3):.2f}", ha="center", va="top",
        color=ORANGE, fontsize=10)
ax.text(re[2] / round(re[4], 3), ys[-1] + 0.3,
        f"lower {REP_RE_CI[0] / round(pct(cp_lo), 3):.2f}", ha="center", va="bottom",
        color=ORANGE, fontsize=10)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.85, 2.0)
ax.set_xticks([0.9, 1.0, 1.25, 1.5, 2.0])
ax.set_xticklabels(["0.9", "1", "1.25", "1.5", "2"])
for y, txt in zip(ys[:4], ["all 22 match (ratio 1.000)",
                           "all 5 match (ratio 1.000)",
                           "0.996 and 0.994",
                           "both match (ratio 1.000)"]):
    ax.text(1.06, y, txt, va="center", color=BLUE, fontsize=10)
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([g[0] for g in groups])
ax.set_ylim(-0.6, len(groups) - 0.4)
ax.set_xlabel("Reported ÷ recomputed Clopper–Pearson bound (log scale)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "o", ms=8, mfc="none", mec=SEC, mew=1.6, label="lower bound")[0]
h2 = ax.plot([], [], "D", ms=8, color=SEC, label="upper bound")[0]
ax.legend(handles=[h1, h2], loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")

print("\nRatios plotted in Figure 2 (reported / recomputed, both rounded as printed):")
for g, lab, rlo, rhi, clo, chi in ROWS:
    nd = 1 if g == "Table 2" else 3
    rl = "n/a" if round(clo, nd) == 0 else f"{rlo / round(clo, nd):.3f}"
    print(f"  {g:12s} {lab:30s} lower {rl:>6s}  upper {rhi / round(chi, nd):.3f}")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)

Abstract says serum-treated human islets came from T2DM donors; the donor table lists none

Nat Metab (2024) · doi:10.1038/s42255-024-01130-8 · PMC12005094

Summary

The abstract's only human result says that "human islets from male and female donors with T2DM" showed fewer senescence markers after treatment with serum collected after an exercise training programme. The paper's own donor table (Extended Data Table 1) lists 17 islet donors, and all 17 are marked T2D = "NO", including the 11 used for serum treatment. In the Results, the Methods and Extended Data Table 2, T2DM status belongs to the people who gave the serum, so the experiment exposed islets from non-diabetic cadaveric donors to serum from exercise-trained people with T2DM. The abstract, the Results subsection heading and the Fig. 8 title instead describe an experiment on T2DM islets; no measured value changes, but the design behind the paper's main translational claim is misstated.

What the paper reports

The abstract (the only human result it states):

"Remarkably, human islets from male and female donors with T2DM treated with serum collected after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." (Abstract)

The same framing appears in a heading and a figure title:

"Postexercise serum reduces senescence markers in T2DM human islets" (Results, subsection heading)

"Postexercise serum reduces senescence markers in T2DM islets." (Fig. 8 title)

The body text describes the design differently. Diabetes is attributed to the serum donors, and the islets come from separate cadaveric donors:

"To test whether exercise training affects islet senescence in humans, we obtained serum from five individuals with T2DM who participated in a moderate-intensity exercise training programme. [...] Serum was matched by sex, age and body mass index (BMI) (Extended Data Table 2) to pancreatic islets from cadaveric donors, received from the Integrated Islet Distribution Program (IIDP) (Extended Data Table 1)." (Results, "Postexercise serum reduces senescence markers in T2DM human islets", para 1)

"Human islet donors were matched by sex, age and BMI to serum donors (individuals with or without T2DM who underwent a moderate-intensity aerobic training programme)." (Methods, "Serum treatment in vitro")

"Notably, serum obtained from exercise-trained people with T2DM decreased the expression of senescence markers in human cadaveric islets, translating these findings to human physiology." (Introduction, last paragraph)

"Details of donors are given in Extended Data Table 1." (Methods, "Human islets")

Extended Data Table 1, "Donor information for human pancreatic samples" (columns: Age (years) | T2D | BMI | Applications), has 17 rows. Every T2D cell reads "NO". Eleven rows list "Serum treatment" among the applications. Two examples:

"54 | NO | 24.5 | Serum treatment, WB, FACS, qPCR"

"39 | NO | 32.4 | Serum treatment, WB"

(Extended Data Table 1, first and eleventh rows)

Elsewhere, the paper describes its human islets as non-diabetic:

"we exposed cadaveric human islets from donors without T2DM to glucagon in vitro for 4 days" (Results, "Glucagon activates AMPK in mouse and human islets", para 4)

"human (nondiabetic, 37–62 years old, male and female, n = 8 for qPCR and n = 11 for β-gal expression) islets" (Fig. 3 legend, panels i–k)

What the paper's own numbers show

Method. Both Extended Data tables were transcribed from the PMC archive into recompute.py.txt (the same values appear in the version-of-record table images). The script counts the T2D column for all islet donors and for the subset whose Applications column includes "Serum treatment". It also checks the table sizes against the counts given in the text.

Quantity Location Reported Recomputed
T2D status of the serum-treated islet donors Abstract "donors with T2DM" 0 of 11 serum-treatment donors have T2D = YES (Extended Data Table 1)
Islet donors listed Extended Data Table 1 17 rows 17
Islet donors with T2D = YES Extended Data Table 1, T2D column all "NO" 0 of 17
Islet donors used for serum treatment Extended Data Table 1, Applications column 11 rows list "Serum treatment" 11 (ages 22–62); 0 with T2D
Serum volunteers listed Extended Data Table 2 vs Methods ("seven female and six male participants") 13 13 (consistent)
Serum volunteers with T2D = YES Extended Data Table 2 vs Results ("serum from five individuals with T2DM") 5 used, per the text 10 of 13 listed; the paper does not say which 5 of the 10 supplied serum for Fig. 8
Islet preparations Methods ("male (17) versus female (3) islets") vs Extended Data Table 1 17 + 3 = 20 17 rows (side discrepancy)

The serum-treatment rows in Extended Data Table 1 fall into two groups. Five rows are marked "Serum treatment, WB, FACS, qPCR", matching the five T2DM serum donors in the Results. Six rows are marked "Serum treatment, WB", with ages 22–57, which matches the age range in the Extended Data Fig. 5 legend for the non-T2D serum experiment ("Samples from individuals ranging from 22 to 57 years old, male, n=6/group"). On this reading the legend's n = 6 counts islet preparations; the legend itself does not say so. Every islet row in both groups is T2D = "NO". The split does not change the finding, but it is consistent with the Fig. 8 islets being five of the non-diabetic donors listed in Extended Data Table 1.

Figure

Abstract wording versus the paper's donor tables

Figure 1. On the left (orange) is the wording reported in the abstract and repeated in the Results subsection heading and the Fig. 8 title. On the right (blue) are counts recomputed from the paper's Extended Data Tables 1 and 2. The abstract says the islets came from donors with T2DM. Extended Data Table 1 lists 17 islet donors, none with T2D, including the 11 used for serum treatment. T2D = YES appears only among the serum volunteers in Extended Data Table 2 (10 of 13), and the Results text attributes T2DM to the serum donors ("serum from five individuals with T2DM"; the paper does not say which five of the ten were used). All counts come from recompute.py.txt.

Donor-level T2D status in Extended Data Tables 1 and 2

Figure 2. Each circle is one row of the paper's donor tables. Filled circles have T2D = YES and open circles have T2D = NO, as recomputed from the tables. The top row holds the 11 islet donors whose Applications column includes "Serum treatment", the middle row the 6 other islet donors, and the bottom row the 13 serum volunteers. Every islet donor is non-diabetic. The orange line gives the abstract's description of the islet donors for comparison.

Why it matters

The abstract sentence is the paper's only stated human result and its translational headline ("Remarkably, ..."). Taken at face value, it says exercise-conditioned serum reduced senescence markers in islets from people who already had T2DM, that is, in diabetic beta cells where senescence is the proposed therapeutic target. The experiment the paper actually documents is different: non-diabetic cadaveric islets cultured with serum from exercise-trained people with T2DM. These two designs answer different questions. The first tests whether diabetic islet tissue responds. The second tests whether circulating factors from trained diabetic individuals act on healthy islets.

No reported value, effect direction or P value changes, and the narrower conclusion (post-exercise serum from people with T2DM lowered senescence markers in cultured human islets) is unaffected. Three statements are framed in terms of people who already have T2DM, and the abstract's wording presents the human experiment as direct evidence for them: the abstract's closing claim of "promising therapeutic implications for T2DM", the Discussion's statement that the findings bear on "mechanisms that can restore β-cell function in individuals with T2DM", and the criterion stated in the Results subsection "Senolytic effects of exercise training" that exercise "must decrease the percentage of senescent cells in populations in which β-cell failure has already developed, such as in older individuals and those with insulin resistance or overt T2DM". These statements also draw on the mouse experiments (that criterion is introduced for the INK-ATTAC mouse model), so they do not depend on the human experiment alone. Because the abstract, the subsection heading and the Fig. 8 title all use the same wording, a reader sees the correct design only in the Results paragraph, the Methods and the Extended Data tables.

Explanations considered

Misplaced modifier or shorthand. The wording probably means "human islets treated with serum from male and female donors with T2DM". The Reporting Summary supports this: "Serum was obtained from male and female volunteers with established type 2 diabetes ... A total of 5 subjects were used in the manuscript, being 2 females and 3 males". The phrase "male and female ... with established type 2 diabetes" there describes the serum donors. This explains how the wording arose but does not make the published sentence accurate. Read normally, "human islets from male and female donors with T2DM" says the islet donors had T2DM, and the heading and Fig. 8 title ("T2DM islets") say the same.

Swapped tables. The footers of the two Extended Data tables appear swapped. Table 1 (islet donors) ends "Human serum information.", and Table 2 (serum volunteers) ends "IIDP cadaveric human islets information." If the tables themselves were swapped, the islets would be the 10 T2D = YES rows and the abstract would be right. The table contents rule this out: - Table 1 is captioned "Donor information for human pancreatic samples" and has an Applications column (serum treatment, WB, FACS, qPCR) that only makes sense for islet preparations. - Table 2 is captioned "Volunteer information for serum from moderate intensity training program", has subject IDs, and has 13 rows, matching the Methods' "seven female and six male participants". - The Methods say "Details of donors are given in Extended Data Table 1" under "Human islets". - The Results text attributes T2DM to the serum donors. - Fig. 3, Fig. 4 and the Results describe the human islets as non-diabetic.

The swapped footers are a separate, minor labelling error that adds to the confusion.

Conversion or archiving artifact. Ruled out. The conflict is categorical (YES/NO cells, the words "with T2DM"), not numeric. The version of record on nature.com has the same abstract sentence and the same Fig. 8 title. The version-of-record images of Extended Data Tables 1 and 2 have the same T2D columns (17 of 17 NO; 10 of 13 YES) and the same swapped footers.

Some islet donors had T2DM but were not listed. Nothing in the paper supports this. The Methods point to Extended Data Table 1 for donor details, the table has a T2D column, and no row is YES. The Methods do count 20 islet preparations ("male (17) versus female (3)") against 17 table rows. That gap is a separate inconsistency, but no passage anywhere in the paper says any islet donor had T2DM.

Open points about the serum donors (not the main finding). The Results say serum came from "five individuals with T2DM" and the Reporting Summary says "A total of 5 subjects were used in the manuscript", while Extended Data Table 2 lists 10 volunteers with T2D = YES. This need not be a contradiction: the Methods say serum samples were received from 13 participants, so the table may list every volunteer while only five T2DM sera were used for Fig. 8. The paper does not say which five. Similarly, the Extended Data Fig. 5 legend ("n=6/group") does not say whether n counts islet preparations or serum donors; six islet preparations treated with serum from the 3 non-T2D volunteers in Extended Data Table 2 would fit. Neither point affects the T2D status of the islet donors.

Status

As of 2026-10-23:

Sources: PMC12005094 (PubMed Central archive, read with src/show.py); https://www.nature.com/articles/s42255-024-01130-8; Reporting Summary https://static-content.springer.com/esm/art%3A10.1038%2Fs42255-024-01130-8/MediaObjects/42255_2024_1130_MOESM2_ESM.pdf; Source Data Fig. 8 (42255_2024_1130_MOESM10_ESM.xlsx, same location); https://api.crossref.org/works?filter=updates:10.1038/s42255-024-01130-8.

Suggested correction

Reword the abstract sentence so that T2DM describes the serum donors, for example: "Remarkably, human islets from non-diabetic donors treated with serum collected from male and female individuals with T2DM after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." Make the same change to the Results subsection heading and the Fig. 8 title (e.g. "Postexercise serum from individuals with T2DM reduces senescence markers in human islets"). In the same correction, the authors could also:

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC12005094 and regenerate fig1/fig2.

Run:  python recompute.py.txt   (writes fig1.svg/png and fig2.svg/png next to this file)

Inputs are transcribed from the article as archived in PubMed Central (PMC12005094):
Extended Data Table 1 ("Donor information for human pancreatic samples") and
Extended Data Table 2 ("Volunteer information for serum from moderate intensity
training program"). The same values appear in the version-of-record table images.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# Extended Data Table 1: (age, T2D, BMI, applications), in table order
EDT1 = [
    (54, "NO", 24.5, "Serum treatment, WB, FACS, qPCR"),
    (45, "NO", 21.9, "Serum treatment, WB, FACS, qPCR"),
    (62, "NO", 23, "Serum treatment, WB, FACS, qPCR"),
    (58, "NO", 58, "Serum treatment, WB, FACS, qPCR"),
    (53, "NO", 31.20, "Serum treatment, WB, FACS, qPCR"),
    (53, "NO", 33.6, "Serum treatment, WB"),
    (22, "NO", 31.79, "Serum treatment, WB"),
    (57, "NO", 30.9, "Serum treatment, WB"),
    (52, "NO", 31.6, "Serum treatment, WB"),
    (52, "NO", 37.5, "Serum treatment, WB"),
    (39, "NO", 32.4, "Serum treatment, WB"),
    (49, "NO", 34.8, "FACS, qPCR"),
    (37, "NO", 28.4, "FACS, qPCR"),
    (56, "NO", 32.8, "FACS, qPCR"),
    (42, "NO", 23.5, "FACS, qPCR"),
    (39, "NO", 31.1, "FACS, qPCR"),
    (58, "NO", 31.8, "FACS, qPCR"),
]
# Extended Data Table 2: (subject id, age, BMI, T2D), in table order
EDT2 = [
    (1, 48, 26.6, "YES"), (3, 51, 34.1, "YES"), (4, 55, 30.1, "YES"), (8, 55, 25, "YES"),
    (9, 54, 25.7, "YES"), (2, 41, 30.3, "YES"), (5, 41, 33.6, "YES"), (6, 50, 32.3, "YES"),
    (7, 46, 33.1, "YES"), (10, 52, 28, "YES"), (11, 53, 33.6, "NO"), (12, 27, 30.2, "NO"),
    (13, 54, 31.7, "NO"),
]

# ---------------------------------------------------------------- recomputation
n_isl = len(EDT1)
isl_t2d = sum(r[1] == "YES" for r in EDT1)
serum_isl = [r for r in EDT1 if "Serum treatment" in r[3]]
serum_isl_t2d = sum(r[1] == "YES" for r in serum_isl)
grpA = [r for r in serum_isl if "qPCR" in r[3]]       # Serum treatment, WB, FACS, qPCR
grpB = [r for r in serum_isl if "qPCR" not in r[3]]   # Serum treatment, WB
n_ser = len(EDT2)
ser_t2d = sum(r[3] == "YES" for r in EDT2)
ser_no_ages = sorted(r[1] for r in EDT2 if r[3] == "NO")

print("== Islet donors, Extended Data Table 1 ==")
print(f"rows listed:                               {n_isl}")
print(f"T2D = YES:                                 {isl_t2d} of {n_isl}")
print(f"rows whose Applications include 'Serum treatment': {len(serum_isl)}")
print(f"  of these T2D = YES:                      {serum_isl_t2d} of {len(serum_isl)}")
print(f"  ages: {min(r[0] for r in serum_isl)}-{max(r[0] for r in serum_isl)}")
print(f"  'Serum treatment, WB, FACS, qPCR' rows:  {len(grpA)} (ages {sorted(r[0] for r in grpA)})")
print(f"  'Serum treatment, WB' rows:              {len(grpB)} (ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)})")
print()
print("== Serum volunteers, Extended Data Table 2 ==")
print(f"rows listed:                               {n_ser}")
print(f"T2D = YES:                                 {ser_t2d} of {n_ser}")
print(f"T2D = NO:                                  {n_ser - ser_t2d} (ages {ser_no_ages})")
print()
print("== Cross-checks against the text ==")
print(f"Methods 'seven female and six male participants' = {7 + 6}; equals EDT2 rows: {7 + 6 == n_ser}")
print(f"Results 'serum from five individuals with T2DM' = 5; EDT2 T2D = YES rows listed: {ser_t2d} "
      f"(the paper does not say which 5 of the {ser_t2d} supplied serum for Fig. 8)")
print(f"Methods 'male (17) versus female (3) islets' = {17 + 3}; EDT1 rows: {n_isl}")
print(f"Ext. Data Fig. 5 legend 'n=6/group', '22 to 57 years old'; EDT1 'Serum treatment, WB' rows: "
      f"{len(grpB)}, ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)}; EDT2 non-T2D volunteers: "
      f"{n_ser - ser_t2d} (the legend does not say whether n counts islet preparations or serum donors)")
print()
print(f"Verdict: islet donors with T2D in EDT1 = {isl_t2d}; the abstract's 'human islets from male and "
      f"female donors with T2DM' is not supported by the paper's own donor table.")

# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
                     "text.color": INK})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"))
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
    plt.close(fig)


# ---------------------------------------------------------------- fig1: text-vs-table diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")


def panel(x, y, w, h, color, title, sub, body):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=color, lw=2))
    ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=color)
    ax.text(x + 0.14, y + h - 0.44, sub, ha="left", va="top", fontsize=10, color=INK2)
    ax.text(x + 0.14, y + h - 0.80, body, ha="left", va="top", fontsize=10, color=INK, linespacing=1.4)


panel(0.15, 1.55, 3.15, 2.95, ORANGE, "As reported (abstract)",
      "Repeated in a heading and Fig. 8 title",
      "\u201chuman islets from male and\nfemale donors with T2DM\ntreated with serum collected\nafter a 10-week endurance\nexercise training programme\u201d\n\nResults heading: \u201c\u2026 in T2DM\nhuman islets\u201d\nFig. 8 title: \u201c\u2026 in T2DM islets\u201d")
panel(3.9, 1.55, 3.15, 2.95, BLUE, "The paper's own tables",
      "Extended Data Tables 1 and 2",
      f"Islet donors (Table 1): {n_isl} listed\n"
      f"  with T2D = YES: {isl_t2d} of {n_isl}\n"
      f"  used for serum treatment: {len(serum_isl)}\n"
      f"  of those with T2D: {serum_isl_t2d} of {len(serum_isl)}\n\n"
      f"Serum volunteers (Table 2): {n_ser}\n"
      f"  with T2D = YES: {ser_t2d} of {n_ser}\n"
      "Results: \u201cserum from five\nindividuals with T2DM\u201d\n"
      "(which 5 is not stated)")
# conflict box joined to both panels
ax.add_patch(FancyBboxPatch((1.0, 0.22), 5.2, 0.86, boxstyle="round,pad=0.02,rounding_size=0.08",
                            fc=BG, ec=INK2, lw=1.2, ls="--"))
ax.text(3.6, 0.83, "Conflict: who had T2DM?", ha="center", va="center", fontsize=11, weight="bold", color=INK)
ax.text(3.6, 0.47, "Abstract: the islet donors.  Tables and Results: only the serum donors;\n"
        f"all {n_isl} islet donors are T2D = NO.", ha="center", va="center", fontsize=10, color=INK)
for x0 in (1.725, 5.475):
    ax.add_patch(FancyArrowPatch((x0, 1.53), (3.6 + (x0 - 3.6) * 0.55, 1.10), arrowstyle="-|>",
                                 mutation_scale=13, lw=2, color=INK2, connectionstyle="arc3,rad=0"))
save(fig, "fig1")

# ---------------------------------------------------------------- fig2: donor-level dot strip
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.34, right=0.95, top=0.80, bottom=0.24)
rows = []
# row 0: islet donors used for serum treatment; row 1: other islet donors; row 2: serum volunteers
for i, r in enumerate(serum_isl):
    rows.append((i, 2, r[1] == "YES"))
for i, r in enumerate([r for r in EDT1 if "Serum treatment" not in r[3]]):
    rows.append((i, 1, r[1] == "YES"))
for i, r in enumerate(EDT2):
    rows.append((i, 0, r[3] == "YES"))
for x, y, yes in rows:
    ax.plot(x, y, "o", ms=11, mec=BLUE, mew=2, mfc=BLUE if yes else BG)
ax.set_yticks([0, 1, 2])
ax.set_yticklabels([f"Serum volunteers\n(Ext. Data Table 2, n={n_ser})",
                    f"Islet donors, other uses\n(Ext. Data Table 1, n={n_isl - len(serum_isl)})",
                    f"Islet donors, serum treatment\n(Ext. Data Table 1, n={len(serum_isl)})"])
ax.set_xlim(-0.7, max(len(serum_isl), n_ser) - 0.3); ax.set_ylim(-0.6, 2.6)
ax.set_xticks([]); ax.tick_params(axis="y", length=0, colors=INK)
for s in ax.spines.values():
    s.set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.text(len(serum_isl) - 0.3, 2, f"  {serum_isl_t2d} of {len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.text(n_isl - len(serum_isl) - 0.3, 1, f"  {isl_t2d} of {n_isl - len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.annotate(f"{ser_t2d} of {n_ser} T2D", xy=(ser_t2d - 1, 0), xytext=(ser_t2d - 1, -0.52),
            ha="center", va="center", fontsize=10, color=BLUE)
handles = [Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BLUE,
                  label="T2D = YES (recomputed from tables)"),
           Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BG, label="T2D = NO")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.text(0.02, 0.10, "As reported in the abstract: the islets were \u201cfrom male and female donors with T2DM\u201d.",
         fontsize=10, color=ORANGE, ha="left")
fig.text(0.02, 0.025, "Each circle is one table row; fill shows the T2D column of that row.", fontsize=10, color=INK2, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")

Abstract reports WGS spanned 49 (98%) of 48 P-VNTRs; the body gives 92% and 45

PLoS Comput Biol (2025) · doi:10.1371/journal.pcbi.1012885 · PMC11975116

Summary

The paper compares targeted and whole-genome (WGS) PacBio HiFi sequencing for genotyping a curated set of 48 phenotype-associated VNTRs (P-VNTRs). The Abstract says the WGS data spanned "49 (98%) of P-VNTRs with at least 15 spanning reads". That count cannot be right, because 49 of 48 is 102.1%. The body gives two other values for the same quantity: 92% (Results 2.5.2 and the Discussion) and "45 VNTRs (out of 48)" (Discussion), which is 93.75%, not 92%. The best-supported figure from the paper's own text is 45 of 48 (about 94%). The Abstract's 98% therefore overstates WGS coverage of the P-VNTR set by roughly 4 to 6 percentage points. The paper's comparison with targeted sequencing (4 of 48) is unaffected.

What the paper reports

In contrast, the 40X WGS HiFi dataset spanned 98% of all VNTRs and 49 (98%) of P-VNTRs with at least 15 spanning reads, albeit with lower coverage.

— Abstract

Similar to G-VNTRs, whole genome sequencing covered 92% of P-VNTRs with median spanning coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 VNTRs were not well-covered likely due to VNTR length.

— Results, section 2.5.2

In contrast, whole genome sequencing was unbiased with at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. For P-VNTRs, the gap in spanning reads between the two sequencing methods was even more stark, with only 4 VNTRs in targeted sequencing and 45 VNTRs (out of 48) in whole genome sequencing supported by at least 15 spanning reads.

— Discussion, paragraph 1

The size of the P-VNTR set is stated, and adds up to 48, in several places:

We curated a set of 10 , 787 gene-proximal (G-)VNTRs, and 48 phenotype-associated (P-)VNTRs of interest.

— Abstract (the spacing in "10 , 787" is as archived)

The P-VNTRs includes 39 pathogenic and 9 non-pathogenic phenotype-associated loci.

— Fig 3 legend, panel (a)

However, 19 (39%) of 48 P-VNTRs had high repeat-unit count (RC) variation, … Another 21 had low RC variation, while the remaining 8 VNTRs associated with phenotypes based not on length but on the presence of specific motifs …

— Results, section 2.5

Only 4 VNTRs of the 48 P-VNTRs were spanned by ≥ 15 targeted spanning reads.

— Results, section 2.5.1

What the paper's own numbers show

Method: take the denominator the paper states (48 P-VNTRs, confirmed by 39 + 9 and by 19 + 21 + 8). Convert each reported count to a percentage, and each reported percentage to the count it implies over 48. Then compare these with the count implied by the three loci that Results 2.5.2 names as not well covered. All arithmetic is in recompute.py.txt.

Quantity Location Reported Recomputed from the paper's own numbers
Size of the P-VNTR set Abstract; Results 2; Fig 3a legend; Results 2.5; Discussion 48 39 + 9 = 48; 19 + 21 + 8 = 48
P-VNTRs spanned by ≥ 15 WGS reads (count) Abstract 49 49/48 = 102.08%, which exceeds the whole set
Same quantity (percentage) Abstract 98% 0.98 × 48 = 47.04, i.e. 47/48 = 97.92%. This is neither 49 nor any count in the body
Same quantity (percentage) Results 2.5.2; Discussion 92% 0.92 × 48 = 44.16, i.e. 44/48 = 91.67%
Loci named as not well covered by WGS Results 2.5.2 CACNA1C, DUX4, ABCA7 48 − 3 = 45, i.e. 45/48 = 93.75% (rounds to 94%)
Same quantity (count) Discussion 45 (out of 48) 45/48 = 93.75%. The same paragraph says 92%
Abstract overstatement, in percentage points Abstract vs body — 98 − 93.75 = 4.25 (vs 45/48); 98 − 91.67 = 6.33 (vs 44/48)
P-VNTRs spanned by ≥ 15 targeted reads Abstract; Results 2.5.1; Discussion 4 (8%) 4/48 = 8.33%, consistent
High-RC P-VNTRs Results 2.5 19 (39%) 19/48 = 39.58% (would round to 40%), a minor rounding slip
Low-RC + motif P-VNTRs Results 2.5 29 (60% of 48) 29/48 = 60.42%, consistent

The paper's statements give three different counts for one quantity: 49 (Abstract), about 44 (the 92% in Results and Discussion) and 45 (Discussion). Only 45 agrees with the three loci the Results names as poorly covered. No count over 48 can produce both "49" and "98%".

Figure

Abstract, Results and Discussion give three incompatible values for WGS coverage of the 48 P-VNTRs

Figure 1. Three passages of the paper, quoted verbatim (orange boxes), report WGS coverage of the same 48 P-VNTRs at the same threshold (at least 15 spanning reads). The blue connectors give the arithmetic from the paper's own numbers. The Abstract's "49" is larger than the set (49/48 = 102.1%), and its 98% would correspond to 47 loci. Results gives 92% (about 44 loci) but names three loci as not well covered, which implies 45. The Discussion gives both 92% and 45 of 48 (93.75%). The set size of 48 is stated and adds up consistently throughout the paper.

Reported counts vs the count implied by the three named poorly covered loci

Figure 2. Each reported value expressed as a count out of 48 (orange circles; the 92% statements are shown as 0.92 × 48 = 44.2), set against 45, the count left after removing the three loci that Results 2.5.2 names as not well covered (blue diamonds). The dashed line marks the whole set of 48, and the shaded region holds counts larger than the set. Only the Abstract's value falls there. The axis starts at zero.

Why it matters

The Abstract is the part of the paper most readers see and cite, and it states a P-VNTR count that is larger than the P-VNTR set. Its 98% looks plausible on its face, so a reader who does not check it against the stated set size of 48 would take WGS coverage of the clinically relevant loci to be about 4 to 6 points higher than the body supports. The 98% also matches the G-VNTR figure in the same sentence, which makes WGS look equally complete for both sets. The body itself says otherwise: three named loci (CACNA1C, DUX4, ABCA7) were not well covered, largely because of their length.

The error does not change the direction of the paper's main comparison. Every value the body supports (44 or 45 of 48, 92% to 94%) remains far above targeted sequencing (4 of 48, 8.3%). The paper's conclusion that WGS is more effective for broad VNTR genotyping stands. The G-VNTR figure (98% of 10,787) is reported consistently and is not affected. The problem is an impossible count, and an inflated percentage, in the Abstract for a secondary result. There is also a smaller internal inconsistency (92% vs 45/48) in the body.

Explanations considered

Status

Status as of 2026-10-23.

Suggested correction

Replace "49 (98%) of P-VNTRs" in the Abstract with the count and percentage supported by the per-locus data behind Fig 3b, over the stated set of 48 P-VNTRs. If the three loci named in Results 2.5.2 are the only ones below the threshold, this would read "45 (94%) of P-VNTRs". Change the "92%" in Results 2.5.2 and in the Discussion to match (for example "94%", or "45 of 48"), so that the Discussion's "92%" and "45 VNTRs (out of 48)" agree. If the coverage rule in the Abstract ("at least 15 spanning reads") differs from the one in Results ("median spanning coverage ≥ 15 … in 28 samples"), state one rule and use it throughout. Optionally, also change "19 (39%)" in Results 2.5 to "19 (40%)".

Recomputation script (python recompute.py.txt)
# Recompute every number in the PMC11975116 write-up and regenerate its figures.
# Usage: python recompute.py.txt   (needs numpy + matplotlib)
# Every input below is copied from the article text (doi:10.1371/journal.pcbi.1012885).
# Figures are written next to this file: fig1.*, fig2.* and five design candidates in candidates/.
import os
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (paper)
N_PATH, N_NONPATH = 39, 9            # Fig 3a legend
N_HIGH, N_LOW, N_MOTIF = 19, 21, 8   # Results 2.5
N_STATED = 48                        # Abstract, Results 2 intro, 2.5.1, Discussion ("out of 48")
ABS_COUNT, ABS_PCT = 49, 98          # Abstract: "49 (98%) of P-VNTRs"
RES_PCT = 92                         # Results 2.5.2 and Discussion: "92% of (the) P-VNTRs"
DIS_COUNT = 45                       # Discussion: "45 VNTRs (out of 48)"
NAMED_POOR = ["CACNA1C", "DUX4", "ABCA7"]  # Results 2.5.2: "not well-covered"
TGT_COUNT, TGT_PCT = 4, 8            # Abstract / Results 2.5.1 / Discussion
S2_ROWS, S2_UNIQUE = 49, 48          # S2 Table: 49 rows, 48 unique coordinates (MUC6 listed twice)

# ---------------------------------------------------------------- arithmetic
def pct(a, b):
    return 100.0 * a / b

print("== Size of the P-VNTR set")
print(f"pathogenic + non-pathogenic = {N_PATH} + {N_NONPATH} = {N_PATH + N_NONPATH}")
print(f"high-RC + low-RC + motif    = {N_HIGH} + {N_LOW} + {N_MOTIF} = {N_HIGH + N_LOW + N_MOTIF}")
assert N_PATH + N_NONPATH == N_HIGH + N_LOW + N_MOTIF == N_STATED
N = N_STATED

print("\n== Abstract: '49 (98%)'")
print(f"49 / 48 = {pct(ABS_COUNT, N):.2f}%  (exceeds 100%)")
print(f"98% of 48 = {ABS_PCT/100*N:.2f} loci; nearest count 47 -> 47/48 = {pct(47, N):.2f}%")

print("\n== Results 2.5.2 / Discussion: '92%'")
print(f"92% of 48 = {RES_PCT/100*N:.2f} loci; nearest count 44 -> 44/48 = {pct(44, N):.2f}%")
implied = N - len(NAMED_POOR)
print(f"48 - {len(NAMED_POOR)} named poorly covered loci ({', '.join(NAMED_POOR)}) = {implied}")
print(f"{implied}/48 = {pct(implied, N):.2f}%  (rounds to {round(pct(implied, N))}%)")

print("\n== Discussion: '45 VNTRs (out of 48)'")
print(f"45/48 = {pct(DIS_COUNT, N):.2f}%  (the same paragraph states 92%)")

print("\n== Counts that would round to each stated percentage (denominator 48)")
for k in range(43, 49):
    print(f"  {k}/48 = {pct(k, N):6.2f}%")

print("\n== Size of the abstract's overstatement (percentage points)")
print(f"98 - 93.75 = {ABS_PCT - pct(45, N):.2f} pp ; 98 - 91.67 = {ABS_PCT - pct(44, N):.2f} pp")

print("\n== Targeted sequencing (context)")
print(f"4/48 = {pct(TGT_COUNT, N):.2f}%  (stated 8%)")

print("\n== Other P-VNTR percentages in Results 2.5 / 2.5.1")
print(f"19/48 = {pct(19, N):.2f}% (stated 39%); 29/48 = {pct(29, N):.2f}% (stated 60%); "
      f"23/48 = {pct(23, N):.2f}% (stated 48%)")

print("\n== Alternative denominator (49 rows in S2 Table)")
for k in (45, 48, 49):
    print(f"  {k}/49 = {pct(k, S2_ROWS):.2f}%")

# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
W = 7.2
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
    "lines.linewidth": 2,
})
LW = 1.5  # points; matplotlib 1.5 pt = 2 px at 96 dpi


def save(fig, *paths):
    for p in paths:
        fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
        fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


def wrap(s, w):
    return "\n".join(textwrap.wrap(s, w))


Q_ABS = ("“…the 40X WGS HiFi dataset spanned 98% of all VNTRs and "
         "49 (98%) of P-VNTRs with at least 15 spanning reads…”")
Q_RES = ("“…whole genome sequencing covered 92% of P-VNTRs with median spanning "
         "coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 "
         "VNTRs were not well-covered…”")
Q_DIS = ("“…at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. "
         "… 45 VNTRs (out of 48) in whole genome sequencing supported by at least "
         "15 spanning reads.”")


# ---------------------------------------------------------------- candidate A: text panels
def cand_A():
    fig = plt.figure(figsize=(W, 5.7))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 84); ax.axis("off")

    def panel(x, y, w, h, title, quote, wid):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
                                    fc="white", ec=ORANGE, lw=LW))
        ax.text(x + 1.5, y + h - 1.6, title, fontsize=11, fontweight="bold", va="top")
        ax.text(x + 1.5, y + h - 6.2, wrap(quote, wid), fontsize=10, va="top", color=INK,
                linespacing=1.35)

    panel(2, 67, 96, 15, "Abstract", Q_ABS, 82)
    panel(2, 18, 43, 29, "Results 2.5.2", Q_RES, 35)
    panel(55, 18, 43, 29, "Discussion, paragraph 1", Q_DIS, 35)

    arw = dict(arrowstyle="<->", color=BLUE, lw=LW)
    ax.annotate("", xy=(14, 47.8), xytext=(14, 66.2), arrowprops=arw)
    ax.text(16, 57, "49 of 48 = 102.1%, an impossible share\n98% of 48 \u2248 47 loci;"
            " Results implies 44\u201345", va="center", fontsize=10, color=BLUE)
    ax.annotate("", xy=(86, 47.8), xytext=(86, 66.2), arrowprops=arw)
    ax.text(84, 57, "98% vs 92%\n49 vs 45 loci", va="center", ha="right", fontsize=10,
            color=BLUE)
    ax.annotate("", xy=(54.2, 32), xytext=(45.8, 32), arrowprops=arw)
    ax.text(50, 12.5, "Results vs Discussion: 92% \u2260 45/48 = 93.75%;  "
            "48 \u2212 3 named loci = 45", ha="center", fontsize=10, color=BLUE)

    ax.text(2, 6.3, "Set size stated throughout: 48 P-VNTRs = 39 pathogenic + 9 non-pathogenic "
            "= 19 high-RC + 21 low-RC + 8 motif", fontsize=10, color=INK2)
    ax.add_patch(Rectangle((2, 0.8), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
    ax.text(5.5, 2.0, "text as printed in the paper", va="center", fontsize=10, color=INK2)
    ax.plot([42, 46], [2.0, 2.0], color=BLUE, lw=LW)
    ax.text(47, 2.0, "recomputed from the paper\u2019s own numbers", va="center",
            fontsize=10, color=INK2)
    return fig


# ---------------------------------------------------------------- candidate B: count dot plot
ROWS = [  # label, reported count (or None), reported pct, recomputed count, recomputed note
    ("Abstract\n“49 (98%)”", ABS_COUNT, ABS_PCT),
    ("Results 2.5.2\n“92%”", RES_PCT / 100 * N, RES_PCT),
    ("Discussion\n“92%”", RES_PCT / 100 * N, RES_PCT),
    ("Discussion\n“45 (out of 48)”", DIS_COUNT, pct(DIS_COUNT, N)),
]


def cand_B():
    fig, ax = plt.subplots(figsize=(W, 3.6))
    ys = list(range(len(ROWS)))[::-1]
    ax.axvline(N, color=INK2, lw=1, ls=(0, (4, 3)))
    ax.text(N - 0.4, len(ROWS) - 0.45, "all 48 P-VNTRs", color=INK2, fontsize=10, va="bottom",
            ha="right")
    ax.axvspan(N, 52, color=GRID, alpha=0.6, lw=0)
    ax.text(50, -0.75, "more loci\nthan exist", ha="center", va="center", color=INK2,
            fontsize=10)
    for y, (lab, cnt, _) in zip(ys, ROWS):
        ax.plot([implied, cnt], [y, y], color=GRID, lw=3, zorder=1)
        ax.scatter(cnt, y, s=170, color=ORANGE, zorder=2, edgecolor="white", linewidth=0.8)
        ax.scatter(implied, y, s=70, color=BLUE, zorder=3, marker="D", edgecolor="white",
                   linewidth=0.8)
        txt = f"{cnt:.0f}" if float(cnt).is_integer() else f"{cnt:.1f}"
        ax.text(cnt + (0.5 if cnt >= implied else -0.5), y + 0.2, txt, color=ORANGE,
                fontsize=10, ha="left" if cnt >= implied else "right")
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
    ax.set_xlim(0, 52); ax.set_ylim(-1.2, len(ROWS) - 0.2)
    ax.set_xlabel("P-VNTRs spanned by ≥ 15 WGS reads (count out of 48)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.scatter([], [], s=120, color=ORANGE, label="reported (92% shown as 0.92 × 48 = 44.2)")
    ax.scatter([], [], s=90, color=BLUE, marker="D",
               label="48 − 3 named poorly covered loci = 45")
    ax.legend(loc="lower left", frameon=False, fontsize=10, bbox_to_anchor=(0, -0.02))
    return fig


# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
    fig = plt.figure(figsize=(W, 3.4))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 48); ax.axis("off")

    def box(x, y, w, h, s, ec, fs=10):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
                                    fc="white", ec=ec, lw=LW))
        ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs)

    def arrow(x0, y0, x1, y1, c=INK2):
        ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
                    arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))

    box(1, 30, 22, 13, "P-VNTR set\n39 + 9 = 48\n19 + 21 + 8 = 48", BLUE)
    box(28, 30, 28, 13, "minus 3 named as\nnot well-covered\n(CACNA1C, DUX4, ABCA7)", BLUE)
    box(61, 30, 15, 13, "45 loci", BLUE, 11)
    box(83, 30, 16, 13, "45/48\n= 93.75%", BLUE, 11)
    arrow(23.6, 36.5, 27.4, 36.5); arrow(56.6, 36.5, 60.4, 36.5); arrow(76.6, 36.5, 82.4, 36.5)

    ax.text(1, 24.5, "As printed:", fontsize=10.5, fontweight="bold", color=INK)
    box(1, 9, 30, 12, "Abstract “49 (98%)”\n49/48 = 102.1%\n98% of 48 ≈ 47",
        ORANGE)
    box(35, 9, 30, 12, "Results / Discussion “92%”\n0.92 × 48 = 44.2\n"
        "→ 44/48 = 91.67%", ORANGE)
    box(69, 9, 30, 12, "Discussion “45 (out of 48)”\n45/48 = 93.75%\n"
        "(matches the 3 named loci)", ORANGE)
    ax.add_patch(Rectangle((1, 1.5), 2.6, 2.4, fc="white", ec=BLUE, lw=LW))
    ax.text(4.5, 2.7, "derived from the paper’s own counts", va="center", fontsize=10,
            color=INK2)
    ax.add_patch(Rectangle((48, 1.5), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
    ax.text(51.5, 2.7, "value as reported, with its arithmetic", va="center", fontsize=10,
            color=INK2)
    return fig


# ---------------------------------------------------------------- candidate D: percentage dumbbell
def cand_D():
    rows = [
        ("Abstract: “49 (98%)”", ABS_PCT, pct(ABS_COUNT, N), "49/48"),
        ("Results 2.5.2: “92%”", RES_PCT, pct(implied, N), "45/48 (3 loci named)"),
        ("Discussion: “92%” …\n“45 VNTRs (out of 48)”", RES_PCT,
         pct(DIS_COUNT, N), "45/48"),
        ("Targeted (context): “4 (8%)”", TGT_PCT, pct(TGT_COUNT, N), "4/48"),
    ]
    fig, ax = plt.subplots(figsize=(W, 3.4))
    ys = list(range(len(rows)))[::-1]
    ax.axvline(100, color=INK2, lw=1, ls=(0, (4, 3)))
    ax.text(99.3, len(rows) - 0.55, "100%", color=INK2, ha="right", fontsize=10)
    for y, (lab, rep, rec, note) in zip(ys, rows):
        ax.plot([rep, rec], [y, y], color=GRID, lw=3, zorder=1)
        ax.scatter(rep, y, s=90, color=ORANGE, zorder=3)
        ax.scatter(rec, y, s=90, color=BLUE, marker="D", zorder=3)
        lo, hi = min(rep, rec), max(rep, rec)
        ax.text(lo - 2, y, f"{rep:g}%", color=ORANGE, ha="right", va="center", fontsize=10)
        ax.text(hi + 2, y, f"{note} = {rec:.2f}%", color=BLUE, va="center", fontsize=10)
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_xlim(0, 106); ax.set_ylim(-0.5, len(rows) - 0.4); ax.set_xticks(range(0, 101, 20))
    ax.set_xlabel("Share of the 48 P-VNTRs spanned by ≥ 15 reads (%)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.scatter([], [], s=90, color=ORANGE, label="percentage as printed")
    ax.scatter([], [], s=90, color=BLUE, marker="D",
               label="percentage implied by the paper’s own count")
    ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False,
              fontsize=10)
    return fig


# ---------------------------------------------------------------- candidate E: unit grid of 48 loci
def cand_E():
    fig = plt.figure(figsize=(W, 3.0))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 40); ax.axis("off")
    cols, size, gap, x0, y0 = 16, 4.2, 0.9, 2, 30
    for i in range(N):
        r, c = divmod(i, cols)
        x, y = x0 + c * (size + gap), y0 - r * (size + gap)
        poor = i >= N - len(NAMED_POOR)
        ax.add_patch(Rectangle((x, y), size, size, fc="white" if poor else BLUE,
                               ec=BLUE, lw=LW))
        if i == N - 2:
            ax.text(x + size / 2, y - 0.8, ", ".join(NAMED_POOR), fontsize=10,
                    ha="center", va="top", color=INK2)
    # the abstract's 49th locus
    xg = x0 + cols * (size + gap) + 2.5
    yg = y0 - 2 * (size + gap)
    ax.add_patch(Rectangle((xg, yg), size, size, fc="none", ec=ORANGE, lw=LW, ls=(0, (3, 2))))
    ax.text(xg + size + 1.2, yg + size / 2,
            "Abstract: “49 (98%)”\nwould need a 49th locus\n(49/48 = 102.1%)",
            fontsize=10, color=ORANGE, va="center")
    ax.text(x0, 37.5, "The 48 P-VNTRs (39 + 9 = 19 + 21 + 8 = 48), one square per locus",
            fontsize=10.5, fontweight="bold")
    ax.add_patch(Rectangle((x0, 7.2), 2.8, 2.8, fc=BLUE, ec=BLUE, lw=LW))
    ax.text(x0 + 4, 8.6, "45 not named as poorly covered: 45/48 = 93.75%", fontsize=10,
            va="center", color=INK2)
    ax.add_patch(Rectangle((x0, 2.2), 2.8, 2.8, fc="white", ec=BLUE, lw=LW))
    ax.text(x0 + 4, 3.6, "3 named as “not well-covered” in Results 2.5.2", fontsize=10,
            va="center", color=INK2)
    ax.text(60, 8.6, "Results and Discussion: “92%” = 0.92 × 48 = 44.2",
            fontsize=10, color=ORANGE, va="center")
    ax.text(60, 3.6, "orange = as printed; blue = recomputed", fontsize=10, color=INK2,
            va="center")
    return fig


designs = {"A_text_panels": cand_A, "B_count_dots": cand_B, "C_calc_flow": cand_C,
           "D_percent_dumbbell": cand_D, "E_unit_grid": cand_E}
for name, fn in designs.items():
    save(fn(), os.path.join(CAND, name))
# Provisional picks for the write-up (swap after the design review):
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_B(), os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)

Abstract gives Padj = 0.010 for a primary macrolide comparison; Results give 0.19 (Table 2: 0.187)

PLoS Med (2024) · doi:10.1371/journal.pmed.1004386 · PMC11073710

Summary

This MORDOR sub-study compares gut macrolide resistance determinants across four azithromycin treatment histories. For the comparison of 4.5 years of treatment (group 1) with 2.5 years (group 2), the abstract reports "fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010". The Results report the same comparison as "0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19", and Table 2(A) gives its min-P adjusted p-value as 0.187. That value rounds to 0.19, not to 0.010, and a 95% interval that reaches 1.00 is not compatible with an adjusted p of 0.010. The paper's own wording ("no significant increase") and its conclusion are consistent with the correct value, 0.19. But anyone who extracts the abstract's number gets a nominally significant result, apparently a significant decrease, for the comparison that underlies the conclusion that "resistance may plateau".

What the paper reports

Abstract, "Methods and findings" (Padj is printed with a subscript in the original):

However, there was no significant increase in macrolide resistance in communities treated 4.5 years (group 1) compared to just the most recent 2.5 years (group 2) (fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010), or between communities that had been treated for 3 years in the past (group 3) versus just 1 year in the past (group 4) (fold change 1.00, 95% CI 0.78 to 2.35, Padj = 0.52).

Results, "Primary outcome", the same comparison:

Finally, we did not find a statistically significant difference in macrolide resistance determinants between 4.5 years of azithromycin treatment (group 1) compared to 2.5 years of treatment (group 2) (0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19, Wilcoxon rank-sum, min-P adjustment) or 3 years of treatment (group 3) (1.60-fold, 95% CI: 1.00 to 3.08, Padj = 0.06, Wilcoxon rank-sum, min-P adjustment).

Results, "Primary outcome", group 1 vs group 4:

In particular, 4.5 years of azithromycin treatments (group 1) resulted in a 2.23-fold (95% CI: 1.20 to 5.17) increase in macrolide resistance determinants compared to 1 year of azithromycin treatment only during year 3 of the study (Padj = 0.01, Wilcoxon rank-sum (i.e., without covariate adjustment), min-P adjustment).

Table 2(A), rows 1 vs. 2 and 1 vs. 4. The columns are Groups | Treatment | Aminoglycosides | Cationic antimicrobial peptides | Macrolides | Metronidazole | Multi drug resistance | Tetracyclines | Trimethoprim | Beta-lactams:

1 vs. 2 | AA-AA-AA-AA-A vs.PP-PP-AA-AA-A | 0.218 | 0.918 | 0.187 | 0.255 | 0.945 | 0.140 | 0.112 | 0.408

1 vs. 4 | AA-AA-AA-AA-A vs.PP-PP-AA-PP-P | 0.218 | 0.791 | 0.014 | 0.884 | 0.848 | 0.927 | 0.730 | 0.957

Table 2 footnote (A):

The P-values derived from the min-P procedure that were prespecified in the Statistical Analysis Plan are presented in this table.

Methods, "Statistical methods > Primary outcome":

Pairwise Wilcoxon rank-sum tests for macrolide resistance determinants between each of the treatment history groups, accounting for multiple comparisons by means of the Westfall–Young min-P method [20], which are reported as adjusted P-values.

Methods, "Statistical methods > Secondary outcomes", on how fold changes and CIs were obtained:

Fold changes were estimated by transforming normalized read counts using the transformation log(x + epsilon) where epsilon = 0.001 (qualitative results are not sensitive to choice of small epsilon) and performing analysis using the unadjusted confidence interval from the Wilcoxon rank-sum test.

Specifically, the Hodges–Lehmann estimator was computed on the transformed scale, and the inverse transformation was used to yield an approximate ratio.

The statements that rest on this comparison. Abstract, "Conclusions":

resistance may plateau after 2 to 3 years of treatment

Author summary:

Longer periods of treatment with azithromycin were not associated with increased macrolide resistance

Discussion, first paragraph:

However, we were unable to find a difference between communities treated for 4.5 years versus just 2.5 years. Thus resistance in the gut may plateau after several MDAs, rather than continue to increase until it saturates the age-group [22].

What the paper's own numbers show

Method. I matched every macrolide p-value printed in the abstract and Results to its cell in Table 2(A) and checked that each text value is the table value rounded to the printed number of decimals. I then asked whether the abstract's 0.010 could come from any other reading of the same comparison, and listed the Table 2(A) cells closest to 0.010. Finally, I used the printed 95% CI to bound the unadjusted p-value for group 1 vs group 2. The interval is the unadjusted Wilcoxon/Hodges–Lehmann interval on the log scale, and its upper bound is 1.00, the null ratio. By test inversion that corresponds to an unadjusted two-sided p of about 0.05 or more. For a rough numerical check I also approximated the standard error on the log scale from the lower half-width, the upper half-width and the full width of the interval. The community-level data are not published, so the permutation test itself cannot be redone. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Macrolides, 1 vs 2, min-P Padj Table 2(A) 0.187 source value
Macrolides, 1 vs 2, Padj Results 0.19 0.187 to 2 dp = 0.19, consistent
Macrolides, 1 vs 2, Padj Abstract 0.010 does not match: the Table value is 18.7 times larger; 0.187 cannot round to 0.010
Significance at 0.05 of the same comparison Abstract vs Results / Table 2(A) 0.010 vs 0.19 / 0.187 0.010 < 0.05; 0.187 > 0.05. The two classifications differ
Macrolides, 1 vs 3, Padj Results / Table 2(A) 0.06 / 0.063 consistent
Macrolides, 1 vs 4, Padj Results / Table 2(A) 0.01 / 0.014 consistent
Macrolides, 2 vs 3, Padj Results / Table 2(A) 0.003 / 0.003 consistent
Macrolides, 2 vs 4, Padj Results / Table 2(A) < 0.001 / < 0.001 consistent
Macrolides, 3 vs 4, Padj Abstract / Table 2(A) 0.52 / 0.517 consistent; the abstract's only other Padj (3 vs 4) matches Table 2(A)
Table 2(A) cells closest to 0.010 Table 2(A) — Aminoglycosides 2 vs 4 = 0.011; Macrolides 1 vs 4 = 0.014; Macrolides 2 vs 3 = 0.003. Macrolides 1 vs 2 is 0.187
1 vs 2 CI on the log scale Results / Abstract 0.52 to 1.00 / 0.50 to 1.00 −0.654 to 0.000 / −0.693 to 0.000; both reach the null (log ratio 0)
Approximate unadjusted p implied by the Results CI (normal approximation, log scale) derived — 0.18 from the full width; 0.05 to 0.31 depending on which half-width sets the SE
Same, from the Abstract CI derived — 0.21 from the full width; 0.05 to 0.35
One-sided reading of the Table value derived — 0.187 / 2 = 0.094, still not 0.010
1 vs 2 CI lower bound Abstract vs Results 0.50 vs 0.52 differ by 0.02; cannot be resolved without the data
Groups 1+2 vs 3+4 CI lower bound Abstract vs Results 1.5 vs 1.49 1.49 to 1 dp = 1.5, rounding only

Two points make 0.010 impossible for this comparison, not merely inconsistent. First, a Westfall–Young min-P adjusted p-value can never be smaller than the unadjusted p-value for the same hypothesis. Second, the unadjusted p implied by an interval that reaches 1.00 is about 0.05 or more. Even if the printed 1.00 were rounded up from a bound just below 1 (for example 0.995), the unadjusted p would be only slightly below 0.05, far above 0.010. Every other macrolide Padj in the text matches Table 2(A). The abstract's 0.010 is the only exception, and it sits close to the 1 vs 4 value (0.014, reported in the Results as 0.01).

Figure

The abstract and the Results/Table 2(A) give conflicting adjusted p-values for the same comparison

Figure 1. Short excerpts for the macrolide comparison of group 1 (4.5 years of azithromycin) with group 2 (2.5 years). The left panel is the abstract; the right panel is the Results text and the Table 2(A) macrolide cell for row "1 vs. 2". Orange values are printed in the paper. The blue lines below are checks derived from the paper's numbers by recompute.py.txt. The Results value is the Table 2(A) value rounded to two decimals. The abstract's 0.010 is 18.7 times smaller than the table value and cannot be reconciled with a 95% CI whose upper bound is 1.00. Read literally, with a fold change of 0.80, it would mean a significant decrease, which the abstract does not report and the Results deny ("did not find a statistically significant difference").

Reported and recomputed p-values for the group 1 vs group 2 macrolide comparison on one log axis

Figure 2. P-values for the group 1 vs group 2 macrolide comparison on one log-scale axis; the dashed line marks 0.05. Orange circles are the three printed values: the abstract (0.010), the Results (0.19) and Table 2(A) (0.187). Blue diamonds are approximate unadjusted two-sided p-values recomputed from each printed 95% CI (normal approximation on the log scale, SE from the full width); the blue bars span the values obtained when the SE is taken from either half-width instead. A min-P adjusted p cannot be smaller than the unadjusted p, and an upper CI bound of 1.00 puts the unadjusted p at about 0.05 or more (the left end of the blue bars). The Results and Table 2(A) values are above that floor; the abstract's 0.010 is well below it.

Why it matters

The comparison of group 1 with group 2 is part of the prespecified primary outcome. It is the evidence behind the abstract's conclusion that "resistance may plateau after 2 to 3 years of treatment", the Author summary's statement that "Longer periods of treatment with azithromycin were not associated with increased macrolide resistance", and the corresponding sentence of the Discussion. With the correct value (Padj = 0.19, Table 2(A) 0.187) these statements are supported as written. The analysis, the body text and the table are consistent with each other, and the paper's conclusions do not change.

The error matters because it sits in the abstract, which is the part most often read on its own, indexed and extracted into reviews and meta-analyses. Read literally, the abstract pairs a fold change of 0.80 and an upper bound of 1.00 with Padj = 0.010. That reads as a statistically significant decrease in macrolide resistance with longer treatment, which the data do not show. The abstract does not report a significant decrease, and the Results deny any significant difference ("did not find a statistically significant difference"). A reader who takes the number at face value would put this comparison in the wrong significance class. The 0.50 vs 0.52 mismatch in the CI lower bound is a separate, minor inconsistency and does not affect any interpretation.

Explanations considered

Status

Suggested correction

  1. In the abstract ("Methods and findings"), replace "Padj = 0.010" for group 1 vs group 2 with the Table 2(A) value, "Padj = 0.19" (0.187), after confirming it against the original analysis output.
  2. Reconcile the lower confidence limit for the same comparison: the abstract gives 0.50 and the Results give 0.52. Use whichever the analysis output supports in both places.
  3. Optionally, harmonize "1.5" (abstract) with "1.49" (Results) for the lower confidence limit of the groups 1+2 vs 3+4 comparison.

No change to the Results, Table 2, the Author summary, the Discussion or the conclusions appears to be needed.

Recomputation script (python recompute.py.txt)
"""Recompute the group 1 vs group 2 macrolide comparison of PMC11073710
(PLoS Med 2024, doi:10.1371/journal.pmed.1004386).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

All inputs are copied from the article as archived in PubMed Central (identical in the
publisher's PDF and HTML): the Abstract, the Results ("Primary outcome") and Table 2(A).
No raw data are used; the community-level data needed to redo the min-P permutation
test are not published.
"""
import os

import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs (from the paper)
# Table 2(A): Westfall-Young min-P adjusted pairwise Wilcoxon p-values (100,000 permutations).
CLASSES = ["Aminoglycosides", "Cationic antimicrobial peptides", "Macrolides", "Metronidazole",
           "Multi drug resistance", "Tetracyclines", "Trimethoprim", "Beta-lactams"]
TABLE2A = {  # "<0.001" stored as 0.001 with a flag
    "1 vs. 2": [0.218, 0.918, 0.187, 0.255, 0.945, 0.140, 0.112, 0.408],
    "1 vs. 3": [0.379, 0.935, 0.063, 0.957, 0.291, 0.908, 0.690, 0.957],
    "1 vs. 4": [0.218, 0.791, 0.014, 0.884, 0.848, 0.927, 0.730, 0.957],
    "2 vs. 3": [0.055, 0.980, 0.003, 0.689, 0.530, 0.157, 0.690, 0.515],
    "2 vs. 4": [0.011, 0.973, "<0.001", 0.840, 0.945, 0.193, 0.611, 0.508],
    "3 vs. 4": [0.533, 0.980, 0.517, 0.957, 0.652, 0.908, 0.927, 0.959],
}
MAC = CLASSES.index("Macrolides")

# Macrolide comparisons as printed in the text: (fold change, CI low, CI high, Padj as printed)
RESULTS = {  # Results, "Primary outcome"
    "1 vs. 4": (2.23, 1.20, 5.17, "0.01"),
    "2 vs. 3": (2.14, 1.38, 3.64, "0.003"),
    "2 vs. 4": (3.12, 1.78, 7.31, "<0.001"),
    "1 vs. 2": (0.80, 0.52, 1.00, "0.19"),
    "1 vs. 3": (1.60, 1.00, 3.08, "0.06"),
}
ABSTRACT = {  # Abstract, "Methods and findings"
    "1 vs. 2": (0.80, 0.50, 1.00, "0.010"),
    "3 vs. 4": (1.00, 0.78, 2.35, "0.52"),
}
ALPHA = 0.05

# ------------------------------------------------------------------ 1. text vs table
print("1. Macrolide comparisons: text vs Table 2(A) (min-P adjusted P)")
for pair in TABLE2A:
    t = TABLE2A[pair][MAC]
    r = RESULTS.get(pair, (None,) * 4)[3]
    a = ABSTRACT.get(pair, (None,) * 4)[3]
    if isinstance(t, str):
        print(f"   {pair}: Table {t}; Results {r}; Abstract {a}")
        continue
    parts = [f"   {pair}: Table {t:.3f}"]
    for lab, v in (("Results", r), ("Abstract", a)):
        if v is None:
            continue
        dec = len(v.split(".")[1])  # decimals as printed in the text
        rounded = round(t, dec)
        ok = abs(rounded - float(v)) < 1e-9
        parts.append(f"{lab} {v} (Table rounded to {dec} dp = {rounded:.{dec}f}: "
                     f"{'consistent' if ok else 'DOES NOT MATCH'})")
    print("; ".join(parts))

t12 = TABLE2A["1 vs. 2"][MAC]
a12 = float(ABSTRACT["1 vs. 2"][3])
r12 = float(RESULTS["1 vs. 2"][3])
print(f"   Abstract 1 vs. 2 Padj {a12:.3f} vs Table {t12:.3f}: ratio Table/Abstract = {t12 / a12:.1f}")
print(f"   Significant at {ALPHA}?  Abstract {a12} -> {a12 < ALPHA};  Results {r12} -> {r12 < ALPHA};"
      f"  Table {t12} -> {t12 < ALPHA}")

# ------------------------------------------------------------------ 2. nearest Table 2(A) entries to 0.010
print("\n2. Table 2(A) entries closest to the abstract's 0.010")
cells = []
for pair, row in TABLE2A.items():
    for cls, v in zip(CLASSES, row):
        if not isinstance(v, str):
            cells.append((abs(v - a12), pair, cls, v))
for d, pair, cls, v in sorted(cells)[:3]:
    print(f"   {cls}, {pair}: {v:.3f} (|difference| = {d:.3f})")
print(f"   Macrolides 1 vs. 2 itself: {t12:.3f} (|difference| = {abs(t12 - a12):.3f})")

# ------------------------------------------------------------------ 3. CI vs p-value
print("\n3. What the 95% CI implies about the unadjusted p-value (fold change 0.80)")
est = RESULTS["1 vs. 2"][0]
print(f"   log(0.80) = {np.log(est):.3f}; log(upper bound 1.00) = {np.log(1.00):.3f}")
print("   The CI is the (unadjusted) Hodges-Lehmann/Wilcoxon interval. An interval from test inversion")
print("   that reaches the null (ratio 1, log 0) corresponds to an unadjusted two-sided p of about 0.05")
print("   or more; an unadjusted p of 0.010 would require an upper bound clearly below 1.")
APPROX = {}
for src, (e, lo, hi, _) in (("Results CI 0.52-1.00", RESULTS["1 vs. 2"]),
                            ("Abstract CI 0.50-1.00", ABSTRACT["1 vs. 2"])):
    res = {}
    for lab, se in (("lower half-width", (np.log(e) - np.log(lo)) / 1.96),
                    ("upper half-width", (np.log(hi) - np.log(e)) / 1.96),
                    ("full width", (np.log(hi) - np.log(lo)) / (2 * 1.96))):
        z = abs(np.log(e)) / se
        p = 2 * norm.sf(z)
        res[lab] = p
        print(f"   {src}, SE from {lab}: SE = {se:.3f}, z = {z:.2f}, approx. unadjusted p = {p:.3f}")
    APPROX[src] = res
lo_p = min(min(r.values()) for r in APPROX.values())
hi_p = max(max(r.values()) for r in APPROX.values())
print(f"   Range of normal-approximation p-values: {lo_p:.3f} to {hi_p:.3f}")
print("   A Westfall-Young min-P adjusted p is never smaller than the unadjusted p for the same")
print(f"   comparison, so Padj = {a12:.3f} is below every value compatible with the printed CI.")
print(f"   Halving the Table value (one-sided reading): {t12 / 2:.4f}, still not 0.010")

# ------------------------------------------------------------------ 4. minor CI mismatches
print("\n4. Other abstract-vs-Results numbers")
print(f"   1 vs. 2 CI lower bound: Abstract 0.50 vs Results 0.52, difference {0.52 - 0.50:.2f}")
print(f"   Groups 1+2 vs 3+4 CI lower bound: Abstract 1.5 vs Results 1.49 -> 1.49 to 1 dp = {round(1.49, 1)}")
print(f"   1 vs. 2 CI on log scale (Results): {np.log(0.52):.3f} to {np.log(1.00):.3f}; "
      f"(Abstract): {np.log(0.50):.3f} to 0.000")

# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
                     "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
                     "svg.fonttype": "none"})


def save(fig, name):
    fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
    fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
    plt.close(fig)


# Figure 1: abstract vs Results / Table 2 text panels
fig = plt.figure(figsize=(7.2, 5.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(-1.0, 4.3)
ax.axis("off")
ax.set_facecolor(BG)

ax.text(0.2, 4.05, "Macrolide resistance determinants, group 1 (4.5 y azithromycin) vs group 2 (2.5 y)",
        fontsize=10.5, weight="bold", va="center")

for x0, title in ((0.2, "Abstract (Methods and findings)"), (3.75, "Results and Table 2(A)")):
    ax.add_patch(FancyBboxPatch((x0, 1.35), 3.25, 2.35, boxstyle="round,pad=0,rounding_size=0.08",
                                fc="white", ec="#c9c8c3", lw=1))
    ax.text(x0 + 0.15, 3.5, title, fontsize=10, weight="bold", color=INK2, va="center")

L = 0.35
ax.text(L, 3.12, "\u201cno significant increase in", fontsize=10, va="center")
ax.text(L, 2.87, "macrolide resistance\u201d", fontsize=10, va="center")
ax.text(L, 2.5, "fold change 0.80,", fontsize=10, va="center")
ax.text(L, 2.25, "95% CI 0.50 to 1.00,", fontsize=10, va="center")
ax.text(L, 1.75, f"Padj = {a12:.3f}", fontsize=13, weight="bold", color=ORANGE, va="center")

R = 3.9
ax.text(R, 3.12, "\u201cdid not find a statistically", fontsize=10, va="center")
ax.text(R, 2.87, "significant difference\u201d", fontsize=10, va="center")
ax.text(R, 2.5, "0.80-fold, 95% CI: 0.52 to 1.00,", fontsize=10, va="center")
ax.text(R, 2.25, f"Padj = {r12:.2f}", fontsize=13, weight="bold", color=ORANGE, va="center")
ax.text(R, 1.75, f"Table 2(A), Macrolides, 1 vs. 2: {t12:.3f}", fontsize=10.5, weight="bold",
        color=ORANGE, va="center")

ax.add_patch(FancyArrowPatch((1.75, 1.75), (3.85, 1.75), arrowstyle="<|-|>", mutation_scale=12,
                             color=INK2, lw=1.5))
ax.text(2.6, 1.93, "conflicting Padj", fontsize=10, color=INK2, ha="center",
        va="center", style="italic")

ax.text(0.2, 0.98, "Same comparison, same Padj definition (Westfall\u2013Young min-P, Wilcoxon rank-sum):",
        fontsize=10, color=INK, va="center")
lines = [
    f"round({t12:.3f}, 2) = {round(t12, 2):.2f}: matches the Results; it cannot give {a12:.3f} "
    f"(Table value is {t12 / a12:.1f}\u00d7 larger)",
    f"Read literally (fold change 0.80), {a12:.3f} would mean a significant decrease; the Results",
    "deny one: \u201cdid not find a statistically significant difference\u201d",
    "An upper CI bound of 1.00 implies an unadjusted p of about 0.05 or more, and a",
    "min-P adjusted p cannot be smaller than the unadjusted p for the same comparison",
    f"Nearest macrolide entry in Table 2(A): 1 vs. 4 = {TABLE2A['1 vs. 4'][MAC]:.3f} "
    f"(Results: Padj = 0.01)",
]
for i, s in enumerate(lines):
    ax.text(0.35, 0.72 - 0.25 * i, s, fontsize=10, color=BLUE, va="center")
ax.text(7.0, -0.95, "orange = as printed in the paper;  blue = checks derived from the paper's numbers",
        fontsize=10, color=INK2, ha="right", va="bottom")
save(fig, "fig1")

# Figure 2: p-values for the same comparison on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
rows = [
    ("Abstract: Padj", a12, None, ORANGE, "o", f"{a12:.3f}"),
    ("Results: Padj", r12, None, ORANGE, "o", f"{r12:.2f}"),
    ("Table 2(A): min-P Padj", t12, None, ORANGE, "o", f"{t12:.3f}"),
    ("Unadjusted p implied by\nResults CI 0.52–1.00", APPROX["Results CI 0.52-1.00"]["full width"],
     (min(APPROX["Results CI 0.52-1.00"].values()), max(APPROX["Results CI 0.52-1.00"].values())),
     BLUE, "D", None),
    ("Unadjusted p implied by\nAbstract CI 0.50–1.00", APPROX["Abstract CI 0.50-1.00"]["full width"],
     (min(APPROX["Abstract CI 0.50-1.00"].values()), max(APPROX["Abstract CI 0.50-1.00"].values())),
     BLUE, "D", None),
]
ys = np.arange(len(rows))[::-1]
for y, (lab, v, rng, c, m, txt) in zip(ys, rows):
    if rng:
        ax.plot(rng, [y, y], color=c, lw=2, solid_capstyle="round", zorder=2)
        txt = f"{v:.2f} (range {rng[0]:.2f}–{rng[1]:.2f})"
    ax.plot(v, y, marker=m, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="none")
    if rng:
        ax.text(v, y + 0.32, txt, ha="center", va="center", fontsize=10, color=INK)
    else:
        ax.text(v * 1.18, y, txt, va="center", fontsize=10, color=INK)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
ax.text(0.05 * 0.93, ys[0] + 0.55, "0.05", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.004, 1.0)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlabel("p-value, macrolides, group 1 vs group 2 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=9, label="as printed in the paper")
ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from the printed CI (normal approx.)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")

The abstract says RNF213 loss lowers IFN-β-induced ISG expression, but Fig 2A shows no difference

PLoS Pathog (2025) · doi:10.1371/journal.ppat.1013333 · PMC12251097

Summary

The abstract states that RNF213 deficiency "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection." For the IFN-β condition the Results say the opposite: the IFN-β-induced increases in Cxcl9 and Cxcl10 "were comparable between WT and Rnf213–/– MSFs," and all 8 IFN-β comparisons of Cxcl9/Cxcl10 in Fig 2A (two cell types, two genes, two time points) are labelled "ns." Bar heights read off the published Fig 2A give a knockout-to-wild-type ratio of about 0.91–1.31 (geometric mean 1.04) under IFN-β, with the knockout numerically higher in half of the comparisons, versus about 0.30–0.63 (geometric mean 0.44, all 16 significant) under A. baumannii or LPS. This matters because the paper uses the intact IFN-β response as its evidence that RNF213 acts upstream of the IFN-α/β receptor (IFNAR), so the abstract sentence points to the opposite location in the pathway from the one the paper argues for.

What the paper reports

In the abstract:

RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection.

— Abstract

In the body, the same experiment (Fig 2A) is described as follows:

The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A).

However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment were comparable between WT and Rnf213–/– MSFs (Fig 2A).

Taken together, our results indicate that RNF213 positively regulates IFN-I signaling possibly by mediating events upstream of IFNAR binding.

— Results, section "IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs"

RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR (Figs 2D, 2E, S3C and S3D).

— Results, section "RNF213 targets TRAF3", first sentence

The cited figure is described in its legend as:

qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments).

— Fig 2 legend, panel A

Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC12251097). Each one occurs exactly once; recompute.py.txt reruns that check.

What the paper's own numbers show

Method. The claim is about direction (a decrease versus no change), so the check has two parts.

  1. The significance label printed above each wild-type/knockout pair in Fig 2A was transcribed exactly, for every stimulated Cxcl9 and Cxcl10 comparison (3 stimuli × 2 cell types × 2 genes × 2 time points = 24). The legend states these labels come from two-sided Student's t-tests without multiple-comparison correction.
  2. To show the size of each difference, the article's Data Availability statement reports no numeric source data, so the bar heights (gene/Hprt) were measured on the publisher's large Fig 2 image. For each panel a straight line was fit through the printed y-axis tick marks (largest residual 0.2% of the axis maximum), and the top edge of each outlined bar was read against it. Where the replicate dots cover a bar's top, the value is bracketed between the top of the bar outline and the middle of the dot cluster; that range is carried through to the ratio. These readings are approximate. The conclusion does not rest on them: the printed "ns" labels and the Results text already state there is no difference under IFN-β.

recompute.py.txt prints every value below and redraws both figures.

Quantity Location Reported Recomputed from the paper's own data
Effect of RNF213 deficiency on ISG expression after IFN-β Abstract "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation" Not supported: no IFN-β ISG comparison in the paper shows a decrease (rows below)
Same effect, as described in the body Results, Fig 2A paragraph "comparable between WT and Rnf213–/– MSFs" Agrees with Fig 2A
Cxcl9/Cxcl10 after IFN-β, MSFs + BMECs, 3 h and 6 h (8 comparisons) Fig 2A, IFN-β row All 8 labelled "ns" Knockout ÷ WT ≈ 0.91–1.31; geometric mean 1.04; knockout numerically higher in 4 of 8
MSF Cxcl10, 6 h, IFN-β Fig 2A "ns" Knockout ÷ WT ≈ 1.31 (≈ 0.61 vs ≈ 0.47; knockout higher)
Cxcl9/Cxcl10 after A. baumannii, MSFs + BMECs, 3 h and 6 h (8 comparisons) Fig 2A, A. baumannii row All *, ** or *** Knockout ÷ WT ≈ 0.35–0.63; geometric mean 0.44
Cxcl9/Cxcl10 after LPS, MSFs + BMECs, 6 h and 12 h (8 comparisons) Fig 2A, LPS row All ** or *** Knockout ÷ WT ≈ 0.30–0.57; geometric mean 0.44
A. baumannii and LPS together (16 comparisons) Fig 2A 16 of 16 significant Knockout ÷ WT ≈ 0.30–0.63 (largest upper bound of a reading range 0.72); geometric mean 0.44
IFN-I production / TBK1 phosphorylation after IFN-β Results, "RNF213 targets TRAF3" "but not in response to IFN-β treatment" No panel cited there (Figs 2D, 2E, S3C–D) includes IFN-β-stimulated cells; their legends list only untreated, A. baumannii and LPS conditions

So under bacterial infection or LPS, knockout cells induce roughly 40–45% as much Cxcl9/Cxcl10 as wild-type cells (geometric mean 0.44 for both stimuli), and every one of the 16 comparisons is marked significant. Under IFN-β the two genotypes are indistinguishable: the ratios scatter on both sides of 1 (geometric mean 1.04; knockout numerically higher in 4 of 8), and all 8 are marked "ns." The paper's body reports exactly this pattern. Only the abstract sentence departs from it.

Figure

Two text panels, "Abstract" and "Results and Fig 2A", holding the conflicting excerpts, joined by a connector labelled "conflict / IFN-β clause only"

Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the IFN-β ISG clause highlighted. Right (blue): the Fig 2A Results sentence, the first sentence of "RNF213 targets TRAF3" (which concerns TBK1 phosphorylation and IFN-I production and cites Figs 2D, 2E, S3C–D), and a count of the significance labels in the IFN-β row of Fig 2A. The abstract's ISG clause and the Fig 2A sentence describe the same experiment and disagree only on the IFN-β condition; on A. baumannii and LPS the abstract and Results agree.

For each of 24 Fig 2A comparisons, the knockout-to-wild-type ratio on a log axis, grouped by stimulus, with the printed significance label at right

Figure 2. Size of the knockout effect for each stimulus in Fig 2A. Each filled blue dot is the ratio Rnf213–/– ÷ WT of mean Cxcl9 or Cxcl10 mRNA (gene/Hprt) for one cell type, gene and time point, from bar heights measured on the published Fig 2A; the short blue line is the range left by the measurement where replicate dots cover a bar top. Open blue diamonds are the per-stimulus geometric means. The orange label at right is the significance label printed in Fig 2A for that pair. The x-axis is logarithmic and the vertical line at 1 marks no difference. The 8 IFN-β ratios cluster around 1 (geometric mean 1.04) and all are "ns"; the 16 A. baumannii/LPS ratios sit between about 0.30 and 0.63 (geometric mean 0.44 for each stimulus) and all are *, ** or ***. The two clusters are far apart relative to the reading uncertainty.

Why it matters

Explanations considered

  1. "In response to IFN-β stimulation" modifies only "IFN-I production," not the ISG clause. This does not resolve the conflict. The paper never measures IFN-I production after IFN-β stimulation (the ELISA panels Fig 2B–C cover only A. baumannii and LPS), and IFN-β is itself a type I interferon, so "IFN-I production in response to IFN-β" is not a coherent separate readout. Under the parsing that attaches the phrase to ISG expression, it contradicts Fig 2A; under the other parsing, the Results already say IFN-I signaling is unaffected by IFN-β. Neither reading matches the body.
  2. The abstract refers to a different cell type or experiment. The only experiment in which Rnf213–/– cells are stimulated with IFN-β and ISGs are measured is Fig 2A, which covers both MSFs and BMECs; all 8 Cxcl9/Cxcl10 comparisons are "ns." Fig 1B–C uses IFN-β to induce Rnf213 in WT versus Ifnar–/– cells, a different genotype contrast. Fig 4C uses IFN-β pretreatment of BMECs, and the Results/Discussion describe the outcome as a rescue ("reversed by treatment with IFN-β" / "rescued by treatment with IFN-β"), which is the opposite of a reduced response. None of the eight supplementary figure legends reports IFN-β-stimulated ISG data in knockout versus WT cells.
  3. The Rnf213 transcript itself. In the IFN-β row of Fig 2A the Rnf213 panels are "***" because the gene is deleted in the knockout. Rnf213 is IFN-inducible (Fig 1B–C), but it is the targeted gene, not an ISG readout of the knockout's downstream signalling capacity, and neither the abstract nor the Results presents it as one.
  4. Low statistical power (absence of evidence). One could argue that "ns" with n = 4 technical replicates does not exclude a small decrease. But the abstract asserts a decrease, which "ns" does not support either way, and the point estimates show no tendency toward one: the geometric-mean ratio is 1.04 and the knockout is numerically higher in half the comparisons. With the same design the same figure detects 2- to 3-fold decreases under A. baumannii and LPS.
  5. Conversion or typesetting artifact. Ruled out. The same wording appears in the PMC-archived text, the Europe PMC full-text XML and the publisher's HTML version of record (labelled "Version 2").
  6. Already corrected or discussed. No correction, erratum, expression of concern or retraction was found (see Status).

None of these accounts for the conflict. The most plausible benign reading is that the abstract meant "LPS" where it wrote "IFN-β," since the Results support a decrease under LPS. That is a conjecture about wording and does not change the fact that the abstract as published contradicts the Results.

Status

As of 2026-10-23:

Suggested correction

Restrict the IFN-β clause in the abstract to what the data show. For example:

RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to Acinetobacter baumannii infection and LPS stimulation, whereas ISG induction by IFN-β is unaffected.

Alternatively, delete "IFN-β stimulation and" from the sentence. Separately, the first sentence of "RNF213 targets TRAF3" states that TBK1 phosphorylation and IFN-I production were unaffected after IFN-β treatment, yet none of the panels it cites (Figs 2D, 2E, S3C, S3D) includes IFN-β-treated samples; Fig 2A holds only Cxcl9/Cxcl10/Rnf213 mRNA after IFN-β, not TBK1 or IFN-I readouts. That sentence could restrict its IFN-β clause to ISG induction and cite Fig 2A (for example, "…but did not affect Cxcl9 and Cxcl10 induction by IFN-β treatment (Fig 2A)"), or drop the IFN-β clause for the TBK1/IFN-I readouts, for which no IFN-β-treated samples are shown.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12251097 (PLoS Pathog 2025, doi:10.1371/journal.ppat.1013333)
# Abstract vs Results / Fig 2A: does RNF213 deficiency lower ISG expression after IFN-beta?
#
# Run:  python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs numpy and matplotlib.
#
# Inputs
#  * Quotes: copied verbatim from the article as archived in PubMed Central (PMC12251097).
#    When run inside the /home/ubuntu/papers repo, the script re-checks each quote against
#    the archived full text with src/show.py (skipped elsewhere).
#  * Fig 2A: the article gives no numeric source data (Data Availability: all data are in
#    the manuscript and its supporting files). The pixel coordinates below were measured on
#    the publisher's large Fig 2 image
#      https://journals.plos.org/plospathogens/article/figure/image?size=large&id=10.1371/journal.ppat.1013333.g002
#    (2232 x 2576 px), by locating the y-axis tick marks and the top edge of each outlined
#    bar (blue = WT, red = Rnf213-/-). Where the replicate dots hide a bar's top edge, the
#    top is bracketed between the middle of the dot cluster and the highest visible pixel of
#    the bar outline; otherwise +/-1 px. Tick values and significance labels are transcribed
#    exactly as printed. All 24 stimulated Cxcl9/Cxcl10 comparisons are included (3 stimuli
#    x 2 cell types x 2 genes x 2 time points); untreated (0 h) bars are not used.
#    The finding does not rest on these measurements: the printed 'ns' labels and the
#    Results text say the same thing. The measurements only show the size of each difference.

import math, os, statistics, subprocess

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ quotes
QUOTES = {
    "abstract": ("Abstract",
        "RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the "
        "expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation "
        "and Acinetobacter baumannii infection."),
    "results_down": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
        "The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection "
        "and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A)."),
    "results_ifnb": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
        "However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment "
        "were comparable between WT and Rnf213–/– MSFs (Fig 2A)."),
    "results_bmec": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
        "Similar patterns of A. baumannii infection- and LPS stimulation-induced Cxcl9 and "
        "Cxcl10 expression were observed in BMECs (Fig 2A)."),
    "results_upstream": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
        "Taken together, our results indicate that RNF213 positively regulates IFN-I signaling "
        "possibly by mediating events upstream of IFNAR binding."),
    "results_traf3": ("Results, 'RNF213 targets TRAF3', first sentence",
        "RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS "
        "stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting "
        "that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR "
        "(Figs 2D, 2E, S3C and S3D)."),
    "fig2_legend": ("Fig 2 legend, panel A",
        "qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without "
        "treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and "
        "IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments)."),
    "fig4c": ("Results, 'RNF213 modulates blood–brain barrier integrity via IFN-I signaling'",
        "Notably, the reduced expression of Claudin-5, Occludin, and Pecam1 in Rnf213–/– BMECs "
        "was reversed by treatment with IFN-β (Fig 4C)."),
}


def check_quotes():
    repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        print("Quote check: src/show.py not available here; skipped.")
        return
    try:
        full = subprocess.run([py, "src/show.py", "PMC12251097", "--full"], cwd=repo,
                              capture_output=True, text=True, timeout=300).stdout
    except Exception as e:
        print(f"Quote check: could not load archived text ({e}); skipped.")
        return
    norm = lambda s: " ".join(s.split())
    full_n = norm(full)
    print("Quote check against the PMC-archived full text (occurrences):")
    for key, (loc, q) in QUOTES.items():
        print(f"  {key:17s} {full_n.count(norm(q))}   ({loc})")


# ------------------------------------------------------------------ Fig 2A pixel data
# Panel: (stimulus, cell, gene) -> y-axis ticks [(pixel y, printed value)], and per time point:
#   (time, WT top-edge pixel range, KO top-edge pixel range, printed significance label)
PANELS = [
    ("A. baumannii", "MSF", "Cxcl9", [(150.5, .18), (217.5, .12), (285.0, .06), (352.5, 0)],
        [("3 h", (296, 296), (320, 320), "***"), ("6 h", (195, 195), (297, 297), "***")]),
    ("A. baumannii", "MSF", "Cxcl10", [(153.5, 4), (203.5, 3), (253.5, 2), (303.5, 1), (353.5, 0)],
        [("3 h", (295, 303), (319, 319), "***"), ("6 h", (190, 190), (277, 277), "***")]),
    ("A. baumannii", "BMEC", "Cxcl9", [(152.5, .18), (220.5, .12), (287.5, .06), (355.0, 0)],
        [("3 h", (262, 262), (321.5, 324), "**"), ("6 h", (205, 205), (302.5, 304), "***")]),
    ("A. baumannii", "BMEC", "Cxcl10", [(150.5, 1.5), (218.5, 1.0), (285.5, .5), (353.0, 0)],
        [("3 h", (271, 271), (307, 307), "*"), ("6 h", (177, 177), (292, 292), "***")]),
    ("LPS", "MSF", "Cxcl9", [(435.5, .20), (486.0, .15), (536.5, .10), (587.0, .05), (637.5, 0)],
        [("6 h", (553, 553), (586.5, 593), "**"), ("12 h", (492, 492), (594, 594), "**")]),
    ("LPS", "MSF", "Cxcl10", [(435.5, 4), (485.5, 3), (535.5, 2), (585.5, 1), (635.0, 0)],
        [("6 h", (498, 498), (567, 567), "**"), ("12 h", (564, 564), (614, 614), "**")]),
    ("LPS", "BMEC", "Cxcl9", [(434.0, .24), (501.5, .16), (569.0, .08), (636.5, 0)],
        [("6 h", (562, 562), (595, 596), "**"), ("12 h", (480, 480), (564, 564), "***")]),
    ("LPS", "BMEC", "Cxcl10", [(434.5, 1.8), (501.5, 1.2), (568.5, .6), (636.0, 0)],
        [("6 h", (548.5, 558), (595.5, 596), "**"), ("12 h", (485, 485), (568, 568), "**")]),
    ("IFN-β", "MSF", "Cxcl9", [(738.5, .04), (789.0, .03), (839.5, .02), (890.0, .01), (940.5, 0)],
        [("3 h", (843, 844), (826, 826), "ns"), ("6 h", (782, 785), (774, 774), "ns")]),
    ("IFN-β", "MSF", "Cxcl10", [(739.5, 2), (790.5, 1.5), (840.5, 1), (891.5, .5), (941.5, 0)],
        [("3 h", (763.5, 767), (779, 779), "ns"), ("6 h", (894, 895), (878, 882), "ns")]),
    ("IFN-β", "BMEC", "Cxcl9", [(739.5, .06), (806.5, .04), (874.5, .02), (941.5, 0)],
        [("3 h", (865, 865), (853, 861), "ns"), ("6 h", (764.5, 769), (777, 777), "ns")]),
    ("IFN-β", "BMEC", "Cxcl10", [(734.5, 2.4), (803.5, 1.6), (871.5, .8), (939.5, 0)],
        [("3 h", (878, 878), (879, 879), "ns"), ("6 h", (767, 767), (782, 782), "ns")]),
]
PAD = 1.0  # px added on each side of every top-edge range (line width / anti-aliasing)


def calib(ticks):
    """Least-squares line value = a + b * pixel_y through the printed tick marks."""
    ys = [t[0] for t in ticks]; vs = [t[1] for t in ticks]
    my, mv = statistics.fmean(ys), statistics.fmean(vs)
    b = sum((y - my) * (v - mv) for y, v in zip(ys, vs)) / sum((y - my) ** 2 for y in ys)
    a = mv - b * my
    resid = max(abs(a + b * y - v) for y, v in zip(ys, vs))
    return a, b, resid


def measure():
    out, worst = [], 0.0
    for stim, cell, gene, ticks, bars in PANELS:
        a, b, resid = calib(ticks)
        worst = max(worst, resid / max(t[1] for t in ticks))
        val = lambda y: a + b * y          # b < 0: smaller pixel y = taller bar
        for t, (w0, w1), (k0, k1), lab in bars:
            wt_mid, ko_mid = val((w0 + w1) / 2), val((k0 + k1) / 2)
            wt_hi, wt_lo = val(w0 - PAD), val(w1 + PAD)
            ko_hi, ko_lo = val(k0 - PAD), val(k1 + PAD)
            out.append(dict(stim=stim, cell=cell, gene=gene, time=t, label=lab,
                            wt=wt_mid, ko=ko_mid, r=ko_mid / wt_mid,
                            r_lo=ko_lo / wt_hi, r_hi=ko_hi / wt_lo))
    print(f"\nAxis calibration: largest tick residual = {worst:.2%} of axis maximum")
    return out


def summarise(rows, stim):
    sel = [d for d in rows if d["stim"] == stim]
    print(f"\n{stim}: {len(sel)} Cxcl9/Cxcl10 comparisons in Fig 2A (Rnf213-/- / WT)")
    for d in sel:
        print(f"  {d['cell']:4s} {d['gene']:6s} {d['time']:>4s}  WT {d['wt']:.3g}  KO {d['ko']:.3g}  "
              f"ratio {d['r']:.2f} [{d['r_lo']:.2f}-{d['r_hi']:.2f}]  printed: {d['label']}")
    rs = [d["r"] for d in sel]
    gm = statistics.geometric_mean(rs)
    n_ns = sum(d["label"] == "ns" for d in sel)
    n_above = sum(r > 1 for r in rs)
    n_cover1 = sum(d["r_lo"] <= 1 <= d["r_hi"] for d in sel)
    print(f"  ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean {gm:.2f}; "
          f"printed 'ns' {n_ns}/{len(sel)}; KO above WT in {n_above}/{len(sel)}; "
          f"measurement interval includes 1 in {n_cover1}/{len(sel)}")
    return sel, gm


# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)
    print(f"wrote {stem}.svg and {stem}.png")


# ------------------------------------------------------------------ figure 1
def fig1(n_ns, n_tot):
    fig = plt.figure(figsize=(7.2, 4.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")

    def panel(x, w, title, sub, body, accent):
        ax.add_patch(FancyBboxPatch((x, 0.92), w, 3.55, boxstyle="round,pad=0.02,rounding_size=0.08",
                                    fc="white", ec=accent, lw=1.5))
        ax.text(x + 0.14, 4.33, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
        ax.text(x + 0.14, 4.03, sub, fontsize=10, va="top", color=accent)
        yy = 3.68
        for line, style in body:
            ax.text(x + 0.14, yy, line, fontsize=10, va="top",
                    color={"n": TXT, "b": accent, "s": TXT2}[style],
                    fontweight="bold" if style == "b" else "normal",
                    fontstyle="italic" if style == "s" else "normal")
            yy -= 0.235

    left = [("“RNF213 deficiency impairs type I", "n"),
            ("interferon (IFN-I) production and", "n"),
            ("decreases the expression of", "b"),
            ("interferon-stimulated genes (ISGs)", "b"),
            ("in response to IFN-β stimulation", "b"),
            ("and Acinetobacter baumannii", "n"),
            ("infection.”", "n"),
            ("", "n"), ("", "n"), ("", "n"),
            ("States: ISGs lower in Rnf213–/–", "s"),
            ("cells after IFN-β.", "s")]
    right = [("Fig 2A paragraph:", "s"),
             ("“…Cxcl9 and Cxcl10 induced by IFN-β", "n"),
             ("treatment were comparable", "b"),
             ("between WT and Rnf213–/– MSFs …”", "b"),
             ("“RNF213 targets TRAF3”:", "s"),
             ("“…inhibited TBK1 phosphorylation and", "n"),
             ("IFN-I production … but not in", "b"),
             ("response to IFN-β treatment …”", "b"),
             ("", "n"),
             ("Fig 2A, IFN-β row (MSFs and BMECs):", "s"),
             (f"{n_ns} of {n_tot} Cxcl9/Cxcl10 comparisons", "n"),
             ("printed “ns”.", "n")]
    panel(0.15, 3.05, "Abstract", "reported claim", left, ORANGE)
    panel(4.0, 3.05, "Results and Fig 2A", "the paper's own text and labels", right, BLUE)

    ax.add_patch(FancyArrowPatch((3.24, 3.0), (3.96, 3.0), arrowstyle="<|-|>", mutation_scale=14,
                                 lw=2, color=TXT))
    ax.text(3.6, 3.3, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
    ax.text(3.6, 2.73, "IFN-β\nclause\nonly", ha="center", va="top", fontsize=10, color=TXT2)

    ax.text(0.15, 0.47,
            "After A. baumannii infection or LPS, abstract and Results agree: Fig 2A shows lower Cxcl9/\n"
            "Cxcl10 in Rnf213–/– cells (16 of 16 comparisons marked *, ** or ***). The conflict is the\n"
            "IFN-β condition, the only one in which Rnf213–/– cells were given IFN-β and ISGs measured.",
            fontsize=10, color=TXT2, va="center")
    save(fig, "fig1")


# ------------------------------------------------------------------ figure 2
def fig2(groups):
    rows = []
    for gname, sel, gm in groups:
        rows.append(("header", gname, gm))
        rows += [("pt", d, None) for d in sel]
    n = len(rows)
    fig, ax = plt.subplots(figsize=(7.2, 8.4))
    fig.subplots_adjust(left=0.25, right=0.85, top=0.90, bottom=0.09)
    ax.set_xscale("log")
    ax.set_xlim(0.15, 2.0)
    ax.set_ylim(n - 0.4, -0.8)
    ax.axvline(1.0, color=TXT2, lw=0.8, zorder=1)
    ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
    ticks = [0.2, 0.3, 0.5, 0.7, 1.0, 1.5, 2.0]
    ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
    ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.spines["bottom"].set_color(TXT2)
    ax.tick_params(axis="y", length=0)

    ylabels, first = [], True
    for i, (kind, d, gm) in enumerate(rows):
        if kind == "header":
            ylabels.append("")
            ax.text(-0.02, i, d, transform=ax.get_yaxis_transform(), ha="right", va="center",
                    fontsize=10.5, fontweight="bold", color=TXT)
            ax.plot([gm], [i], marker="D", ms=9, mfc="white", mec=BLUE, mew=2, ls="none", zorder=3,
                    label="geometric mean of the 8 ratios (recomputed)" if d == "IFN-β" else None)
            ax.text(gm * 1.07, i, f"{gm:.2f}", va="center", fontsize=10, color=BLUE)
        else:
            ylabels.append(f"{d['cell']}  {d['gene']}  {d['time']}")
            ax.plot([d["r_lo"], d["r_hi"]], [i, i], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
            ax.plot([d["r"]], [i], marker="o", ms=8, color=BLUE, ls="none", zorder=3,
                    label="Rnf213–/– ÷ WT, bar means measured on Fig 2A\n(line: measurement range; recomputed)"
                    if first else None)
            first = False
            ax.text(1.03, i, d["label"], transform=ax.get_yaxis_transform(), va="center", ha="left",
                    fontsize=10.5, fontweight="bold", color=ORANGE)
    ax.set_yticks(range(n)); ax.set_yticklabels(ylabels, fontsize=10, color=TXT)
    ax.text(1.03, -0.75, "label printed\nin Fig 2A\n(reported)", transform=ax.get_yaxis_transform(),
            va="bottom", ha="left", fontsize=10, color=ORANGE)
    ax.set_xlabel("Rnf213–/– ÷ WT, mean Cxcl9 or Cxcl10 mRNA (gene/Hprt), log scale\n"
                  "← lower in Rnf213–/–      1 = no difference      higher in Rnf213–/– →",
                  labelpad=6, color=TXT)
    fig.legend(loc="upper left", bbox_to_anchor=(0.02, 0.997), ncol=1, frameon=False, fontsize=10,
               handletextpad=0.4)
    save(fig, "fig2")


if __name__ == "__main__":
    check_quotes()
    rows = measure()
    groups = []
    for stim in ("IFN-β", "A. baumannii", "LPS"):
        sel, gm = summarise(rows, stim)
        groups.append((stim, sel, gm))
    infl = [d for d in rows if d["stim"] != "IFN-β"]
    rs = [d["r"] for d in infl]
    print(f"\nA. baumannii + LPS together: ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean "
          f"{statistics.geometric_mean(rs):.2f}; largest upper bound {max(d['r_hi'] for d in infl):.2f}; "
          f"printed *, ** or ***: {sum(d['label'] != 'ns' for d in infl)}/{len(infl)}")
    ifnb = groups[0][1]
    print(f"IFN-β: smallest lower bound {min(d['r_lo'] for d in ifnb):.2f}, "
          f"largest upper bound {max(d['r_hi'] for d in ifnb):.2f}")
    fig1(sum(d["label"] == "ns" for d in ifnb), len(ifnb))
    fig2(groups)

The abstract reports increased heat sensitivity, but the paper's own data show increased heat resistance

PLoS Pathog (2018) · doi:10.1371/journal.ppat.1007004 · PMC5927469

Summary

The abstract states that loss of the outer spore protein CsA "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid," grouping heat with the two chemicals. For heat this reverses the direction of the paper's own result. The Results report that the csA-mutant spores had a 15% increase in colony-forming efficiency at 75 °C (p = 0.0286), and, by the paper's stated definition of spore viability, a higher post-treatment colony-forming efficiency means more spores survive — that is, greater resistance, i.e. less sensitivity. The Discussion agrees, stating the mutant spores "were more resistant to heat." The two chemical treatments are decreases in colony-forming efficiency (−51% and −29%), so "increased sensitivity" is correct for them; only the heat term is reversed. The error is confined to a secondary characterization phenotype and the body of the paper carries the correct direction, but the reversed claim sits in the abstract and is echoed in the Discussion's closing summary.

What the paper reports

The absence of CsA reduced sporulation levels and increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid.

— Abstract

Spores of the csA mutant showed an 15% increase in colony forming efficiency when incubated at 75°C (Fig 4A), a 51% decrease in colony forming efficiency when incubated with hydrochloric acid (Fig 4B) and a 29% decrease in colony forming efficiency when incubated with sodium hydroxide (Fig 4C) when compared to wild-type spores (p = 0.0286).

— Results, "The csA and csB mutations do not affect the colony forming efficiency"

Although csA mutant spores had greater sensitivity to hydrochloric acid, they were more resistant to heat and did not show an increased sensitivity to ethanol in comparison to wild-type spores (Fig 4A and 4B, S5C Fig).

— Discussion

CsA is required for wild-type levels of spore resistance to heat, sodium hydroxide and hydrochloric acid, while CsB is required for normal levels of spore adherence to cervical cells.

— Discussion (closing summary; "resistance to heat" here implies the mutant was less resistant, again reversed)

Percentage spore viability (the ability of spores to form colonies) following treatment was calculated as follows: (log10 CFU ml-1 after plating spores on HIS agar post treatment/log10 CFU ml-1 after plating untreated spores on HIS agar) X 100.

— Materials and methods, "Spore resistance" (the paper's own outcome definition)

What the paper's own numbers show

Method. This is a sign/direction error, not an arithmetic slip, so there is no hidden quantity to recompute from raw data (no numerical CFU values for Fig 4 are deposited). The check is a mechanical mapping from the paper's stated outcome definition to the implied direction of each effect: by "percentage spore viability = (log₁₀ CFU after treatment ÷ log₁₀ CFU untreated) × 100," a higher post-treatment colony-forming efficiency (CFE) means more spores survive the treatment = more resistant = less sensitive, and a lower CFE means the opposite. Applying this to the three reported CFE changes gives the direction each should carry, which is then compared with the abstract. Separately, the single reported p-value is checked against the stated n = 4 per group design. Every value below is printed by recompute.py.txt.

Quantity Location Reported Direction implied by the paper's own definition
csA mutant heat (75 °C), CFE vs wild-type Results; Fig 4A +15% (increase) more survival → more resistant = LESS sensitive — opposite to the abstract's "increased sensitivity to heat"
csA mutant hydrochloric acid, CFE vs wild-type Results; Fig 4B −51% (decrease) less survival → more sensitive — abstract agrees (no contradiction)
csA mutant sodium hydroxide, CFE vs wild-type Results; Fig 4C −29% (decrease) less survival → more sensitive — abstract agrees (no contradiction)
Heat direction, Discussion body Discussion "more resistant to heat" = LESS sensitive; consistent with the Results, both disagree with the abstract
Reported p-value Results; Fig 4 legend ("average of four independent experiments"); Methods (Mann-Whitney) p = 0.0286 2 ÷ C(8,4) = 2/70 = 0.0286, the smallest exact two-sided rank-test p for n = 4 vs 4; internally consistent and not the error

The body of the paper (Results, Fig 4 legend's neutral "altered ... resistance," and the Discussion sentence "they were more resistant to heat") is internally consistent on the direction of all three treatments. The abstract sentence is the single outlier, and the Discussion's closing one-line summary repeats it by listing heat among the resistances for which "CsA is required."

Figure

Two-panel diagram contrasting the abstract's claim that loss of CsA increased heat sensitivity with the Results and Discussion, which report a 15% increase in colony-forming efficiency at 75 °C and describe the mutant as more resistant to heat; a per-treatment strip shows the abstract matches the data for the two chemicals but contradicts it for heat.

Figure 1. The direction conflict for heat. Left (orange): the abstract groups heat with sodium hydroxide and hydrochloric acid as treatments to which loss of CsA raised sensitivity. Right (blue): the Results report a +15% colony-forming efficiency at 75 °C and the Discussion calls the mutant spores "more resistant to heat." By the paper's own viability definition a higher post-treatment colony-forming efficiency is more survival, hence greater resistance and less sensitivity, so the two descriptions of heat point in opposite directions. The lower strip applies the same mapping to each treatment: the abstract's "increased sensitivity" matches the data for the two chemicals (both are efficiency decreases) but contradicts it for heat (an efficiency increase).

Horizontal dot plot of the csA-mutant versus wild-type percentage change in colony-forming efficiency for the three treatments: +15% for heat on the "less sensitive" side of zero, and -51% and -29% for hydrochloric acid and sodium hydroxide on the "more sensitive" side; the heat row is annotated as contradicting the abstract while the two chemical rows agree.

Figure 2. The three reported effects on one axis. The csA-mutant-versus-wild-type change in colony-forming efficiency is plotted from zero (no change). Heat alone falls on the positive side (more survival = less sensitive); the two chemicals fall on the negative side (less survival = more sensitive). The abstract's blanket "increased spore sensitivity" is correct for the two chemicals but has the wrong sign for heat.

Why it matters

Spore resistance to heat and chemicals is one of the characterization assays supporting the paper's claim that CsA contributes to spore integrity. The reversed heat term appears in the abstract — a high-exposure, headline location — and is repeated in the Discussion's closing summary, so a reader who takes the abstract at face value would conclude that CsA normally protects spores against heat, the opposite of what the data show (the mutant was more heat-resistant). The magnitude (15%) and the statistical significance are reported correctly in the Results; only the direction word is wrong, and only for heat. The paper's headline conclusions — that the outer spore proteins maintain spore structure, and that CsB loss affects gastrointestinal colonization and clearance — do not depend on the heat direction and are untouched. This is therefore a wrong secondary result rather than a wrong key result or conclusion.

Explanations considered

Status

Suggested correction

Correct the heat term in the abstract so that it is not grouped with the two chemicals — for example, "loss of CsA reduced sporulation levels, increased spore sensitivity to sodium hydroxide and hydrochloric acid, and altered (increased) resistance to heat" — and revise the Discussion's closing summary accordingly so that it no longer implies the mutant was less heat-resistant. The Results text, the Fig 4 legend and the Discussion sentence "they were more resistant to heat" already carry the correct direction and need no change.

Recomputation script (python recompute.py.txt)
"""Recompute / verify the heat-sensitivity direction in PMC5927469
(PLoS Pathog 2018, doi:10.1371/journal.ppat.1007004).

Run:  python recompute.py.txt
Needs matplotlib (and the standard library). Prints every number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.

The dispute is a SIGN/DIRECTION error, not an arithmetic slip, so the "recompute"
is (a) a mechanical mapping from the paper's own outcome definition to the implied
direction of each effect, and (b) an exact check that the single reported p-value is
consistent with the stated n = 4 per group design. All inputs are copied from the
paper as archived in PubMed Central (src/show.py PMC5927469).

Paper's own outcome definition (Materials and methods, "Spore resistance"):
  percentage spore viability after a treatment =
    (log10 CFU/ml after treatment / log10 CFU/ml untreated) x 100
  -> a HIGHER colony-forming efficiency (CFE) after a treatment means MORE spores
     survive it  =>  MORE resistant  =>  LESS sensitive.
  -> a LOWER CFE after a treatment means fewer survive  =>  LESS resistant
     =>  MORE sensitive.
"""
import os
from math import comb

HERE = os.path.dirname(os.path.abspath(__file__))

# ---------------------------------------------------------------- inputs (from the paper)
# csA mutant vs wild-type, % change in colony-forming efficiency after each treatment
# (Results, "The csA and csB mutations do not affect the colony forming efficiency"; Fig 4)
REPORTED_CFE_CHANGE = {          # treatment : % change in CFE for csA mutant vs WT
    "heat (75 C)":        +15,
    "hydrochloric acid":  -51,
    "sodium hydroxide":   -29,
}

# What the abstract says about each treatment. The abstract groups all three together:
#   "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid"
ABSTRACT_SAYS = {t: "MORE sensitive" for t in REPORTED_CFE_CHANGE}

# What the Discussion body says about each treatment (Discussion #p4).
DISCUSSION_SAYS = {
    "heat (75 C)":        "MORE resistant / LESS sensitive",   # "they were more resistant to heat"
    "hydrochloric acid":  "MORE sensitive",                    # "greater sensitivity to hydrochloric acid"
    "sodium hydroxide":   "MORE sensitive",                    # "increased sensitivity to sodium hydroxide"
}

REPORTED_P = 0.0286          # Results, single p-value for the csA vs WT comparisons
N_PER_GROUP = 4              # Fig 4 legend: "the average of four independent experiments";
                             # Methods: Mann-Whitney test


def cfe_change_to_sensitivity(pct_change):
    """Map a csA-vs-WT % change in post-treatment CFE to a sensitivity direction,
    using the paper's own definition of percentage spore viability."""
    if pct_change > 0:
        return "LESS sensitive (= MORE resistant)"
    if pct_change < 0:
        return "MORE sensitive (= LESS resistant)"
    return "no change"


# ---------------------------------------------------------------- 1. direction check
print("=== 1. Direction implied by the paper's own outcome definition, per treatment ===")
print(f"{'treatment':20s} {'reported CFE':>12s}  {'implied direction':34s} "
      f"{'abstract says':16s} {'discussion says':26s} abstract vs data")
print("-" * 118)
contradictions = []
for t, chg in REPORTED_CFE_CHANGE.items():
    implied = cfe_change_to_sensitivity(chg)
    implied_more = implied.startswith("MORE sensitive")
    abstract_more = ABSTRACT_SAYS[t].startswith("MORE")
    agree = (implied_more == abstract_more)
    if not agree:
        contradictions.append(t)
    verdict = "agree" if agree else "CONTRADICTION"
    print(f"{t:20s} {chg:+11d}%  {implied:34s} {ABSTRACT_SAYS[t]:16s} "
          f"{DISCUSSION_SAYS[t]:26s} {verdict}")

print()
print(f"Treatments where the abstract contradicts the paper's own CFE data: {contradictions}")
print(f"Results (+15% CFE at 75 C) implies: {cfe_change_to_sensitivity(+15)}")
print(f"Discussion body says for heat:      {DISCUSSION_SAYS['heat (75 C)']}")
print("Results and Discussion agree with each other on heat: True")
print("Abstract agrees with them on heat:                    False "
      "(abstract groups heat under 'increased sensitivity')")
print("Abstract agrees on the two chemicals (HCl, NaOH):     True "
      "(both are CFE decreases = increased sensitivity)")

# ---------------------------------------------------------------- 2. p-value sanity check
print("\n=== 2. Is the single reported p = 0.0286 consistent with n = 4 per group? ===")
total = comb(2 * N_PER_GROUP, N_PER_GROUP)       # C(8,4)
smallest_two_sided = 2.0 / total
print(f"n1 = n2 = {N_PER_GROUP}: number of ways to split 8 ranks = C(8,4) = {total}")
print(f"smallest attainable exact two-sided Mann-Whitney p = 2/{total} = {smallest_two_sided:.4f}")
print(f"reported p = {REPORTED_P}  matches 2/{total} = {smallest_two_sided:.4f}: "
      f"{abs(smallest_two_sided - REPORTED_P) < 5e-4}")
print("=> the p-value is exactly the minimum a 4-vs-4 rank test can give; it is internally "
      "consistent and is NOT the error.")

# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
GREEN_OK, PANEL = "#3a8a4f", "#f4f3ef"

import matplotlib                                   # noqa: E402
matplotlib.use("Agg")
import matplotlib.pyplot as plt                     # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402
from matplotlib.lines import Line2D                 # noqa: E402

plt.rcParams.update({
    "font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
    "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none",
})


def save(fig, stem):
    fig.savefig(os.path.join(HERE, stem + ".svg"))
    fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
    plt.close(fig)


# ---- Figure 1: two-panel abstract-vs-body contradiction diagram -----------------
fig, ax = plt.subplots(figsize=(7.2, 5.6))
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")


def panel(x, y, w, h, title, title_col, body, foot, foot_col):
    box = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.6,rounding_size=2.5",
                         linewidth=1.6, edgecolor=title_col, facecolor=PANEL, zorder=1)
    ax.add_patch(box)
    ax.text(x + w / 2, y + h - 3.2, title, ha="center", va="top",
            fontsize=12, fontweight="bold", color=title_col)
    ax.text(x + w / 2, y + h - 11, body, ha="center", va="top",
            fontsize=9.3, color=TXT, linespacing=1.32)
    ax.text(x + w / 2, y + 3.5, foot, ha="center", va="bottom",
            fontsize=9.1, color=foot_col, fontweight="bold", linespacing=1.25)


panel(3, 60, 43, 37, "Abstract", ORANGE,
      '"The absence of CsA reduced\n'
      'sporulation levels and increased\n'
      'spore sensitivity to heat, sodium\n'
      'hydroxide and hydrochloric acid."',
      "→ heat grouped with the chemicals\nas MORE sensitive", ORANGE)

panel(54, 60, 43, 37, "Results  +  Discussion", BLUE,
      'Results / Fig 4A: csA mutant had a\n'
      '"15% increase in colony forming\n'
      'efficiency when incubated at 75 C"\n'
      'Discussion: "they were more\n'
      'resistant to heat"',
      "→ heat: MORE resistant\n= LESS sensitive", BLUE)

# connector naming the conflict
arr = FancyArrowPatch((46, 78), (54, 78), arrowstyle="<|-|>", mutation_scale=18,
                      linewidth=2, color="#b03020", zorder=3)
ax.add_patch(arr)
ax.text(50, 73, "heat\ndirection\nreversed", ha="center", va="top",
        fontsize=9.5, fontweight="bold", color="#b03020", linespacing=1.15)

# bottom strip: the three treatments and whether abstract matches the data
ax.text(50, 52, "By the paper’s own definition "
        "(viability = log₁₀ CFU after ÷ log₁₀ CFU untreated × 100), a higher\n"
        "post-treatment CFE means more survival = more resistant = less sensitive:",
        ha="center", va="top", fontsize=9.0, color=SEC, linespacing=1.3)

cells = [
    ("heat (75 C)", "+15% CFE", "less sensitive", "“more sensitive”", False),
    ("hydrochloric acid", "−51% CFE", "more sensitive", "“more sensitive”", True),
    ("sodium hydroxide", "−29% CFE", "more sensitive", "“more sensitive”", True),
]
cx = [17, 50, 83]
for i, (t, cfe, implied, says, ok) in enumerate(cells):
    col = GREEN_OK if ok else "#b03020"
    mark = "✓ abstract matches" if ok else "✗ abstract contradicts"
    ax.text(cx[i], 38, t, ha="center", fontsize=9.6, fontweight="bold", color=TXT)
    ax.text(cx[i], 31, f"data: {cfe} → {implied}", ha="center", fontsize=8.7, color=BLUE)
    ax.text(cx[i], 25.5, f"abstract: {says}", ha="center", fontsize=8.7, color=ORANGE)
    ax.text(cx[i], 18, mark, ha="center", fontsize=9.2, fontweight="bold", color=col)

legend = [Line2D([0], [0], color=ORANGE, lw=6, label="paper’s abstract claim"),
          Line2D([0], [0], color=BLUE, lw=6, label="direction from the paper’s own numbers")]
ax.legend(handles=legend, loc="lower center", bbox_to_anchor=(0.5, 0.0),
          ncol=2, frameon=False, fontsize=9, handlelength=1.4)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.04)
save(fig, "fig1")

# ---- Figure 2: per-treatment CFE change, heat as the sign outlier ---------------
fig, ax = plt.subplots(figsize=(7.2, 3.8))
treatments = list(REPORTED_CFE_CHANGE.keys())
vals = [REPORTED_CFE_CHANGE[t] for t in treatments]
ypos = list(range(len(treatments)))[::-1]

# shade the two half-planes and label what each side means
ax.axvspan(-80, 0, color="#f3e6df", zorder=0)
ax.axvspan(0, 60, color="#e3edf6", zorder=0)
ax.text(-40, 2.62, "CFE decrease → fewer survive\n= MORE sensitive",
        ha="center", va="center", fontsize=8.6, color="#9a3412")
ax.text(30, 2.62, "CFE increase → more survive\n= LESS sensitive",
        ha="center", va="center", fontsize=8.6, color="#1e4e79")
ax.axvline(0, color=SEC, lw=1.4, zorder=2)

for y, t, v in zip(ypos, treatments, vals):
    ax.plot([0, v], [y, y], color="#b7b6b1", lw=2, zorder=1)
    ax.plot(v, y, "o", ms=13, color=ORANGE, zorder=3)
    ax.text(v + (4 if v > 0 else -4), y + 0.17, f"{v:+d}%", va="center",
            ha="left" if v > 0 else "right", fontsize=11, fontweight="bold", color=TXT)
    agree = (v < 0)  # abstract says "increased sensitivity" for all three
    if agree:
        ax.text(v - 4, y - 0.26, "✓ abstract agrees", va="center", ha="right",
                fontsize=8.6, fontweight="bold", color=GREEN_OK)
    else:
        ax.text(58, y - 0.26, "✗ abstract says “more sensitive” — contradicts",
                va="center", ha="right", fontsize=8.6, fontweight="bold", color="#b03020")

ax.set_yticks(ypos)
ax.set_yticklabels(treatments, fontsize=10.5)
ax.set_ylim(-0.6, 3.1)
ax.set_xlim(-80, 60)
ax.set_xlabel("csA mutant vs wild-type: % change in colony-forming efficiency after treatment "
              "(Results, Fig 4)", fontsize=9.3)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.set_title("Only heat is a CFE increase; the abstract groups all three as "
             "“increased sensitivity”", fontsize=10, color=TXT, pad=8)
fig.tight_layout()
save(fig, "fig2")

print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)

Abstract reports phantom-sensation scores as phantom range of motion in degrees

PNAS (2021) · doi:10.1073/pnas.2019555118 · PMC7936324

Summary

The abstract states that subjects with an agonist–antagonist myoneural interface below-knee amputation (AB, n = 15) reported "significantly greater phantom range of motion" than subjects with a traditional below-knee amputation (TB, n = 7), quantified as "AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees". In the body of the paper, 12.47 ± 2.41 and 10.14 ± 1.9 are the group means of the phantom sensation score, a unitless sum of 1–3 grades over foot regions. The per-subject scores in Fig. 1B reproduce these values exactly (recomputed 12.47 ± 2.42 and 10.14 ± 1.95). Phantom range of motion is a different measure: the paper reports it as a percentage of the contralateral limb, separately for each direction, and never as a single value in degrees. The abstract therefore attaches another outcome's numbers, a unit and a TB standard deviation (1.45) that none of the paper's data reproduce to one of its headline comparisons. Readers of the abstract would take the difference as about 2.3 degrees of phantom motion. It is in fact a 23% higher sensation score.

What the paper reports

The abstract:

"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)." (Abstract)

The Results give the same means, with a different label and a different TB SD:

"AB subjects reported significantly higher phantom sensations (12.47 ± 2.41, P = 0.03) compared to TB subjects (10.14 ± 1.9) (P < 0.05, Student’s t test)." (Results, "Phantom Sensations and Pain")

The Methods define the phantom sensation score as a grade per foot region:

"Subjects were then specifically asked to relate the level of perception of their lateral and medial malleoli, heel pad, arch of the foot, balls of the footpad, and toes. The more thorough and tangible the phantom limb felt to subjects, the greater the phantom sensation scored. Responses were independently graded on a 1 to 3 scale" (Methods, "Phantom Limb Sensation")

Fig. 1B is a per-subject table. It has five region columns (Malleoli, Heelpad, Arch of foot, Balls of feet, Toes), a "Phantom Sensation Score" column and a "Phantom Pain Score" column, with summary rows:

"Average Std Dev 12.47 +/- 2.4" (AB) and "Average Std Dev 10.14 +/- 1.9" (TB) (Fig. 1B, Phantom Sensation Score column; transcribed from the version-of-record PDF)

"Phantom sensations in the AB cohort (n = 15) are significantly greater than those in the TB cohort (n = 7) (P < 0.05, Student’s t test)." (Fig. 1 legend, panel B)

Phantom range of motion (ROM) is measured and reported differently:

"Data were processed through custom scripts in MATLAB (Mathworks) to identify the maximum amplitude for each movement and normalized as a percentage of the amplitude of the contralateral limb." (Methods, "ROM Percepts")

"The demonstrated ROM of the phantom limb in each direction (PF, DF, IN, EV) were normalized as a percentage of the unaffected joints’ ROM (Fig. 2C)." (Results, "ROM Percepts")

"Further, AB subjects experienced significantly greater ROM than TB subjects following amputation for PF, DF, and EV (P < 0.05, Student’s two-tailed t test)." (Results, "ROM Percepts")

What the paper's own numbers show

Method. The 22 per-subject phantom sensation scores and pain scores were transcribed from Fig. 1B of the publisher's PDF into recompute.py.txt:

The script computes each group's mean, its sample SD (n − 1), its population SD (n) and its SEM, and runs Student’s and Welch's two-sample t tests on the raw scores and on the reported summary statistics. It also derives the possible range of the score from the number of graded regions. The Methods, the Results ("ROM Percepts") and the Fig. 2 legend were searched for any phantom ROM value expressed in degrees.

Quantity Location Reported Recomputed
AB value in the "phantom range of motion" sentence Abstract 12.47 ± 2.41 degrees Equals the AB phantom sensation score: Fig. 1B mean 12.467 (187/15), SD 2.416
TB value in the "phantom range of motion" sentence Abstract 10.14 ± 1.45 degrees Mean equals the TB phantom sensation score: Fig. 1B mean 10.143 (71/7). SD is 1.952, not 1.45
AB phantom sensation score Results; Fig. 1B 12.47 ± 2.41 (Fig. 1B: ± 2.4) 12.467 ± 2.416
TB phantom sensation score Results; Fig. 1B 10.14 ± 1.9 10.143 ± 1.952
TB dispersion statistics that could produce 1.45 Abstract 1.45 Sample SD 1.95, population SD 1.81, SEM 0.74: none equals 1.45
Possible range of the phantom sensation score Methods; Fig. 1B 1–3 per region; 5 region columns 5–15, with no unit (6–18 if the two malleoli were graded separately). Observed scores run from 8 to 15, and 2 AB subjects score 15
Phantom ROM, the measure the abstract names Methods and Results, "ROM Percepts"; Fig. 2C % of the contralateral limb, per direction (PF, DF, IN, EV) No pooled value and no value in degrees appears anywhere in the paper
AB − TB difference as stated in the abstract Abstract 12.47 − 10.14 "degrees" 2.33 score points; ratio of the means 1.230 (23% higher score)
Phantom sensation test, raw Fig. 1B scores Results P = 0.03 (Student’s t test) Student t = 2.220, P = 0.038; Welch t = 2.405, P = 0.030
Phantom sensation test, summary statistics with TB SD 1.9 Results P = 0.03 Student t = 2.243, P = 0.036; Welch P = 0.027
Phantom sensation test, summary statistics with TB SD 1.45 Abstract P < 0.05 Student t = 2.349, P = 0.029; Welch P = 0.011

Whichever SD is used, the phantom-sensation comparison remains significant at P < 0.05. (The reported P = 0.03 matches Welch's test on the raw scores, 0.030. Student's test, which the paper names, gives 0.038. Both are below 0.05.) The discrepancy lies in what the abstract's numbers are said to measure, in their unit and in the TB SD. It does not affect whether the comparison is significant.

Other observations (outside the main finding). The pain values in the text do not match Fig. 1B either. The pain score column of Fig. 1B gives AB 3.467 ± 4.96 and TB 9.286 ± 10.23 (printed summary rows: 3.46 ± 4.9 and 9.28 ± 10.2). The Results give 12.25 ± 4.9 and 17.29 ± 10.22: the SDs agree with Fig. 1B but the means do not. The abstract gives the AB pain SD as 5.37 rather than 4.9. Fig. 1B lists 7 AB subjects with a pain score of 0, and the Results say 6 of the 15 "indicated zero pain in all categories". From the reported summary statistics, the AB versus TB pain comparison gives Student P = 0.128 (SD 4.9) or 0.141 (SD 5.37), against the reported P = 0.08. The paper does not describe enough of how the pain score was built to resolve these differences, so they are listed here only for completeness.

Figure

The abstract's "phantom range of motion" numbers are the Results' phantom sensation scores

Figure 1. The abstract sentence (top, orange outline) gives "12.47 ± 2.41" and "10.14 ± 1.45 degrees" as phantom range of motion. The same means appear in the Results (bottom left) as phantom sensation scores, which are sums of 1–3 grades over the five foot regions of Fig. 1B. The possible range is 5–15 and the score has no unit. The TB SD there is 1.9, and the Fig. 1B per-subject scores give 1.95 (blue, recomputed in recompute.py.txt). Phantom range of motion, the measure the abstract names (bottom right), is reported only as a percentage of the contralateral limb, separately for each direction, and never in degrees. Orange text marks values printed in the paper, blue text marks values recomputed from the paper's own per-subject data.

The abstract's TB SD of 1.45 matches no dispersion statistic of the TB scores

Figure 2. Dispersion of the seven TB phantom sensation scores listed in Fig. 1B (8, 8, 11, 12, 13, 9, 10). Orange circles: SDs printed in the abstract (1.45) and in the Results and Fig. 1B (1.9). Blue diamonds: the sample SD (1.95), population SD (1.81) and SEM (0.74) recomputed from the per-subject scores. The dashed line marks the abstract's 1.45. None of the recomputed statistics equals it, so the abstract's SD cannot be explained as a different summary of the same data.

Why it matters

The sentence is the abstract's only quantitative statement about phantom perception, and the paper's Significance statement and Discussion present phantom perception as a main benefit of the procedure. Read as written, the sentence tells the reader that AB subjects could perceive about 2.3 degrees more phantom joint motion than TB subjects. That would be a very small absolute range for whole-ankle motion. It would also conflict with the paper's own ROM method, which expresses ROM as a percentage of the unaffected limb. The numbers actually describe something else: AB subjects' summed phantom sensation grades were about 23% higher than TB subjects'. The abstract also gives a TB SD (1.45) smaller than the one in the Results (1.9) and the one the data support (1.95), which makes the TB group look less variable than it was.

The direction and statistical significance of both underlying findings are unaffected. In the Results, AB subjects have higher phantom sensation scores (P < 0.05 under every recomputation above), and significantly greater phantom ROM in three of the four directions (PF, DF, EV). The error is in how the abstract reports these findings. It merges two outcomes into one sentence, gives the magnitude of one under the name and a unit of the other, and leaves the actual ROM result (percent of contralateral ROM by direction) unquantified in the abstract. Anyone extracting effect sizes from the abstract, for example in a systematic review or meta-analysis of phantom limb outcomes after amputation, would record a degree-valued ROM difference that the study did not measure.

Explanations considered

Status

Suggested correction

In the abstract, replace

"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)."

with a sentence that keeps the two outcomes separate and reports the values given in the Results and Fig. 1B, for example:

"AB subjects reported significantly higher phantom sensation scores (AB: 12.47 ± 2.41, TB: 10.14 ± 1.9; P < 0.05) and significantly greater phantom range of motion, as a percentage of the unaffected limb, for plantarflexion, dorsiflexion and eversion (P < 0.05) when compared to TB subjects."

If the authors intended a different TB SD or a phantom-ROM summary in degrees, the source of 1.45 and of any degree values should be stated, since neither can be derived from the data presented. Separately, the authors could clarify how the text pain values (12.25 ± 4.9 and 17.29 ± 10.22, with 5.37 in the abstract) relate to the Fig. 1B pain scores (3.46 ± 4.9 and 9.28 ± 10.2).

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7936324 and regenerate fig1/fig2.

Run:  python recompute.py.txt   (writes fig1.svg/png and fig2.svg/png next to this file)

Inputs are transcribed from the article (PNAS 2021, doi:10.1073/pnas.2019555118):
- Abstract and Results text, as archived in PubMed Central (PMC7936324) and identical
  in the publisher's typeset PDF ("Final published version").
- Fig. 1B, the per-subject table of Phantom Sensation Score and Phantom Pain Score
  (15 AB rows, 7 TB rows, in printed order), read from the version-of-record PDF.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))

# ------------------------------------------------------------------ inputs
# Fig. 1B, "Phantom Sensation Score" column
PS_AB = np.array([14, 11, 8, 13, 15, 14, 14, 8, 13, 14, 10, 14, 14, 15, 10])
PS_TB = np.array([8, 8, 11, 12, 13, 9, 10])
# Fig. 1B, "Phantom Pain Score" column
PAIN_AB = np.array([0, 2, 0, 1, 5, 0, 10, 15, 12, 3, 0, 0, 0, 0, 4])
PAIN_TB = np.array([0, 5, 0, 11, 11, 8, 30])
# Fig. 1B region columns: Malleoli, Heelpad, Arch of foot, Balls of feet, Toes
N_REGION_COLS = 5
GRADE_MIN, GRADE_MAX = 1, 3          # Methods: "graded on a 1 to 3 scale"

# Reported values
ABS = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.45)       # Abstract, "degrees"
RES = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.9, p=0.03)  # Results, phantom sensation
FIG1B = dict(ab_m=12.47, ab_s=2.4, tb_m=10.14, tb_s=1.9,
             pain_ab_m=3.46, pain_ab_s=4.9, pain_tb_m=9.28, pain_tb_s=10.2)
PAIN_RES = dict(ab_m=12.25, ab_s=4.9, tb_m=17.29, tb_s=10.22, p=0.08)
PAIN_ABS_AB_S = 5.37
nA, nT = len(PS_AB), len(PS_TB)


def desc(x):
    return x.mean(), x.std(ddof=1), x.std(ddof=0), x.std(ddof=1) / np.sqrt(len(x))


print("== 1. Scale of the phantom sensation score ==")
lo, hi = N_REGION_COLS * GRADE_MIN, N_REGION_COLS * GRADE_MAX
print(f"Fig. 1B region columns: {N_REGION_COLS}; graded {GRADE_MIN}-{GRADE_MAX} -> possible total {lo}-{hi}")
print(f"(if lateral and medial malleoli were graded separately, 6 regions -> {6*GRADE_MIN}-{6*GRADE_MAX})")
print(f"Observed Fig. 1B scores: AB {PS_AB.min()}-{PS_AB.max()}, TB {PS_TB.min()}-{PS_TB.max()}")
print(f"AB subjects at the {hi}-point maximum: {(PS_AB == hi).sum()}")
print()

print("== 2. Phantom sensation scores recomputed from Fig. 1B ==")
mA, sA, pA, seA = desc(PS_AB)
mT, sT, pT, seT = desc(PS_TB)
print(f"AB: n={nA}, sum={PS_AB.sum()}, mean={mA:.3f}, SD(n-1)={sA:.3f}, SD(n)={pA:.3f}, SEM={seA:.3f}")
print(f"TB: n={nT}, sum={PS_TB.sum()}, mean={mT:.3f}, SD(n-1)={sT:.3f}, SD(n)={pT:.3f}, SEM={seT:.3f}")
print(f"Reported Results/Fig. 1B: AB {RES['ab_m']} +/- {RES['ab_s']}, TB {RES['tb_m']} +/- {RES['tb_s']}")
print(f"Reported Abstract ('degrees'): AB {ABS['ab_m']} +/- {ABS['ab_s']}, TB {ABS['tb_m']} +/- {ABS['tb_s']}")
cands = {"SD (n-1)": sT, "SD (n)": pT, "SEM": seT}
print("TB dispersion candidates vs abstract 1.45:",
      ", ".join(f"{k} {v:.2f}" for k, v in cands.items()),
      "-> any equal to 1.45 after rounding:", any(round(v, 2) == 1.45 for v in cands.values()))
print(f"Abstract AB-TB difference: {ABS['ab_m'] - ABS['tb_m']:.2f} (abstract unit: degrees); "
      f"as a ratio of score means: {ABS['ab_m'] / ABS['tb_m']:.3f} "
      f"({100 * (ABS['ab_m'] / ABS['tb_m'] - 1):.0f}% higher score)")
print()

print("== 3. Two-sample t tests ==")
def tt(m1, s1, n1, m2, s2, n2, eq):
    r = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=eq)
    return r.statistic, r.pvalue
for lab, s in [("Results TB SD 1.9", RES["tb_s"]), ("Abstract TB SD 1.45", ABS["tb_s"])]:
    t, p = tt(12.47, 2.41, nA, 10.14, s, nT, True)
    tw, pw = tt(12.47, 2.41, nA, 10.14, s, nT, False)
    print(f"Summary stats, {lab}: Student t={t:.3f} p={p:.3f}; Welch t={tw:.3f} p={pw:.3f}")
r_s = stats.ttest_ind(PS_AB, PS_TB, equal_var=True)
r_w = stats.ttest_ind(PS_AB, PS_TB, equal_var=False)
print(f"Fig. 1B per-subject scores: Student t={r_s.statistic:.3f} p={r_s.pvalue:.3f}; "
      f"Welch t={r_w.statistic:.3f} p={r_w.pvalue:.3f} (reported P = {RES['p']}, 'Student's t test')")
print()

print("== 4. Side observations: pain scores (not part of the main finding) ==")
pmA, psA, _, _ = desc(PAIN_AB)
pmT, psT, _, _ = desc(PAIN_TB)
print(f"Fig. 1B pain, recomputed: AB mean={pmA:.3f} SD={psA:.2f}; TB mean={pmT:.3f} SD={psT:.2f}")
print(f"Fig. 1B pain summary rows: AB {FIG1B['pain_ab_m']} +/- {FIG1B['pain_ab_s']}; "
      f"TB {FIG1B['pain_tb_m']} +/- {FIG1B['pain_tb_s']}")
print(f"Results pain: AB {PAIN_RES['ab_m']} +/- {PAIN_RES['ab_s']}; TB {PAIN_RES['tb_m']} +/- {PAIN_RES['tb_s']}; "
      f"Abstract AB SD {PAIN_ABS_AB_S}")
print(f"AB subjects with a Fig. 1B pain score of 0: {(PAIN_AB == 0).sum()} of {nA} "
      f"(Results: '6 of the 15 subjects ... indicated zero pain in all categories')")
for lab, s in [("Results AB SD 4.9", 4.9), ("Abstract AB SD 5.37", 5.37)]:
    t, p = tt(12.25, s, nA, 17.29, 10.22, nT, True)
    print(f"Pain, summary stats, {lab}: Student t={t:.3f} p={p:.3f} (reported P = {PAIN_RES['p']})")

# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
                     "text.color": INK, "axes.labelcolor": INK,
                     "xtick.color": INK2, "ytick.color": INK2})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


def box(ax, x, y, w, h, title, body, width_chars, edge=INK2):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.004,rounding_size=0.012",
                                fc="white", ec=edge, lw=1.2, transform=ax.transAxes))
    ax.text(x + 0.015, y + h - 0.022, title, fontsize=10, fontweight="bold", va="top",
            transform=ax.transAxes)
    ax.text(x + 0.015, y + h - 0.075, "\n".join(textwrap.wrap(body, width_chars)),
            fontsize=10, va="top", color=INK, transform=ax.transAxes, linespacing=1.35)


# ---- Figure 1: abstract vs body text diagram
fig = plt.figure(figsize=(7.2, 5.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_facecolor(BG)
box(ax, 0.03, 0.80, 0.94, 0.17, "Abstract",
    "\u201cAB subjects reported significantly greater phantom range of motion postamputation "
    "(AB: 12.47 \u00b1 2.41, TB: 10.14 \u00b1 1.45 degrees) when compared to TB subjects (P < 0.05).\u201d",
    92, edge=ORANGE)
box(ax, 0.03, 0.15, 0.455, 0.285, "Results: Phantom Sensations and Pain",
    "\u201cAB subjects reported significantly higher phantom sensations (12.47 \u00b1 2.41, P = 0.03) "
    "compared to TB subjects (10.14 \u00b1 1.9)\u201d. Fig. 1B lists the per-subject scores: "
    "five foot regions, each graded 1\u20133 (Methods).",
    40)
box(ax, 0.515, 0.15, 0.455, 0.285, "Methods / Results: ROM Percepts",
    "Phantom ROM was \u201cnormalized as a percentage of the amplitude of the contralateral limb\u201d "
    "and reported per direction (PF, DF, IN, EV; Fig. 2C). No pooled phantom ROM in degrees "
    "appears anywhere in the paper.",
    40)
for x in (0.10, 0.585):
    ax.add_patch(FancyArrowPatch((x, 0.79), (x, 0.445), transform=ax.transAxes,
                                 arrowstyle="-|>", mutation_scale=14, lw=2, color=INK2))
def note(x, y, s, **kw):
    ax.text(x, y, s, fontsize=10, va="top", transform=ax.transAxes, linespacing=1.35, **kw)
note(0.12, 0.76, "Same numbers, different measure", fontweight="bold")
note(0.12, 0.72, "Unitless score, possible range 5\u201315\n(Fig. 1B scores span 8\u201315)")
note(0.12, 0.635, "TB SD  1.45  abstract (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.595, "TB SD  1.9    Results (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.555, f"TB SD  {sT:.2f}  Fig. 1B (recomputed)", color=BLUE, fontweight="bold")
note(0.605, 0.76, "The measure the abstract names", fontweight="bold")
note(0.605, 0.72, "% of the contralateral limb,\nper direction; never pooled\nand never given in degrees")
ax.legend(handles=[Patch(fc="white", ec=ORANGE, lw=1.5, label="Text containing the conflicting statement"),
                   Line2D([], [], color=ORANGE, lw=0, marker="s", ms=9, label="Value reported in the paper"),
                   Line2D([], [], color=BLUE, lw=0, marker="s", ms=9,
                          label="Recomputed from the paper's Fig. 1B per-subject scores")],
          loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")

# ---- Figure 2: TB dispersion, abstract value vs everything the Fig. 1B data give
rows = [
    ("Abstract TB SD", ABS["tb_s"], ORANGE, "o", "1.45"),
    ("Results / Fig. 1B TB SD", RES["tb_s"], ORANGE, "o", "1.9"),
    ("Sample SD (n−1), Fig. 1B scores", sT, BLUE, "D", f"{sT:.2f}"),
    ("Population SD (n), Fig. 1B scores", pT, BLUE, "D", f"{pT:.2f}"),
    ("SEM, Fig. 1B scores", seT, BLUE, "D", f"{seT:.2f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, c, m, txt) in zip(ys, rows):
    ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
    ax.plot(v, y, marker=m, ms=10, color=c, zorder=3, lw=0)
    ax.text(v + 0.08, y, txt, va="center", fontsize=10, color=c, fontweight="bold")
ax.axvline(ABS["tb_s"], color=ORANGE, lw=1, ls=(0, (3, 3)), zorder=0)
ax.set_yticks(ys, [r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 2.4)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Dispersion of the TB phantom-sensation score (score points), n = 7")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported in the paper"),
                   Line2D([], [], color=BLUE, marker="D", ms=9, lw=0,
                          label="Recomputed from Fig. 1B (TB scores 8, 8, 11, 12, 13, 9, 10)")],
          loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")

Abstract reverses the direction of the white-light calcium responses shown in Results and Fig. 4B

Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1802592115 · PMC6217433

Summary

The paper reports that a single bistable opsin, parapinopsin, gives zebrafish pineal photoreceptors opposite calcium responses to UV and visible light. In one experiment (Fig. 4B), white light was shifted toward UV or toward visible wavelengths. The abstract says these shifts "respectively increased vs. decreased" the calcium signal. The Results paragraph, the Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405-nm and 588-nm two-photon experiments, and the paper's proposed mechanism all give the opposite: UV-shifted light lowered calcium and visible-shifted light raised it. The existence of opponency and its dependence on parapinopsin are unaffected. But the abstract, the part most readers and indexers rely on, states the sign of the naturalistic-light result backwards, and nothing in the abstract lets a reader notice.

What the paper reports

Abstract:

Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.

Results, section "Color Opponency Based on Photoequilibrium of Parapinopsin Two States", white-light paragraph:

We found decreased and increased calcium levels following exposure to UV and visible light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B).

Fig. 4 legend, panel B:

Note that the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively, in WT and PT-/-.

Results, section "Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell" (two-photon imaging, Fig. 1C):

stimulation with 405-nm light caused a decrease in fluorescence intensity.

In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV and visible light (Fig. 1C).

Results, mechanism paragraph (Fig. 3):

The 405-nm light stimulus increases the short-wavelength component of incident light and thus increases the fractional concentration of the signaling active photoproduct from that produced by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease).

Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, calcium increase).

Discussion, first paragraph:

with lights biased toward UV driving greater hyperpolarization than those biased toward visible light.

Fig. 4 legend, panels A and B (stimulus intensities and sample sizes):

The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and ∼7.6 × 1013 photons per cm2·s, respectively.

Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed triangle; n = 16)

(The exponents read "1014", "1013" and "1017" in the archived text. They are superscripts lost in conversion: 10^14, 10^13, 10^17.)

What the paper's own numbers show

This is a check of direction, not of arithmetic. Each sentence that uses "respectively" pairs the i-th stimulus with the i-th direction word. recompute.py.txt pulls both word orders out of the quoted sentences with a regular expression, builds the stimulus-to-direction mapping for each sentence, and compares the mappings. For the Fig. 4B traces, the direction of the WT and PT-/- deflections at the UV-mixed and visible-mixed bars, and the later course of the traces, were read by eye from the published figure image. No values were measured from it. The script also checks every quotation above against the PMC archive and does the arithmetic on the Fig. 4A intensities.

Quantity Location Reported Recomputed from the paper's body
Calcium response to UV-enhanced white light Abstract increased decreased
Calcium response to visible-enhanced white light Abstract decreased increased
Calcium response to UV- / visible-mixed white light Results, white-light paragraph (Fig. 4B) decreased / increased decreased / increased (agrees)
Calcium response to UV- / visible-mixed white light Fig. 4 legend, panel B decreased / increased decreased / increased (agrees)
Transient deflection of WT and PT-/- traces at UV- / visible-mixed bars Fig. 4B, plotted data downward / upward decreased / increased (agrees; for the later rebound see Explanations, item 4)
Fluorescence response to 405-nm / 588-nm light (two-photon) Results, Fig. 1C decrease / elevated decreased / increased (agrees)
Predicted calcium change for 405-nm / 588-nm light Results, mechanism (Fig. 3) calcium decrease / calcium increase decreased / increased (agrees)
UV-biased vs visible-biased light Discussion greater hyperpolarization for UV-biased lower calcium under UV-biased light (agrees)
Body passages with UV → decrease and visible → increase all of the above except the abstract — 5 of 5, plus the Discussion's comparison
White + UV LED; white + visible LED (photons per cm²·s) Fig. 4 legend A ~9.3 × 10^14; ~9.9 × 10^14 9.33 × 10^14 (LED adds 1.4%); 9.96 × 10^14 (LED adds 8.3%). Both agree with the legend within the rounding of its approximate (~) inputs; white light is itself ~9.2 × 10^14
n per genotype, Fig. 4B Fig. 4 legend B WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16 unaffected by the direction issue

The abstract gives UV → increase and visible → decrease. Every other passage that states a direction gives UV → decrease and visible → increase. Both added LEDs raise the total photon flux above the same white background (by 1.4% and 8.3%), so the two stimuli differ in spectral composition, not in the sign of the intensity change. The opposite responses are therefore spectral, as the paper argues, and the direction of each is fixed by the body of the paper.

Figure

Two text panels, Abstract and Results plus Fig. 4 legend, with the stimulus-to-direction mapping each one implies

Figure 1. The left panel quotes the abstract and the right panel quotes the Results sentence on Fig. 4B. Under each quotation is the stimulus-to-direction mapping its "respectively" construction implies. Orange marks the abstract as published. Blue marks what the paper's own body reports. Both directions are reversed between the two panels. The Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405/588-nm experiments and the mechanism all match the right panel. Every word is from the paper or from the output of recompute.py.txt.

Grid of every passage that states a direction of calcium change, for UV-shifted and visible-shifted light

Figure 2. For each passage of the paper that states a direction, this grid gives the stated calcium change for UV-shifted light and for visible-shifted light. The abstract (orange, top row) is the only passage with UV → increase and visible → decrease. Six body passages (blue) agree with each other, including the plotted Fig. 4B traces and the mechanism. The Discussion row is a comparison between the two lights rather than a change from baseline.

[Internal, to be removed with candidates/ before sending: five candidate figure designs are in candidates/ for selection. candidate_1 = Figure 1, candidate_2 = Figure 2, candidate_3 = a sign dot plot on one axis, candidate_4 = the "respectively" pairing, candidate_5 = the paper's mechanism chain against the abstract's outcome.]

Why it matters

Explanations considered

  1. The abstract describes a different quantity. If the abstract meant hyperpolarization, photoproduct amount or ganglion-cell firing, the sign could legitimately differ. It says "calcium signal", the same quantity as the Results and legend (GCaMP6s ΔF/F). This does not account for the conflict.
  2. The abstract describes a different experiment or population. "Toward enhanced UV or visible wavelengths" matches the UV-mixed and visible-mixed white light of Fig. 4A/B. "Parapinopsin-sufficient but not parapinopsin-deficient" matches WT and PT-/- versus PP-/- and PP-/-/PT-/-. The only experiments that shift the spectrum of white light are Fig. 4B and its repeats at other intensities and durations in SI Appendix Fig. S7. This does not account for the conflict.
  3. "Respectively" can be read the other way. The abstract lists "UV or visible" and then "increased vs. decreased". The only grammatical pairing is UV with increased. The Results and legend use the same construction with the verbs in the opposite order. This does not account for the conflict.
  4. The abstract refers to a different time point or phase, such as what follows the stimulus. The plotted traces do show a later change. In WT and PT-/-, calcium dips at the UV-mixed bar, then rebounds above the pre-stimulus baseline and stays elevated until the visible-mixed bar. Taken alone, that rebound has the sign of the UV half of the abstract (UV → increased). It does not account for the abstract, for three reasons. (a) The Results and the Fig. 4 legend name the transient decrease as the response to UV-mixed light: "the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively". The legend adds that the relaxation of these responses was "not observed as a result of time-lapse recoding with low time resolution". (b) The Discussion describes "slow elevation of calcium levels after these levels were greatly decreased" under continuous blue or xenon white light (SI Appendix, Figs. S2 and S6). It relates this to "the kinetics of signaling-active photoproduct", which "could constitute a mechanism of light adaptation", and not to the spectral content of a stimulus. Applying that to the rebound in Fig. 4B is this write-up's inference, but it is the only account of such a rise that the paper offers. (c) The visible half of the abstract (visible → decreased) has no counterpart in the figure. After the visible-mixed peak, the traces return to about the level they had just before the visible-mixed bar, and no decrease follows. So even read as a later phase, the abstract's full mapping is not shown. This does not account for the conflict.
  5. The difference reflects total intensity rather than spectrum. Both mixed lights add photons (1.4% and 8.3% above the white light), so intensity cannot give opposite signs, and it does not bear on which sign is which. This does not account for the conflict.
  6. One Discussion sentence uses "increase". The Discussion speaks of "maximum changes in increase of calcium level in PP cells (10–20 s; SI Appendix, Fig. S7)". It is ambiguous and does not say which stimulus produced the increase. The SI text speaks only of "calcium decrease profiles in Fig. S7 (~20 s for reaching the maximum)" and does not name a stimulus; Fig. S7 repeats the UV- and visible-mixed stimuli, so linking that phrase to the UV-mixed responses is an inference. Neither passage supports the abstract's mapping.
  7. Text-conversion artifact. The conflict is in words, not numbers, and the same abstract wording appears in the Europe PMC XML, PubMed and Semantic Scholar. A lost minus sign, superscript or table cell cannot produce it. This does not account for the conflict.
  8. Already corrected. No correction or erratum was found (see Status).

None of these accounts for the reversal. The abstract's direction words for the Fig. 4B experiment are in the opposite order to the rest of the paper.

Status

As of 2026-10-23:

Suggested correction

In the Abstract, replace

Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.

with

Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively decreased vs. increased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.

This makes the abstract agree with the Results, the Fig. 4 legend, the plotted Fig. 4B data and the paper's proposed mechanism. No other part of the paper needs to change.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC6217433 (Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1802592115)
# Abstract vs Results / Fig. 4B: direction of the calcium response to UV- and visible-enriched white light
#
# Run:  python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates, in the folder that
# holds this file:
#   fig1.svg / fig1.png         two text panels: Abstract vs Results + Fig. 4 legend
#   fig2.svg / fig2.png         direction of the calcium change in every passage that states one
#   candidates/candidate_{1..5}.svg/.png   five candidate figure designs (1 = fig1, 2 = fig2), for selection
# Needs matplotlib (numpy only for the arithmetic).
#
# Inputs
#  * Quotes: copied verbatim from the article as archived in PubMed Central (PMC6217433).
#    When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
#    archived full text with src/show.py --grep (skipped elsewhere).
#  * Light intensities: Fig. 4 legend (A).
#  * Fig. 4B plotted traces: the direction of the WT and PT-/- deflections was read by eye from
#    the published figure image (pnas.1802592115fig04.jpg in the PMC / Europe PMC package).
#    No values are measured from it and the image is not reproduced. The later course of the
#    traces (rebound above baseline after the UV-mixed dip) was read the same way.

import os, re, subprocess, sys, textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
                     "axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": TXT,
                     "ytick.color": TXT, "svg.fonttype": "none"})

# ------------------------------------------------------------------ quotes
Q = {
    "abstract": ("Abstract",
        "Changes in the spectral composition of white light toward enhanced UV or visible wavelengths "
        "respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not "
        "parapinopsin-deficient photoreceptors."),
    "results4b": ("Results, 'Color Opponency Based on Photoequilibrium of Parapinopsin Two States', white-light paragraph",
        "We found decreased and increased calcium levels following exposure to UV and visible "
        "light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B)."),
    "legend4b": ("Fig. 4 legend, panel B",
        "Note that the calcium levels decreased and increased transiently in response to exposure to "
        "UV and visible light-mixed white lights, respectively, in WT and PT-/-."),
    "fig1c_405": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
        "stimulation with 405-nm light caused a decrease in fluorescence intensity."),
    "fig1c_588": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
        "In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating "
        "the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV "
        "and visible light (Fig. 1C)."),
    "mech_405": ("Results, mechanism paragraph (Fig. 3)",
        "The 405-nm light stimulus increases the short-wavelength component of incident light and thus "
        "increases the fractional concentration of the signaling active photoproduct from that produced "
        "by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease)."),
    "mech_588": ("Results, mechanism paragraph (Fig. 3)",
        "Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light "
        "stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, "
        "resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, "
        "calcium increase)."),
    "discussion": ("Discussion, first paragraph",
        "with lights biased toward UV driving greater hyperpolarization than those biased toward visible light."),
    "intro": ("Introduction, second paragraph",
        "with firing of these neurons suppressed by UV and enhanced by visible light"),
    "legend4a": ("Fig. 4 legend, panel A",
        "The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed "
        "white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 "
        "photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and "
        "∼7.6 × 1013 photons per cm2·s, respectively."),
    "legend4b_n": ("Fig. 4 legend, panel B",
        "Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed "
        "circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed "
        "triangle; n = 16)"),
}


def norm(s):
    return re.sub(r"\s+", " ", s).strip()


def check_quotes():
    repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
    show = os.path.join(repo, "src", "show.py")
    if not os.path.exists(show):
        print("[quote check skipped: src/show.py not found]")
        return
    py = "/home/ubuntu/papers-data/venv/bin/python"
    py = py if os.path.exists(py) else sys.executable
    print("=== Verbatim quote check against the PMC archive (src/show.py --grep) ===")
    for k, (loc, q) in Q.items():
        probe = q[:60]
        out = subprocess.run([py, show, "PMC6217433", "--grep", probe], cwd=repo,
                             capture_output=True, text=True).stdout
        ok = norm(q) in norm(out)
        print(f"  {'OK  ' if ok else 'MISS'} {k:11s} ({loc})")


check_quotes()

# ------------------------------------------------------------------ direction mapping
# 'respectively' pairs the i-th stimulus with the i-th direction word; both orders are
# extracted from the quoted sentences themselves.
def dirword(w):
    w = w.lower()
    return "increase" if w.startswith("increas") else "decrease"


m = re.search(r"toward enhanced (\w+) or (\w+) wavelengths respectively (\w+) vs\. (\w+) calcium",
              Q["abstract"][1])
ABS = {m.group(1): dirword(m.group(3)), m.group(2): dirword(m.group(4))}
m = re.search(r"found (\w+) and (\w+) calcium levels following exposure to (\w+) and (\w+) light-mixed",
              Q["results4b"][1])
RES = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
m = re.search(r"calcium levels (\w+) and (\w+) transiently in response to exposure to (\w+) and (\w+) light-mixed",
              Q["legend4b"][1])
LEG = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
M405 = "decrease" if "caused a decrease" in Q["fig1c_405"][1] else "?"
M588 = "increase" if "588-nm light elevated" in Q["fig1c_588"][1] else "?"
MECH405 = "decrease" if "calcium decrease" in Q["mech_405"][1] else "?"
MECH588 = "increase" if "calcium increase" in Q["mech_588"][1] else "?"
DISC = ("lower", "higher") if "biased toward UV driving greater hyperpolarization" in Q["discussion"][1] else ("?", "?")
FIG4B_IMAGE = ("decrease", "increase")  # transient deflection at each bar, read by eye from the published Fig. 4B image (WT, PT-/-)
FIG4B_LATER = {  # later course of the WT and PT-/- traces, read by eye from the same image
    "after UV-mixed dip": "rebound above the pre-stimulus baseline, elevated until the visible-mixed bar",
    "after visible-mixed peak": "back to about the level just before the visible-mixed bar; no decrease follows",
}

print("\n=== Direction of the calcium change stated for each stimulus ===")
print(f"  Abstract                         UV-enhanced: {ABS['UV']:8s}  visible-enhanced: {ABS['visible']}")
print(f"  Results, Fig. 4B paragraph       UV-mixed:    {RES['UV']:8s}  visible-mixed:    {RES['visible']}")
print(f"  Fig. 4 legend (B)                UV-mixed:    {LEG['UV']:8s}  visible-mixed:    {LEG['visible']}")
print(f"  Fig. 4B traces (published image) UV-mixed:    {FIG4B_IMAGE[0]:8s}  visible-mixed:    {FIG4B_IMAGE[1]}")
print(f"  Results, Fig. 1C (two-photon)    405 nm:      {M405:8s}  588 nm:           {M588}")
print(f"  Results, mechanism (Fig. 3)      405 nm:      {MECH405:8s}  588 nm:           {MECH588}")
print(f"  Discussion (relative)            UV-biased:   {DISC[0]:8s}  visible-biased:   {DISC[1]}  (calcium, via 'greater hyperpolarization')")
for k in ("UV", "visible"):
    print(f"  {k:8s} abstract == Results? {ABS[k] == RES[k]}   Results == legend? {RES[k] == LEG[k]}")
body = [(RES["UV"], RES["visible"]), (LEG["UV"], LEG["visible"]), FIG4B_IMAGE, (M405, M588), (MECH405, MECH588)]
n_agree = sum(1 for b in body if b == ("decrease", "increase"))
print(f"  Body statements with UV -> decrease and visible -> increase: {n_agree} of {len(body)}"
      f" (plus the Discussion's relative statement)")
print(f"  Abstract matches the body: {(ABS['UV'], ABS['visible']) == ('decrease', 'increase')}")
print("  Fig. 4B, later course of the WT and PT-/- traces (published image, by eye):")
for k, v in FIG4B_LATER.items():
    print(f"    {k:25s} {v}")
print("    -> the UV half of the abstract (UV -> increased) has a later counterpart (the rebound);")
print("       the visible half (visible -> decreased) has none, so the abstract's full mapping is not shown.")

# ------------------------------------------------------------------ light intensities (Fig. 4 legend A)
print("\n=== Fig. 4A light intensities (photons per cm2·s, 360-780 nm) ===")
white, uv_mix, vis_mix = 9.2e14, 9.3e14, 9.9e14
uv_led, vis_led = 1.3e13, 7.6e13
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
    tot = white + led
    print(f"  {lab:14s} white + LED = {tot:.3g} (reported ~{rep:.2g}); LED adds {led / white:.1%} to the white light")
# rounding check: each '~' value is taken as exact to its last printed digit (half a unit either way)
def rng(v, step):
    return v - step / 2, v + step / 2
w_lo, w_hi = rng(white, 0.1e14)
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
    l_lo, l_hi = rng(led, 0.1e13)
    r_lo, r_hi = rng(rep, 0.1e14)
    lo, hi = w_lo + l_lo, w_hi + l_hi
    print(f"  {lab:14s} sum within rounding: {lo:.3e}-{hi:.3e}; reported ~{rep:.2g} covers {r_lo:.3e}-{r_hi:.3e};"
          f" overlap: {lo <= r_hi and r_lo <= hi}")
print("  Both mixed lights add photons to the same white background (total intensity rises in both),")
print("  so the two stimuli differ in spectral composition, not in the sign of the intensity change.")
print("\n=== Fig. 4B sample sizes (legend; not affected by the direction issue) ===")
print("  WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16; total", 24 + 23 + 17 + 16)

# ------------------------------------------------------------------ figure helpers
def newfig(h):
    fig = plt.figure(figsize=(7.2, h), facecolor=BG)
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, 7.2); ax.set_ylim(0, h); ax.axis("off"); ax.set_facecolor(BG)
    return fig, ax


def save(fig, *names):
    for n in names:
        fig.savefig(n + ".svg", facecolor=BG)
        fig.savefig(n + ".png", dpi=200, facecolor=BG)
    plt.close(fig)


def panel(ax, x, y, w, h, color, title, sub, body, width_chars, fs=10.5):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
                                fc="white", ec=color, lw=2))
    ax.add_patch(FancyBboxPatch((x, y + h - 0.08), w, 0.08, boxstyle="square,pad=0", fc=color, ec=color))
    ax.text(x + 0.15, y + h - 0.27, title, fontsize=12, fontweight="bold", color=color, va="top")
    ax.text(x + 0.15, y + h - 0.55, sub, fontsize=10, color=SEC, va="top")
    ax.text(x + 0.15, y + h - 0.85, textwrap.fill(body, width_chars), fontsize=fs, va="top",
            linespacing=1.35)


# ------------------------------------------------------------------ candidate 1 / fig1: two text panels
def cand1():
    fig, ax = newfig(4.4)
    ax.text(0.15, 4.25, "Abstract and Results give opposite directions for the Fig. 4B experiment",
            fontsize=11.5, fontweight="bold", va="top")
    panel(ax, 0.15, 0.95, 3.25, 3.0, ORANGE, "Abstract", "as published",
          "“…white light toward enhanced UV or visible wavelengths respectively "
          "increased vs. decreased calcium signal in parapinopsin-sufficient … photoreceptors.”", 36)
    panel(ax, 3.8, 0.95, 3.25, 3.0, BLUE, "Results + Fig. 4 legend", "what the paper's body reports",
          "“We found decreased and increased calcium levels following exposure to UV and visible "
          "light-mixed white light, respectively, in WT and PT-/- … (Fig. 4B).”", 36)
    # mapping rows inside panels
    for x0, col, uvd, visd in ((0.3, ORANGE, ABS["UV"], ABS["visible"]), (3.95, BLUE, RES["UV"], RES["visible"])):
        for yy, lab, dd in ((1.55, "UV-enriched", uvd), (1.2, "visible-enriched", visd)):
            ax.text(x0, yy, lab, fontsize=10, color=col, fontweight="bold")
            ax.text(x0 + 1.34, yy, f"→ calcium {dd}d", fontsize=10, color=col, fontweight="bold")
    ax.add_patch(FancyArrowPatch((3.42, 1.48), (3.78, 1.48), arrowstyle="<->", mutation_scale=14,
                                 color=SEC, lw=1.5))
    ax.text(3.6, 0.72, "Conflict: both directions are reversed.", ha="center", fontsize=10.5,
            fontweight="bold", color=TXT)
    ax.text(3.6, 0.42, "The transient Fig. 4B responses, the 405/588-nm experiments (Fig. 1C) and the mechanism (Fig. 3)",
            ha="center", fontsize=10, color=SEC)
    ax.text(3.6, 0.17, "agree with the Results: UV-enriched light lowers calcium, visible-enriched light raises it.",
            ha="center", fontsize=10, color=SEC)
    return fig


save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "candidate_1"))

# ------------------------------------------------------------------ candidate 2 / fig2: direction grid
ROWS = [
    ("Abstract", "enhanced UV / enhanced visible white light", ABS["UV"], ABS["visible"], ORANGE),
    ("Results, Fig. 4B paragraph", "UV- / visible-mixed white light", RES["UV"], RES["visible"], BLUE),
    ("Fig. 4 legend, panel B", "UV- / visible-mixed white light", LEG["UV"], LEG["visible"], BLUE),
    ("Fig. 4B traces (WT, PT-/-)", "transient, from the published image", FIG4B_IMAGE[0], FIG4B_IMAGE[1], BLUE),
    ("Results, Fig. 1C", "405-nm / 588-nm stimulus, two-photon", M405, M588, BLUE),
    ("Results, mechanism (Fig. 3)", "405-nm / 588-nm stimulus", MECH405, MECH588, BLUE),
    ("Discussion", "UV-biased / visible-biased light", "lower*", "higher*", BLUE),
]
SYM = {"decrease": "▼ decrease", "increase": "▲ increase", "lower*": "▼ lower*", "higher*": "▲ higher*"}


def cand2():
    h = 0.95 + 0.48 * len(ROWS) + 0.75
    fig, ax = newfig(h)
    top = h - 0.25
    ax.text(0.15, top, "Direction of the calcium change in every passage that states one", fontsize=11.5,
            fontweight="bold", va="top")
    ax.text(3.95, top - 0.5, "UV-shifted light", ha="center", fontsize=10.5, fontweight="bold")
    ax.text(5.9, top - 0.5, "visible-shifted light", ha="center", fontsize=10.5, fontweight="bold")
    y = top - 0.95
    for i, (src, stim, d1, d2, col) in enumerate(ROWS):
        if i == 0:
            ax.add_patch(FancyBboxPatch((0.1, y - 0.2), 7.0, 0.44, boxstyle="round,pad=0.01,rounding_size=0.05",
                                        fc="#fdf0ea", ec=ORANGE, lw=2))
        else:
            ax.plot([0.15, 7.05], [y + 0.24, y + 0.24], color=GRID, lw=1)
        ax.text(0.2, y + 0.05, src, fontsize=10.5, fontweight="bold" if i == 0 else "normal", va="center", color=TXT)
        ax.text(0.2, y - 0.13, stim, fontsize=10, color=SEC, va="center")
        ax.text(3.95, y, SYM[d1], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
        ax.text(5.9, y, SYM[d2], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
        y -= 0.48
    ax.text(0.15, y + 0.05, "Orange: the abstract as published.  Blue: the paper's own body (text, legend, figure, mechanism).",
            fontsize=10, color=SEC, va="top")
    ax.text(0.15, y - 0.22, "* A comparison: UV-biased light drives “greater hyperpolarization”, i.e. lower calcium.",
            fontsize=10, color=SEC, va="top")
    return fig


save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "candidate_2"))


# ------------------------------------------------------------------ candidate 3: sign dot plot on one axis
def cand3():
    fig = plt.figure(figsize=(7.2, 3.9), facecolor=BG)
    ax = fig.add_axes([0.14, 0.2, 0.52, 0.66]); ax.set_facecolor(BG)
    sign = {"increase": 1, "decrease": -1}
    body_src = [("Results text", RES), ("Fig. 4 legend", LEG),
                ("Fig. 4B traces", {"UV": FIG4B_IMAGE[0], "visible": FIG4B_IMAGE[1]}),
                ("Fig. 1C 405/588 nm", {"UV": M405, "visible": M588}),
                ("Fig. 3 mechanism", {"UV": MECH405, "visible": MECH588})]
    for xi, k in enumerate(("UV", "visible")):
        ax.plot(xi - 0.18, sign[ABS[k]], "o", ms=11, color=ORANGE, zorder=3)
        for j, (_, d) in enumerate(body_src):
            ax.plot(xi + 0.04 + 0.11 * j, sign[d[k]], "s", ms=8, color=BLUE, zorder=3)
    ax.axhline(0, color=SEC, lw=0.8)
    ax.set_xlim(-0.55, 1.85); ax.set_ylim(-1.6, 1.6)
    ax.set_xticks([0.1, 1.1]); ax.set_xticklabels(["UV-shifted light", "visible-shifted light"])
    ax.set_yticks([-1, 0, 1]); ax.set_yticklabels(["calcium\ndecrease", "no change", "calcium\nincrease"])
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    fig.text(0.02, 0.95, "Sign of the calcium response: abstract vs the paper's body", fontsize=11.5, fontweight="bold")
    fig.text(0.69, 0.74, "● Abstract, as published", color=ORANGE, fontsize=10.5, fontweight="bold")
    fig.text(0.69, 0.66, "■ Paper's body,\n   5 passages:", color=BLUE, fontsize=10.5,
             fontweight="bold", va="top")
    fig.text(0.71, 0.53, "\n".join(n for n, _ in body_src), color=SEC, fontsize=10, va="top", linespacing=1.4)
    fig.text(0.02, 0.04, "Only the sign is plotted; the paper states directions, not a single summary value.",
             fontsize=10, color=SEC)
    return fig


save(cand3(), os.path.join(CAND, "candidate_3"))


# ------------------------------------------------------------------ candidate 4: 'respectively' pairing
def token(ax, x, y, s, col, w=1.55):
    ax.add_patch(FancyBboxPatch((x - w / 2, y - 0.16), w, 0.32, boxstyle="round,pad=0.02,rounding_size=0.06",
                                fc="white", ec=col, lw=2))
    ax.text(x, y, s, ha="center", va="center", fontsize=10.5, color=TXT)


def cand4():
    fig, ax = newfig(3.9)
    ax.text(0.15, 3.75, "How “respectively” pairs each light with a direction", fontsize=11.5,
            fontweight="bold", va="top")
    for x0, col, title, stims, verbs, mp in (
            (0.25, ORANGE, "Abstract", ("enhanced UV", "enhanced visible"), ("increased", "decreased"), ABS),
            (3.85, BLUE, "Results (Fig. 4B)", ("UV-mixed", "visible-mixed"), ("decreased", "increased"), RES)):
        ax.text(x0, 3.25, title, fontsize=12, fontweight="bold", color=col)
        ax.text(x0, 2.98, "word order in the sentence", fontsize=10, color=SEC)
        for i in range(2):
            xs = x0 + 0.85 + i * 1.65
            token(ax, xs, 2.5, stims[i], col)
            token(ax, xs, 1.55, verbs[i], col)
            ax.add_patch(FancyArrowPatch((xs, 2.32), (xs, 1.73), arrowstyle="-|>", mutation_scale=12, color=col, lw=2))
            ax.text(xs + 0.08, 2.02, f"{i + 1}st" if i == 0 else "2nd", fontsize=10, color=SEC)
        ax.text(x0, 1.0, f"UV → calcium {mp['UV']}", fontsize=10.5, color=col, fontweight="bold")
        ax.text(x0, 0.72, f"visible → calcium {mp['visible']}", fontsize=10.5, color=col, fontweight="bold")
    ax.plot([3.6, 3.6], [0.5, 3.4], color=GRID, lw=1)
    ax.text(3.6, 0.2, "Same pairing rule, opposite verb order: the two sentences assign opposite directions to each light.",
            ha="center", fontsize=10, color=SEC)
    return fig


save(cand4(), os.path.join(CAND, "candidate_4"))


# ------------------------------------------------------------------ candidate 5: mechanism chain
def box(ax, x, y, s, col, w=1.45, h=0.62, fc="white"):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06", fc=fc, ec=col, lw=2))
    ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=10, color=TXT, linespacing=1.25)


def cand5():
    fig, ax = newfig(4.3)
    ax.text(0.15, 4.15, "The paper's mechanism predicts the Results' direction, not the abstract's",
            fontsize=11.5, fontweight="bold", va="top")
    ax.text(0.15, 3.82, "Chain from the Results (mechanism paragraph, Fig. 3, 405/588 nm), generalized in the Discussion", fontsize=10,
            color=SEC, va="top")
    rows = ((2.75, "UV-shifted\nwhite light", "more active\nphotoproduct", "hyperpolarization", "calcium\ndecreases",
             f"calcium\n{ABS['UV']}d"),
            (1.45, "visible-shifted\nwhite light", "less active\nphotoproduct", "hyperpolarization\nends",
             "calcium\nincreases", f"calcium\n{ABS['visible']}d"))
    for y, a, b, c, d, e in rows:
        xs = [0.15, 1.6, 3.05, 4.5]
        for x, s in zip(xs, (a, b, c, d)):
            box(ax, x, y, s, BLUE if x == 4.5 else SEC, w=1.3, fc="#eef4fc" if x == 4.5 else "white")
        for x in xs[:-1]:
            ax.add_patch(FancyArrowPatch((x + 1.32, y + 0.31), (x + 1.44, y + 0.31), arrowstyle="-|>",
                                         mutation_scale=12, color=SEC, lw=1.5))
        box(ax, 6.1, y, e, ORANGE, w=0.98, fc="#fdf0ea")
        ax.text(5.95, y + 0.31, "≠", ha="center", va="center", fontsize=16, color=TXT, fontweight="bold")
    ax.text(5.15, 3.5, "Results / Fig. 4B", ha="center", fontsize=10, color=BLUE, fontweight="bold")
    ax.text(6.59, 3.5, "Abstract", ha="center", fontsize=10, color=ORANGE, fontweight="bold")
    ax.text(0.15, 1.0, "Blue: the outcome the paper's body reports, which follows from its mechanism.", fontsize=10, color=SEC)
    ax.text(0.15, 0.72, "Orange: the outcome the abstract states, which reverses it for both lights.", fontsize=10, color=SEC)
    ax.text(0.15, 0.44, "Same two lights, same calcium indicator (GCaMP6s), same parapinopsin-sufficient cells (WT, PT-/-).",
            fontsize=10, color=SEC)
    return fig


save(cand5(), os.path.join(CAND, "candidate_5"))
print("\nFigures written to", HERE, "and", CAND)

Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts

Sci Adv (2020) · doi:10.1126/sciadv.abb8930 · PMC10662364

Summary

The Results report that offspring of a single N. pachyderma parent coiled sinistrally more often in the 12-hour light flask than in the 24-hour dark flask: 87.5% (n = 9) against 50% (n = 12), with P < 0.0001 by a proportion z test. The counts that fit the paper's own percentages and totals are 7 of 8 and 6 of 12. They add up to the stated 13 of 20 sinistral offspring, and no count out of 9 gives 87.5%. A two-proportion z test on these counts gives z = 1.72 and two-sided P = 0.085, and Fisher's exact test gives P = 0.158. None of the standard variants I tried comes close to 0.0001. The Discussion relies on this contrast when it suggests that light conditions, or the pH and O2 changes that come with them, could contribute to coiling direction. That inference loses its statistical support, but the paper's main observations of asexual reproduction and morphological variability among clones do not depend on it.

What the paper reports

Results, "Morphology of parent and offspring", second paragraph:

Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%). There were differences in coiling direction proportion between the 12-hour light and 24-hour dark treatments (see Materials and Methods): 87.5% were sinistral (n = 9) in the 12-hour light treatment (flask “O”), and 50% (n = 12) in the 24-hour dark treatment (flask “D”) (P < 0.0001, by a proportion z test).

Materials and Methods, third paragraph:

No shells we recovered from flask “A,” eight offspring and the parent were recovered from flask “O,” and 17 offspring from flask “D.”

Discussion, "Phenotypic variation in a genetically similar population", second paragraph:

The relative abundance of shells coiling in the nondominant direction was greater in the 24-hour dark treatment (50%) than in the 12-hour light treatment (12.5%), indicating that factors such as light conditions or associated changes in pH and O2 could contribute to coiling direction.

What the paper's own numbers show

Method. I first found the whole-number counts consistent with each printed percentage, its printed n, the Methods recovery counts and the stated total of 13 sinistral out of 20. I then ran the two-group test the paper names (a two-proportion z test with pooled standard error, two-sided) on those counts, along with the common variants: unpooled standard error, continuity correction, one-sided, Fisher's exact test, Boschloo's exact test, and the same tests with the sinistral parent counted as a ninth shell in flask O. Last, I searched for any calculation that yields P < 0.0001. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Flask O, sinistral Results 87.5% (n = 9) No count out of 9 gives 87.5% (87.5% × 9 = 7.875; 7/9 = 77.8%, 8/9 = 88.9%). 7/8 = 87.5%, and Methods report 8 offspring (plus the parent) from flask O
Flask D, sinistral Results 50% (n = 12) 6/12 = 50%; 12 of the 17 offspring recovered from flask D
All offspring scored Results 13 of 20 sinistral (65%) 7 + 6 = 13, 8 + 12 = 20, 13/20 = 65%. With n = 9 the total would be 21
Nondominant (dextral) coiling Discussion 12.5% vs 50% 1/8 = 12.5%, 6/12 = 50%: the same contrast as above
Pooled two-proportion z test, 7/8 vs 6/12 Results P < 0.0001 pooled p = 0.65, SE = 0.2177, z = 1.723, two-sided P = 0.085 (one-sided 0.043)
Same, continuity corrected Results P < 0.0001 z = 1.244, P = 0.213 (equal to chi-square with Yates correction)
Unpooled (Wald) z test Results P < 0.0001 z = 2.019, P = 0.044
Fisher's exact test Results P < 0.0001 P = 0.158 (odds ratio 7.0)
Boschloo's exact test Results P < 0.0001 P = 0.119
Parent counted, 8/9 vs 6/12 Results P < 0.0001 pooled z = 1.871, P = 0.061; unpooled z = 2.181, P = 0.029; Fisher P = 0.159
95% intervals (derived) — Wilson: flask O 52.9% to 97.8%, flask D 25.4% to 74.6%; Newcombe for the 37.5-point difference: −5.0 to 64.2 points, which includes 0
Equal group size needed for P < 0.0001 at 87.5% vs 50% (derived) n = 9 and n = 12 47 per group (pooled z test)
One-sample z test, flask D against flask O's estimate (candidate explanation) P < 0.0001 6/12 against a fixed 0.875: z = −3.93, P = 8.6e-5 (exact binomial P = 0.0018), but this is not a two-group comparison
Percentages entered as counts out of 100 (candidate explanation) P < 0.0001 87.5/100 vs 50/100: z = 5.72, P = 1.1e-8, but this is not a valid test of these data

Figure

Two-sided P values for flask O versus flask D on a log axis: the reported P < 0.0001 against values recomputed from 7/8 and 6/12 sinistral offspring

Figure 1. Two-sided P values for the light-versus-dark comparison of sinistral coiling, on a logarithmic axis. The orange bar and arrow mark the paper's reported value, P < 0.0001, which is an upper bound. The filled blue dots are P values recomputed from the counts that fit the paper's percentages and totals (7 of 8 versus 6 of 12, or 8 of 9 if the sinistral parent is counted). They range from 0.044 to 0.213 and lie near or above the dashed 0.05 line, more than 400 times the reported bound. The two hollow blue dots are the only calculations found that give a value below 0.0001: a one-sample z test of flask D's 6 of 12 against flask O's 87.5% treated as a fixed value (P = 8.6e-5), and the two percentages treated as counts out of 100 (P = 1.1e-8). They are shown only for comparison; neither is a valid two-group comparison of these data. All values come from recompute.py.txt.

Why it matters

The P value attaches to a secondary comparison, and the direction of the observed difference stays as reported: 1 of 8 dextral shells in the light flask against 6 of 12 in the dark flask. What changes is the strength of the evidence. As printed, P < 0.0001 tells the reader that a difference this large is very unlikely to arise by chance between two groups that coil alike. With 8 and 12 shells, the correct pooled z test gives P = 0.085, and the 95% interval for the difference runs from about −5 to 64 percentage points. The data are compatible with no effect of the treatment on coiling.

The Discussion uses this contrast to suggest that "light conditions or associated changes in pH and O2 could contribute to coiling direction". That is presented as a possibility, but readers may take the printed P < 0.0001 as strong support for it. The paper notes that the two species are "generally distinguished by shell coiling direction", so a claimed environmental effect on coiling bears on how field specimens are assigned to species. The comparison involves one parent's clonal offspring in two flasks, so even a significant result would not separate light from any other difference between the flasks, and the recomputed P values (0.03 to 0.21, two-sided) leave it without strong statistical support.

The paper's headline findings do not rest on this test: that the parent reproduced asexually, that its clonal offspring spanned four morphotypes, and that offspring coiled in both directions (13 of 20 sinistral). The title and abstract do not report the light-versus-dark P value, and the paper has no tables.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

In Results, "Morphology of parent and offspring", second paragraph, give the flask O denominator that matches 87.5% (n = 8 offspring), or, if n = 9 includes the parent, state that and give the matching percentage (88.9%). Replace "P < 0.0001, by a proportion z test" with the value for the counts actually compared, for example "7 of 8 versus 6 of 12; P = 0.085, two-sided two-proportion z test (Fisher's exact test, P = 0.16)", and state that the difference is not statistically significant. In the Discussion, "Phenotypic variation in a genetically similar population", second paragraph, the sentence on light conditions and coiling direction could be reworded to present the 12.5% versus 50% contrast as a small-sample observation from two flasks that was not statistically significant.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- re-derivation for PMC10662364 (Sci Adv 2020, doi:10.1126/sciadv.abb8930)
# Run:  python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.svg/.png
# plus five candidate figure designs (candidates/fig1_A..E) in this script's folder.
#
# Inputs, all taken from the article text:
#   Results:  "Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%)."
#   Results:  "87.5% were sinistral (n = 9) in the 12-hour light treatment (flask "O"), and 50% (n = 12)
#              in the 24-hour dark treatment (flask "D") (P < 0.0001, by a proportion z test)."
#   Methods:  "eight offspring and the parent were recovered from flask "O," and 17 offspring from flask "D.""
#   Discussion: nondominant coiling 50% (dark) vs 12.5% (light).
import os
import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- 1. counts
print("== 1. Which counts fit the printed percentages")
for k in range(0, 10):
    print(f"  {k}/9 = {k/9:.4f}", "<- 87.5%?" if abs(k/9 - 0.875) < 1e-9 else "")
print(f"  87.5% of 9 = {0.875*9:.3f} (not a whole number)")
print(f"  7/8 = {7/8:.4f}; 8/9 = {8/9:.4f}; 7/9 = {7/9:.4f}; 6/12 = {6/12:.4f}")
print(f"  offspring total 8 + 12 = {8+12}; sinistral 7 + 6 = {7+6}; 13/20 = {13/20:.4f} (paper: 13 of 20, 65%)")
print(f"  with n = 9 the total would be 9 + 12 = {9+12}, not 20")
print(f"  nondominant (dextral): 1/8 = {1/8:.4f}, 6/12 = {6/12:.4f} (paper: 12.5% and 50%)")
print(f"  flask D: 12 scored for coiling of 17 offspring recovered (Methods)")

xO, nO, xD, nD = 7, 8, 6, 12


# ---------------------------------------------------------------- 2. tests
def z2(x1, n1, x2, n2, pooled=True, cc=False):
    p1, p2 = x1 / n1, x2 / n2
    if pooled:
        p = (x1 + x2) / (n1 + n2)
        se = np.sqrt(p * (1 - p) * (1 / n1 + 1 / n2))
    else:
        se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
    d = abs(p1 - p2)
    if cc:
        d = max(0.0, d - 0.5 * (1 / n1 + 1 / n2))
    z = d / se
    return z, 2 * stats.norm.sf(z), se


print("\n== 2. Two-group tests, flask O vs flask D")
p_pool = (xO + xD) / (nO + nD)
z, p, se = z2(xO, nO, xD, nD)
print(f"  pooled p = {p_pool:.3f}; SE = sqrt({p_pool:.2f}*{1-p_pool:.2f}*(1/8+1/12)) = {se:.4f}")
print(f"  difference = {7/8 - 6/12:.3f}")
R = {}
for lab, (a, b, c, d) in {"7/8 vs 6/12": (7, 8, 6, 12), "8/9 vs 6/12 (parent counted)": (8, 9, 6, 12)}.items():
    for pooled in (True, False):
        for cc in (False, True):
            z, p, _ = z2(a, b, c, d, pooled, cc)
            R[(lab, pooled, cc)] = (z, p)
            print(f"  {lab:30s} {'pooled  ' if pooled else 'unpooled'} {'Yates' if cc else 'no cc':5s}"
                  f"  z = {z:.3f}  two-sided P = {p:.4f}  one-sided P = {p/2:.4f}")
for a, b, c, d in [(7, 8, 6, 12), (8, 9, 6, 12)]:
    tab = [[a, b - a], [c, d - c]]
    fe = stats.fisher_exact(tab)
    chy = stats.chi2_contingency(tab, correction=True)
    chn = stats.chi2_contingency(tab, correction=False)
    bo = stats.boschloo_exact(tab, alternative="two-sided")
    R[("fisher", a, b)] = fe.pvalue
    R[("yates", a, b)] = chy[1]
    R[("boschloo", a, b)] = bo.pvalue
    print(f"  {a}/{b} vs {c}/{d}: Fisher exact P = {fe.pvalue:.4f} (OR = {fe.statistic:.1f}); "
          f"chi-square Yates = {chy[0]:.3f}, P = {chy[1]:.4f}; no Yates = {chn[0]:.3f}, P = {chn[1]:.4f}; "
          f"Boschloo exact P = {bo.pvalue:.4f}")
    print(f"    (uncorrected chi-square P equals the pooled z test P: "
          f"{chn[1]:.4f} vs {z2(a, b, c, d)[1]:.4f})")


# Wilson intervals and Newcombe hybrid-score interval for the difference
def wilson(x, n, conf=0.95):
    zc = stats.norm.ppf(1 - (1 - conf) / 2)
    ph = x / n
    den = 1 + zc ** 2 / n
    cen = (ph + zc ** 2 / (2 * n)) / den
    half = zc * np.sqrt(ph * (1 - ph) / n + zc ** 2 / (4 * n * n)) / den
    return cen - half, cen + half


lO, uO = wilson(xO, nO)
lD, uD = wilson(xD, nD)
dd = xO / nO - xD / nD
dl = dd - np.sqrt((xO / nO - lO) ** 2 + (uD - xD / nD) ** 2)
du = dd + np.sqrt((uO - xO / nO) ** 2 + (xD / nD - lD) ** 2)
print(f"  Wilson 95% CI flask O 7/8: {lO*100:.1f}% to {uO*100:.1f}%")
print(f"  Wilson 95% CI flask D 6/12: {lD*100:.1f}% to {uD*100:.1f}%")
print(f"  Newcombe 95% CI for the difference 37.5 points: {dl*100:.1f} to {du*100:.1f} points (includes 0)")

# ---------------------------------------------------------------- 3. routes to P < 0.0001
print("\n== 3. Calculations that would give P < 0.0001")
z_pc, p_pc, _ = z2(87.5, 100, 50, 100)
print(f"  percentages treated as counts out of 100 (87.5/100 vs 50/100): z = {z_pc:.2f}, P = {p_pc:.1e}")
z1 = (0.5 - 0.875) / np.sqrt(0.875 * 0.125 / 12)
p1s = 2 * stats.norm.sf(abs(z1))
pb = stats.binomtest(6, 12, 0.875).pvalue
print(f"  one-sample z, 6/12 against a fixed 0.875: z = {z1:.2f}, P = {p1s:.1e}; exact binomial P = {pb:.4f}")
z1b = (0.875 - 0.5) / np.sqrt(0.5 * 0.5 / 8)
p1b = 2 * stats.norm.sf(z1b)
pbb = stats.binomtest(7, 8, 0.5).pvalue
print(f"  one-sample z, 7/8 against a fixed 0.5: z = {z1b:.2f}, P = {p1b:.4f}; exact binomial P = {pbb:.4f}")
print("  (neither one-sample route is a two-group comparison: each treats one flask's estimate as a known constant)")
need = None
curve_n = np.arange(4, 121)
curve_p = []
for n in curve_n:
    zz, pp, _ = z2(0.875 * n, n, 0.5 * n, n)
    curve_p.append(pp)
    if need is None and pp < 1e-4:
        need = n
curve_p = np.array(curve_p)
print(f"  equal groups needed for pooled-z P < 1e-4 at 87.5% vs 50%: n = {need} per group "
      f"(P = {z2(0.875*need, need, 0.5*need, need)[1]:.1e}); at n = {need-1}: "
      f"P = {z2(0.875*(need-1), need-1, 0.5*(need-1), need-1)[1]:.1e}")
print(f"  at n = 8 and 12 (actual): P = {R[('7/8 vs 6/12', True, False)][1]:.3f}")
print("  (different question) nondominant coiling against the 3% population upper bound, exact binomial, one-sided:")
for x, n, lab in [(1, 8, "flask O"), (6, 12, "flask D"), (7, 20, "all offspring")]:
    print(f"    {lab}: {x}/{n}, P = {stats.binomtest(x, n, 0.03, alternative='greater').pvalue:.2e}")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2, "axes.labelcolor": INK,
    "xtick.color": INK2, "ytick.color": INK2, "text.color": INK, "axes.linewidth": 0.8,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
    "lines.linewidth": 1.5, "lines.markersize": 8,
})
W = 7.2


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def clean(ax, xgrid=True, ygrid=False):
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.grid(False)
    if xgrid:
        ax.xaxis.grid(True, color=GRID, lw=0.8)
    if ygrid:
        ax.yaxis.grid(True, color=GRID, lw=0.8)
    ax.set_axisbelow(True)


# ---- Candidate A: P values on one log axis
def fmt_small(p):
    m, e = f"{p:.1e}".split("e")
    return f"P = {m}e{int(e)}"


def fig_A(path):
    rows = [
        ("Reported in the paper\n(\"proportion z test\")", None, "rep"),
        ("Pooled two-proportion z test", R[("7/8 vs 6/12", True, False)][1], "ok"),
        ("Unpooled (Wald) z test", R[("7/8 vs 6/12", False, False)][1], "ok"),
        ("Pooled z test, parent counted (8/9)", R[("8/9 vs 6/12 (parent counted)", True, False)][1], "ok"),
        ("Fisher exact test", R[("fisher", 7, 8)], "ok"),
        ("Chi-square with Yates correction", R[("yates", 7, 8)], "ok"),
        ("One-sample z test, 6/12 against a\nfixed 0.875 (not a two-group test)", p1s, "bad"),
        ("Percentages entered as counts\nout of 100 (not a valid test)", p_pc, "bad"),
    ]
    fig, ax = plt.subplots(figsize=(W, 5.3))
    clean(ax)
    ax.set_xscale("log")
    ax.set_xlim(1e-9, 1)
    ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
    ys = np.arange(len(rows))[::-1]
    for y, (lab, p, kind) in zip(ys, rows):
        if kind == "rep":
            ax.annotate("", xy=(3e-9, y), xytext=(1e-4, y),
                        arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
            ax.plot([1e-4], [y], marker="|", ms=14, mew=2, color=ORANGE)
            ax.text(1e-6, y + 0.18, "P < 0.0001", color=INK, va="bottom", ha="center", fontsize=10.5)
        elif kind == "ok":
            ax.plot([p], [y], "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
            ax.text(p * 1.9, y, f"P = {p:.3f}", va="center", ha="left", fontsize=10.5, color=INK)
        else:
            ax.plot([p], [y], "o", mfc=BG, mec=BLUE, mew=2, ms=9, zorder=3)
            ax.text(p * 2.6, y, fmt_small(p), va="center", ha="left", fontsize=10.5, color=INK2)
    ax.text(0.05, -0.55, " 0.05", color=INK2, ha="left", va="bottom", fontsize=10)
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows], fontsize=10.5)
    ax.set_ylim(-0.6, len(rows) - 0.3)
    ax.set_xticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
    ax.set_xticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
    ax.set_xlabel("Two-sided P value, log scale; flask O vs flask D")
    h = [Line2D([], [], color=ORANGE, marker="|", ms=12, mew=2, lw=2, label="Reported (upper bound)"),
         Line2D([], [], color=BLUE, marker="o", ls="none", ms=8, label="Recomputed from paper's counts"),
         Line2D([], [], mfc=BG, mec=BLUE, marker="o", ls="none", ms=8, mew=2, label="Recomputed, invalid")]
    fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=3, frameon=False,
               fontsize=10, handletextpad=0.4, columnspacing=1.2)
    fig.tight_layout(rect=(0, 0.07, 1, 1))
    save(fig, path)


# ---- Candidate B: proportions with 95% CIs and the difference, one percentage axis
def fig_B(path):
    fig, ax = plt.subplots(figsize=(W, 4.0))
    clean(ax)
    rows = [("Flask O, 12-h light\n7 of 8 sinistral", 87.5, (lO * 100, uO * 100)),
            ("Flask D, 24-h dark\n6 of 12 sinistral", 50.0, (lD * 100, uD * 100)),
            ("Difference, O minus D\n(percentage points)", 37.5, (dl * 100, du * 100))]
    ys = [2, 1, 0]
    for y, (lab, v, (lo, hi)) in zip(ys, rows):
        ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="round")
        ax.plot([v], [y], "o", color=ORANGE if y > 0 else BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
        ax.text(hi + 2, y, f"{v:.1f}  [{lo:.1f} to {hi:.1f}]", va="center", fontsize=10.5)
    ax.axvline(0, color=INK2, lw=0.8)
    ax.text(1, -0.55, "no difference", color=INK2, fontsize=10, va="center")
    ax.set_yticks(ys)
    ax.set_yticklabels([r[0] for r in rows])
    ax.set_xlim(-20, 130)
    ax.set_xticks([-20, 0, 20, 40, 60, 80, 100])
    ax.set_ylim(-0.8, 2.5)
    ax.set_xlabel("Percent sinistral (rows 1-2); difference in percentage points (row 3)")
    h = [Line2D([], [], color=ORANGE, marker="o", ls="none", ms=8, label="Reported percentage"),
         Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed difference and 95% intervals")]
    fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
    fig.suptitle("Paper: P < 0.0001. The 95%% interval for the difference includes 0\n"
                 "(pooled z test P = %.3f; Wilson intervals, Newcombe interval for the difference)"
                 % R[("7/8 vs 6/12", True, False)][1], fontsize=10.5, x=0.02, ha="left", color=INK)
    fig.tight_layout(rect=(0, 0.08, 1, 1))
    save(fig, path)


# ---- Candidate C: calculation-flow diagram
def fig_C(path):
    fig, ax = plt.subplots(figsize=(W, 5.8))
    ax.set_xlim(0, 100)
    ax.set_ylim(0, 100)
    ax.axis("off")

    def box(x, y, w, h, title, body, edge):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.5",
                                    fc=BG, ec=edge, lw=1.5))
        ax.text(x + 1.5, y + h - 2.5, title, fontsize=10, color=INK2, va="top", fontweight="bold")
        ax.text(x + 1.5, y + h - (15 if "\n" in title else 10), body, fontsize=10, color=INK, va="top", linespacing=1.4)

    def arrow(x0, y0, x1, y1):
        ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14, color=INK2, lw=1.5))

    pz = R[("7/8 vs 6/12", True, False)]
    box(1, 54, 29, 42, "1  PRINTED",
        "Flask O: 87.5%\n  sinistral, n = 9\nFlask D: 50%\n  sinistral, n = 12\nAll: 13 of 20 (65%)", ORANGE)
    box(35.5, 54, 29, 42, "2  FITTING COUNTS",
        f"87.5% x 9 = {0.875*9:.3f}\n  (not whole)\nO: 7 of 8\n  (Methods: 8\n  offspring + parent)\nD: 6 of 12\n7+6 = 13, 8+12 = 20", BLUE)
    box(70, 54, 29, 42, "3  RECOMPUTED",
        f"Pooled z = {pz[0]:.2f}\n  P = {pz[1]:.3f}\nFisher exact\n  P = {R[('fisher', 7, 8)]:.3f}\n"
        f"Parent counted\n  (8/9), P = {R[('8/9 vs 6/12 (parent counted)', True, False)][1]:.3f}", BLUE)
    box(1, 6, 29, 38, "4  PRINTED", "P < 0.0001,\n\"by a proportion\n  z test\"", ORANGE)
    box(35.5, 6, 63.5, 38, "5  ROUTES TO P < 0.0001 (NEITHER VALID)",
        f"a) One-sample z test, 6/12 against a fixed 0.875:\n"
        f"    z = {z1:.2f}, {fmt_small(p1s)} (exact binomial P = {pb:.4f})\n"
        f"b) Percentages entered as counts out of 100:\n"
        f"    87.5/100 vs 50/100: z = {z_pc:.2f}, {fmt_small(p_pc)}\n"
        f"A valid z test would need about {need} shells per group.", BLUE)
    arrow(30.6, 75, 35.0, 75)
    arrow(65.1, 75, 69.5, 75)
    arrow(30.6, 25, 35.0, 25)
    ax.plot([15.5, 15.5], [45, 53], color=INK2, lw=0.8, ls=":")
    ax.text(16.5, 49, "same sentence", color=INK2, fontsize=10, va="center")
    h = [Line2D([], [], color=ORANGE, lw=2, label="Box holds values printed in the paper"),
         Line2D([], [], color=BLUE, lw=2, label="Box holds values recomputed from them")]
    ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.09), ncol=2, frameon=False, fontsize=10)
    fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.08)
    save(fig, path)


# ---- Candidate D: P value against group size at 87.5% vs 50%
def fig_D(path):
    fig, ax = plt.subplots(figsize=(W, 3.9))
    clean(ax, xgrid=False, ygrid=True)
    ax.set_yscale("log")
    ax.plot(curve_n, curve_p, color=BLUE, lw=2, label="Pooled z test, 87.5% vs 50%, n per group")
    pa = R[("7/8 vs 6/12", True, False)][1]
    neq = 2 / (1 / 8 + 1 / 12)
    ax.plot([neq], [pa], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
    ax.annotate(f"Actual groups, 8 and 12 (plotted at their\nequivalent equal size, {neq:.1f}): P = {pa:.3f}",
                xy=(neq, pa), xytext=(18, 0.2), fontsize=10.5,
                arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    pn = z2(0.875 * need, need, 0.5 * need, need)[1]
    ax.plot([need], [pn], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
    ax.annotate(f"P < 0.0001 first reached\nat n = {need} per group", xy=(need, pn), xytext=(need + 10, 3e-3),
                fontsize=10.5, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    p100 = z2(87.5, 100, 50, 100)[1]
    ax.plot([100], [p100], "o", mfc=BG, mec=BLUE, mew=2, ms=9)
    ax.annotate("n = 100: percentages\nread as counts (invalid)", xy=(100, p100), xytext=(72, 4e-7), fontsize=10.5,
                color=INK2, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    ax.fill_between([0, 125], 1e-10, 1e-4, color=ORANGE, alpha=0.10, lw=0)
    ax.axhline(1e-4, color=ORANGE, lw=1.5)
    ax.text(122, 1.4e-4, "Reported: P < 0.0001 (shaded region)", color=INK, ha="right", va="bottom", fontsize=10.5)
    ax.axhline(0.05, color=INK2, lw=0.8, ls="--")
    ax.text(122, 0.058, "0.05", color=INK2, ha="right", va="bottom", fontsize=10)
    ax.set_xlim(0, 125)
    ax.set_ylim(1e-9, 1)
    ax.set_yticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
    ax.set_yticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
    ax.set_xlabel("Shells scored per treatment group (equal groups)")
    ax.set_ylabel("Two-sided P value (log scale)")
    h = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed (paper's percentages, varying n)"),
         Line2D([], [], color=ORANGE, lw=1.5, label="Reported threshold")]
    ax.legend(handles=h, loc="lower left", frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


# ---- Candidate E: every scored shell as a marker
def fig_E(path):
    fig, ax = plt.subplots(figsize=(W, 3.6))
    ax.set_xlim(0, 30)
    ax.set_ylim(-2.3, 3.4)
    ax.axis("off")
    x0, dx = 7.6, 0.78
    rows = [(2.0, "Flask O, 12-h light", 7, 1, "7 of 8 = 87.5%\n(printed: 87.5%, n = 9)"),
            (0.6, "Flask D, 24-h dark", 6, 6, "6 of 12 = 50%\n(printed: 50%, n = 12)")]
    for y, lab, s_, d_, txt in rows:
        ax.text(0.2, y, lab, ha="left", va="center", fontsize=10.5)
        for i in range(s_ + d_):
            x = x0 + i * dx
            if i < s_:
                ax.plot([x], [y], "o", ms=11, color=INK2)
            else:
                ax.plot([x], [y], "o", ms=11, mfc=BG, mec=INK2, mew=1.8)
        ax.text(x0 + 12 * dx + 0.4, y, txt, va="center", fontsize=10.5, color=INK, linespacing=1.3)
    ax.plot([x0 + 8 * dx], [2.0], "s", ms=11, mfc=BG, mec=INK2, mew=1.2, ls=(0, (2, 1)))
    ax.text(x0 + 8 * dx, 2.5, "9th?", ha="center", fontsize=10, color=INK2)
    ax.plot([0.5], [-0.75], "s", ms=10, color=ORANGE)
    ax.text(1.2, -0.75, "Printed:  P < 0.0001, \"by a proportion z test\"", fontsize=10.5, color=INK, va="center")
    ax.plot([0.5], [-1.5], "o", ms=10, color=BLUE)
    ax.text(1.2, -1.5, f"Recomputed from these counts:  pooled z test P = {R[('7/8 vs 6/12', True, False)][1]:.3f};"
            f"  Fisher exact P = {R[('fisher', 7, 8)]:.3f}", fontsize=10.5, color=INK, va="center")
    h = [Line2D([], [], marker="o", ls="none", ms=10, color=INK2, label="sinistral offspring"),
         Line2D([], [], marker="o", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.8, label="dextral offspring"),
         Line2D([], [], marker="s", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.2,
                label="shell implied by n = 9 (the parent?)")]
    ax.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 1.04), ncol=3, frameon=False, fontsize=10,
              handletextpad=0.3, columnspacing=1.0)
    fig.tight_layout()
    save(fig, path)


for name, fn in zip("ABCDE", (fig_A, fig_B, fig_C, fig_D, fig_E)):
    fn(os.path.join(CAND, f"fig1_{name}"))
fig_A(os.path.join(HERE, "fig1"))  # provisional choice; swap after design review

# contact sheet of the five candidates, for the design review
import matplotlib.image as mpimg
titles = {"A": "P values on one log axis (dot plot)", "B": "Proportions and difference with 95% intervals",
          "C": "Calculation-flow diagram", "D": "P value vs group size at 87.5% vs 50%",
          "E": "Every scored shell as a marker"}
imgs = {k: mpimg.imread(os.path.join(CAND, f"fig1_{k}.png")) for k in "ABCDE"}
hs = [imgs[k].shape[0] / imgs[k].shape[1] for k in "ABCDE"]
fig = plt.figure(figsize=(W, W * (sum(hs) + 5 * 0.06)))
gs = fig.add_gridspec(5, 1, height_ratios=hs, hspace=0.12)
for i, k in enumerate("ABCDE"):
    ax = fig.add_subplot(gs[i])
    ax.imshow(imgs[k])
    ax.axis("off")
    ax.set_title(f"{k}  {titles[k]}", loc="left", fontsize=11, fontweight="bold", color=INK)
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=120, bbox_inches="tight")
plt.close(fig)
print("\nFigures written: fig1.svg/.png (= candidate A) and candidates/fig1_A..E .svg/.png")

Abstract says the mAb reduces neutrophil apoptosis; the body and Fig. 3 show an increase

Sci Adv (2021) · doi:10.1126/sciadv.abd1453 · PMC11057782

Summary

The abstract says the agonist anti-ChemR23 antibody "reduces neutrophil apoptosis at the site of inflammation". Every other statement in the paper about this effect says the opposite. The Results say the antibody "accelerated neutrophil apoptotic cell death", with more Caspase-3/7 activation and annexin V staining. The safety paragraph's premise is that it "accelerates neutrophil apoptosis". The Discussion reports "increased apoptosis of remaining neutrophils" at the inflammatory site. The Fig. 3 F–H data also show more apoptotic or dead neutrophils with the antibody than with the isotype control, in vitro and in mouse air-pouch exudates. The data and the headline conclusions are not affected. But the abstract, which is the part most readers and text-mining tools see, states one of the paper's two cellular mechanisms backwards.

What the paper reports

Abstract, fourth sentence:

We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.

Abstract, first sentence. This sentence sets out the effects that proresolving agonists are expected to have:

Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.

Results, "Agonist anti-ChemR23 mAb as new proresolving therapeutic modality", paragraph 1:

In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death (Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V staining (Fig. 3F and fig. S10E).

Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS production (fig. S10D).

Results, "ChemR23 activation with agonist mAb triggers resolution of acute and chronic intestinal inflammation", paragraph 4 (the safety assessment):

Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo effects on the peripheral immune cell population.

Discussion, paragraph 3:

We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, including in promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis.

However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.

Fig. 3 legend, panels F–H:

(F) Percentage of Caspase-3/7 human neutrophils determined by microcopy after 10 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 3). (G) Percentage of dead human neutrophils determined using Live and Dead assay after 24 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 8). (H) Neutrophils absolute number (left) and percentage of dead neutrophils (right) measured in mouse exudates of TNF-induced inflammation air pouch model at indicated time points.

The paper's framing of why apoptosis matters. Introduction, paragraph 2:

In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the development of UC and CD (22–26).

Discussion, paragraph 2:

Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation

What the paper's own numbers show

Method. The discrepancy is one of direction, not arithmetic, so the check is a census. I extracted from the archived full text every sentence that mentions neutrophil apoptosis or death. I kept the sentences that state the direction of the antibody's effect, and recorded the word that sets the direction in each. I then compared that direction with the direction of the three data panels that measure it (Fig. 3F, G and the right side of H). The paper gives no numeric values for these panels. The values below were read by eye from the published figure image, are approximate, and are used only to show direction and rough size. recompute.py.txt holds all the quotes and confirms that each one appears verbatim in the PMC text (9 of 9). It prints the tally and the approximate readings, and draws the figure.

Quantity Location Reported Recomputed / observed from the paper's own content
Direction of the mAb's effect on neutrophil apoptosis Abstract, 4th sentence "reduces neutrophil apoptosis at the site of inflammation" (decrease) Opposite to every statement and data panel below
Neutrophil apoptotic death in vitro Results, para 1 of the mAb section (Fig. 3G, fig. S10C) "accelerated neutrophil apoptotic cell death" Increase
Caspase-3/7 activation and annexin V Same sentence (Fig. 3F, fig. S10E) "an increase in Caspase-3/7 activation and annexin V staining" Increase
Mechanism Results, next sentence (fig. S10D) "this accelerated neutrophil apoptosis was associated with increased ROS production" Increase
Premise of the safety assessment Results, colitis section, para 4 "anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)" Increase
Summary of in vitro activity Discussion, para 3 "promoting … inflammatory neutrophil migration and apoptosis" Increase
Effect at the inflammatory site in vivo Discussion, para 3 "increased apoptosis of remaining neutrophils" Increase
Caspase-3/7+ human neutrophils, 10 h (n = 3) Fig. 3F Marked * (P < 0.05) About 45% with hIgG1 vs about 85% with the mAb (read by eye): higher with the mAb
Dead human neutrophils, Live/Dead, 24 h (n = 8) Fig. 3G Marked ** (P < 0.01) About 20% vs about 55% (read by eye): higher with the mAb
Dead neutrophils in air-pouch exudate (n = 4 to 5 per group) Fig. 3H, right Marked * at 24 h and ** at 48 h Higher with the mAb at both times
Tally of directional text statements Whole article — 1 decrease (abstract only) vs 5 increase (Results 3, Discussion 2); 0 statements in the body say decrease
Tally of data panels Fig. 3F, G, H right — 3 of 3 show more apoptosis or death with the mAb; 0 of 3 show less

Figure

The abstract's stated direction of the antibody's effect on neutrophil apoptosis, set against the Results, Discussion and Fig. 3

Figure 1. The left panel (orange) shows the abstract's sentence as published, which states a decrease in neutrophil apoptosis. Below it, in gray, is the abstract's own first sentence, which says proresolving GPCR agonists "induce neutrophil apoptosis". The right panel (blue) shows short verbatim excerpts from the Results and the Discussion, plus a description of Fig. 3F–H read from the published image. All of them state or show an increase. The connector marks the conflict: both panels describe the same experiments, but in opposite directions. Of the five body statements counted by recompute.py.txt, four are drawn here. The fifth, the Discussion's "promoting … inflammatory neutrophil migration and apoptosis", is listed in the table above. No values are plotted; the paper's own figure is not reproduced.

Why it matters

The abstract names two cellular mechanisms for the antibody: macrophage efferocytosis and an effect on neutrophil apoptosis. It gives the direction of the second one backwards. The paper argues that accelerated neutrophil apoptosis followed by efferocytosis is how resolution happens ("Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation"), and that delayed apoptosis is part of IBD pathology. Read on its own, the abstract therefore describes a mechanism that runs against the paper's own model.

The direction also matters for safety. The Results say the antibody was screened for immune-cell depletion and neutropenia because it "accelerates neutrophil apoptosis" and is a cytotoxicity-capable hIgG1. An abstract that says it reduces apoptosis hides the reason for that check. Readers who see only the abstract (PubMed, indexing services, literature-mining tools that extract direction-of-effect relations) could record ChemR23 agonism as anti-apoptotic for neutrophils. That contradicts both this paper's data and the resolvin E1 literature it cites.

The data, the statistics and the headline conclusions are not affected. These are the title's "reduces tissue neutrophil accumulation", the faster resolution in the air-pouch model, and the reduced lesions, fibrosis and tumors in the colitis models. Only the wording of one abstract sentence is wrong.

Explanations considered

A smaller wording problem sits in the same part of the paper. The Discussion's "promoting … inflammatory neutrophil migration and apoptosis" also reads as promoting migration, whereas the Results report that the antibody "significantly reduced human neutrophil transendothelial migration". This is noted for completeness and is not part of the main finding.

Status

Suggested correction

In the abstract, replace

…promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.

with wording that matches the Results and Discussion, for example

…promotes macrophage efferocytosis, and induces neutrophil apoptosis and reduces neutrophil accumulation at the site of inflammation.

The data, figures and statistics need no change. Optionally, the Discussion sentence "promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis" could be reworded to "promoting macrophage resolutive polarization and neutrophil apoptosis while reducing inflammatory neutrophil migration", which matches the transmigration result.

Recomputation script (python recompute.py.txt)
"""Direction-of-effect check for PMC11057782
(Sci Adv 2021;7(14):eabd1453, doi:10.1126/sciadv.abd1453).

Run:  python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.

The error is in the text, not the arithmetic. The abstract says the agonist
anti-ChemR23 mAb "reduces neutrophil apoptosis at the site of inflammation"; every
statement in the body, and the direction of the Fig. 3 F-H data, says it increases
(accelerates) it. This script lists every sentence in the article that states the
direction of the mAb's effect on neutrophil apoptosis or death, records the word
that sets the direction (classified by hand, and checked to occur in the quote),
and tallies them.

Quotes are copied verbatim from the article as archived in PubMed Central. If this
file sits in the papers repository, the script also re-checks each quote against
the archived text with src/show.py (skipped silently elsewhere).

The Fig. 3 values below were read by eye from the published figure image
(abd1453-f3.jpg); the paper prints no numeric values for these panels. They are
approximate and are used only for direction and rough size, never for a test.
"""
import os
import subprocess
import textwrap

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC11057782"

# ------------------------------------------------------------------ statements
# (location, verbatim quote, word(s) that set the direction, direction)
# direction: +1 = increase / accelerate, -1 = decrease, 0 = context (not about the mAb)
STATEMENTS = [
    ("Abstract, 4th sentence",
     "We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, "
     "promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.",
     "reduces neutrophil apoptosis", -1),
    ("Results, 'Agonist anti-ChemR23 mAb as new proresolving therapeutic modality', para 1",
     "In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death "
     "(Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V "
     "staining (Fig. 3F and fig. S10E).",
     "accelerated neutrophil apoptotic cell death", +1),
    ("Results, same paragraph, next sentence",
     "Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS "
     "production (fig. S10D).",
     "accelerated neutrophil apoptosis", +1),
    ("Results, 'ChemR23 activation with agonist mAb triggers resolution...', para 4",
     "Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human "
     "immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo "
     "effects on the peripheral immune cell population.",
     "accelerates neutrophil apoptosis", +1),
    ("Discussion, para 3 (in vitro summary)",
     "We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, "
     "including in promoting macrophage resolutive polarization as well as inflammatory neutrophil "
     "migration and apoptosis.",
     "promoting", +1),
    ("Discussion, para 3 (in vivo, inflammatory site)",
     "However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly "
     "shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory "
     "site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.",
     "increased apoptosis of remaining neutrophils", +1),
]

# Context sentences: what the paper says proresolving agonists are expected to do.
CONTEXT = [
    ("Abstract, 1st sentence",
     "Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce "
     "neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.",
     "induce neutrophil apoptosis"),
    ("Introduction, para 2",
     "In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the "
     "development of UC and CD (22–26).",
     "delayed neutrophil apoptosis"),
    ("Discussion, para 2",
     "Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation",
     "Appropriate neutrophil apoptosis is fundamental"),
]

# Fig. 3 data panels (legend text is in the paper; values read by eye from the image).
# (panel, what is measured, n, control approx %, mAb approx %, significance marker)
FIG3 = [
    ("F", "Caspase-3/7+ human neutrophils, 10 h in vitro", "n = 3", 45, 85, "*"),
    ("G", "Dead human neutrophils (Live/Dead), 24 h in vitro", "n = 8", 20, 55, "**"),
    ("H right", "Dead neutrophils in mouse air-pouch exudate, 24 h and 48 h", "n = 4 to 5 per group",
     None, None, "* at 24 h, ** at 48 h"),
]


def check_against_archive():
    """Re-check every quote against the PMC text when the repository tool is present."""
    repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
    show = os.path.join(repo, "src", "show.py")
    py = "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(show) and os.path.exists(py)):
        return None
    try:
        full = subprocess.run([py, show, PMCID, "--full"], cwd=repo, capture_output=True,
                              text=True, timeout=120).stdout
    except Exception:
        return None
    norm = " ".join(full.split())
    out = []
    for loc, q, *_ in STATEMENTS + CONTEXT:
        out.append((loc, " ".join(q.split()) in norm))
    return out


def main():
    print(f"{PMCID}: statements on the direction of the mAb's effect on neutrophil apoptosis\n")
    for loc, q, key, d in STATEMENTS:
        assert key in q, (loc, key)
        lab = {+1: "INCREASE", -1: "DECREASE"}[d]
        print(f"  {lab:8s} | {loc}\n           governing words: '{key}'")
    n_dec = sum(1 for s in STATEMENTS if s[3] == -1)
    n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
    dec_locs = [s[0] for s in STATEMENTS if s[3] == -1]
    print(f"\nText statements saying DECREASE: {n_dec} ({'; '.join(dec_locs)})")
    print(f"Text statements saying INCREASE: {n_inc} (all in Results or Discussion)")
    print(f"Body statements saying DECREASE: {sum(1 for s in STATEMENTS[1:] if s[3] == -1)}")

    print("\nContext statements (direction proresolving agonists are expected to act):")
    for loc, q, key in CONTEXT:
        assert key in q
        print(f"  {loc}: '{key}'")

    print("\nFig. 3 data panels (values read by eye from the image; approximate):")
    n_up = 0
    for p, what, n, c, m, sig in FIG3:
        if c is not None:
            print(f"  Fig. 3{p}: {what} ({n}): hIgG1 ~{c}% vs anti-ChemR23 ~{m}% "
                  f"(difference ~{m - c} points, {m / c:.1f}x), marked {sig}")
        else:
            print(f"  Fig. 3{p}: {what} ({n}): higher with anti-ChemR23 than hIgG1, marked {sig}")
        n_up += 1
    print(f"Fig. 3 panels in which apoptosis/death is higher with the mAb: {n_up} of {len(FIG3)}")
    print(f"Fig. 3 panels in which it is lower: 0 of {len(FIG3)}")

    chk = check_against_archive()
    if chk is not None:
        ok = sum(1 for _, v in chk if v)
        print(f"\nQuotes found verbatim in the archived PMC text: {ok} of {len(chk)}")
        for loc, v in chk:
            if not v:
                print(f"  NOT FOUND: {loc}")

    make_fig1()


# ------------------------------------------------------------------ figure
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"


def wrap(s, width):
    return "\n".join(textwrap.wrap(s, width))


def make_fig1():
    plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
                         "svg.fonttype": "none", "figure.facecolor": BG, "savefig.facecolor": BG})
    W, H = 7.2, 6.0
    fig = plt.figure(figsize=(W, H))
    ax = fig.add_axes([0, 0, 1, 1])
    ax.set_xlim(0, W)
    ax.set_ylim(0, H)
    ax.axis("off")
    LH = 0.158  # line height in inches at 10 pt

    def box(x0, y0, x1, y1, col):
        ax.add_patch(FancyBboxPatch((x0, y0), x1 - x0, y1 - y0, boxstyle="round,pad=0,rounding_size=0.06",
                                    fc="white", ec=col, lw=1.5))

    def put(x, y, s, n_lines=None, **kw):
        ax.text(x, y, s, va="top", ha="left", linespacing=1.25, **kw)
        k = s.count("\n") + 1 if n_lines is None else n_lines
        return y - k * LH * 1.0

    ax.text(0.12, H - 0.14, "Direction of the mAb's effect on neutrophil apoptosis, as stated in the paper",
            fontsize=11.5, fontweight="bold", va="top")

    # ---- left: abstract
    lx0, lx1, top, bot = 0.12, 2.55, H - 0.52, 0.86
    box(lx0, bot, lx1, top, ORANGE)
    y = top - 0.12
    y = put(lx0 + 0.12, y, "ABSTRACT", fontsize=10.5, fontweight="bold", color=ORANGE)
    y = put(lx0 + 0.12, y, "4th sentence (as published)", fontsize=10, color=SEC) - 0.06
    s = STATEMENTS[0][1]
    y = put(lx0 + 0.12, y, wrap("“" + s + "”", 28), fontsize=10)
    y -= 0.14
    y = put(lx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
    y = put(lx0 + 0.12, y, "↓ DECREASE", fontsize=12, fontweight="bold", color=ORANGE) - 0.30
    ax.plot([lx0 + 0.12, lx1 - 0.12], [y + 0.12, y + 0.12], color=GRID, lw=1)
    y = put(lx0 + 0.12, y, "Same abstract, 1st sentence:", fontsize=10, color=SEC)
    y = put(lx0 + 0.12, y, wrap("“…proresolving lipids, which activate GPCRs to induce "
                                 "neutrophil apoptosis…”", 28), fontsize=10, color=SEC)

    # ---- right: body
    rx0, rx1 = 3.45, 7.08
    box(rx0, bot, rx1, top, BLUE)
    y = top - 0.12
    y = put(rx0 + 0.12, y, "RESULTS, DISCUSSION, FIG. 3", fontsize=10.5, fontweight="bold", color=BLUE) - 0.06
    items = [
        ("Results (Fig. 3F, G; in vitro)",
         "“accelerated neutrophil apoptotic cell death … an increase in Caspase-3/7 activation "
         "and annexin V staining”"),
        ("Results (mechanism)",
         "“this accelerated neutrophil apoptosis was associated with increased ROS production”"),
        ("Results (safety, colitis section)",
         "“anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)”"),
        ("Discussion (in vivo, inflammatory site)",
         "“an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis "
         "of remaining neutrophils”"),
        ("Fig. 3F, G, H (data, read from the image)",
         "Caspase-3/7+ and dead neutrophils higher with the mAb than with hIgG1 in vitro, and dead "
         "neutrophils higher in air-pouch exudate at 24 h and 48 h"),
    ]
    for loc, txt in items:
        y = put(rx0 + 0.12, y, loc, fontsize=10, color=SEC)
        y = put(rx0 + 0.12, y, wrap(txt, 43), fontsize=10) - 0.08
    y -= 0.04
    y = put(rx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
    n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
    y = put(rx0 + 0.12, y, "↑ INCREASE", fontsize=12, fontweight="bold", color=BLUE)
    put(rx0 + 0.12, y - 0.04, f"all {n_inc} body statements; all 3 data panels",
        fontsize=10, fontweight="bold", color=BLUE)

    # ---- connector
    ymid = top - 1.55
    arr = FancyArrowPatch((lx1 + 0.04, ymid), (rx0 - 0.04, ymid), arrowstyle="<|-|>", mutation_scale=14,
                          lw=2, color=TXT)
    ax.add_patch(arr)
    ax.text((lx1 + rx0) / 2, ymid + 0.12, "opposite\ndirection", ha="center", va="bottom",
            fontsize=10.5, fontweight="bold")
    ax.text((lx1 + rx0) / 2, ymid - 0.12, "same\ndata\n(Fig. 3F–H)", ha="center", va="top",
            fontsize=10, color=SEC, linespacing=1.2)

    ax.text(0.12, 0.70, "Orange: the abstract's wording as published.", fontsize=10, color=SEC, va="top")
    ax.text(0.12, 0.50, "Blue: what the Results, the Discussion and the paper's own Fig. 3 data show.",
            fontsize=10, color=SEC, va="top")
    ax.text(0.12, 0.30, "Quotes verbatim from the PMC archive copy (PMC11057782); "
            "… marks omitted words.", fontsize=10, color=SEC, va="top")

    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
    plt.close(fig)
    print("\nwrote fig1.svg and fig1.png")


if __name__ == "__main__":
    main()

Abstract and Introduction state the reverse of the paper's own Schwann cell depletion result

Sci Adv (2023) · doi:10.1126/sciadv.add6995 · PMC9891701

Summary

The Abstract states that "Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors", and the Introduction repeats it. (TASc are tumor-associated nonmyelinating Schwann cells.) The paper's only pancreas-specific depletion is intrapancreatic AAV8-GFAP-DTA in KPC mice. For that experiment the Results report improved survival (Fig. 2C; log-rank; n = 8, 12 and 13 mice), fewer PanIN 2/3 lesions and inhibited tumor growth (Fig. 2D–G). The Fig. 2 title reads "Depletion of TASc attenuates pancreatic cancer tumorigenesis". The main text contains 17 statements (14 sentences, one Results subsection heading and two figure titles) that state a direction for the effect of depleting, inhibiting or targeting TASc (or TASc-expressed PVT1) on tumors. Fifteen say depletion is antitumor. Only the Abstract and Introduction sentences say it is protumor. The data, statistics and the paper's central thesis are unaffected. Still, the Abstract, the paper's own summary of its findings, reverses the direction of its headline in vivo result, and no correction has been issued.

What the paper reports

The two sentences that state the depletion result as protumor:

Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.

Abstract, sentence 4

Depletion of TASc promoted the tumorigenesis of PDAC.

Introduction, final (third) paragraph

The same experiment as described in the Results and in the Fig. 2 legend:

TASc promotes PDAC tumorigenesis

Results, subsection heading

KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group (Fig. 2C). Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).

Results, "TASc promotes PDAC tumorigenesis", paragraph 1

Depletion of TASc attenuates pancreatic cancer tumorigenesis.

Fig. 2 title

Log-rank test; n = 8, 12, and 13 mice, respectively.

Fig. 2 legend, panel C (WT, AAV8-GFAP-Blank and AAV8-GFAP-DTA KPC mice)

Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.

Fig. 2 legend, panel G (tumor volumes at 22 weeks)

The paper's own summary of the add-back experiment in the same subsection points the same way:

Together, these results suggested that the presence of TASc in PDAC facilitates tumorigenesis and resistance to immunotherapy.

Results, "TASc promotes PDAC tumorigenesis", paragraph 2, last sentence

So do the pharmacological depletion with fluorocitrate (FC), the Discussion, and a later sentence of the same Abstract:

Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the infiltration and cytotoxicity of CD8+ T cells.

Results, "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy", last sentence

Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic microenvironment and improved the antitumor effects of immune checkpoint inhibitors.

Discussion, paragraph 2

Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.

Abstract, last sentence

Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC9891701 --grep), and the key sentences also against the publisher's typeset PDF (see Status).

What the paper's own numbers show

Method. No quantity can be recomputed, because the error is in the stated direction of an effect. The tumor volumes and survival curves appear only as plotted data in Fig. 2, and the text and figure title state their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every statement (sentence, Results subsection heading or figure title) outside Materials and Methods that states a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth, tumorigenesis, survival or response to immunotherapy. To build it, every sentence of the Abstract, Introduction, Results and Discussion, every Results subsection heading and every figure title was read. A statement was included when its subject is the removal or inhibition of TASc (by AAV8-GFAP-DTA, by fluorocitrate, by "a small-molecule" agent, or by targeting in general) or of TASc-expressed PVT1, and it states a direction for one of those outcomes. Statements left out were: those about the presence, rather than the removal, of TASc or PVT1 (such as the heading "TASc promotes PDAC tumorigenesis"); deletions of PVT1 nucleotides, RAF1 or TDO2; immune-cell or TASc-abundance readouts only; the cytotoxicity, safety or specificity of FC; and sentences calling TASc a therapeutic target without stating a tumor outcome. The script checks each listed statement verbatim against the archived text and confirms its section, labels each with the direction it states, and tallies them. Run with the publisher's PDF and supplement as text (instructions in the script header), it also checks those. It prints every number below.

Quantity Location Reported Recomputed from the paper's own text
Effect of depleting pancreatic TASc on tumorigenesis Abstract, sentence 4 "promoted the tumorigenesis" Opposite: depletion attenuates tumorigenesis (Results; Fig. 2 title; fig. S4 title)
Same Introduction, final paragraph "promoted the tumorigenesis" Opposite, as above
Survival after AAV8-GFAP-DTA depletion Results; Fig. 2C (log-rank; n = 8 WT, 12 Blank, 13 DTA) "improved survival time compared to control group" Depletion is antitumor
PanIN grade and tumor growth after depletion Results; Fig. 2D–G (Fig. 2F n = 8, 12, 13; Fig. 2G n = 12 vs 13) "reduced PanIN 2/3, and inhibited tumor growth" Depletion is antitumor
Fig. 2 title Fig. 2 legend "Depletion of TASc attenuates pancreatic cancer tumorigenesis." Depletion is antitumor
Directional statements on depleting, inhibiting or targeting TASc or TASc-expressed PVT1 Main text (Abstract, Introduction, Results incl. subsection headings, Discussion, figure titles) — 17 statements (14 sentences, 1 heading, 2 figure titles): 2 protumor (Abstract, Introduction), 15 antitumor
Statements about the AAV8-GFAP-DTA experiment only Main text — 5 statements: 2 protumor (Abstract, Introduction), 3 antitumor (Results ×2, Fig. 2 title)
Same, including the supplementary figure titles Main text + figs. S4, S13, S16 — 20 statements, 2 protumor, 18 antitumor (19 and 17 if the fig. S16 schematic is left out); DTA experiment 6 statements, 2 protumor, 4 antitumor
Fluorocitrate or small-molecule statements Main text — 7, all antitumor
PVT1-loss statements Main text — 3, all antitumor
Statements naming both PVT1 loss and FC Main text (Discussion) — 2, both antitumor
Sections that contain a protumor statement Main text — Abstract and Introduction only
Supplement sentences of the form "deplet… promot… tumorigen…" Supplementary Materials PDF — 0

The Abstract itself contains three depletion statements in opposite directions. Sentence 4 says depletion is protumor. The PVT1 sentence and the last sentence say depletion is antitumor.

A minor side observation that does not affect the finding: the Fig. 2G legend names three groups (WT, AAV8-GFAP-Blank, AAV8-GFAP-DTA) but gives n for only two ("n = 12 and 13 animals").

Figure

The Abstract and Introduction sentences beside the Results and Fig. 2 statements for the same depletion experiment

Figure 1. The left panel (orange) shows the two sentences, in the Abstract and the Introduction, that state the depletion result as protumor. The right panel (blue) shows what the paper reports for the same experiment, intrapancreatic AAV8-GFAP-DTA in KPC mice: the Results phrases for Fig. 2C and Fig. 2D–G, with sample sizes from the Fig. 2 legend, and the Fig. 2 title. The connector marks the conflict: one experiment, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.

Every directional statement on depleting or inhibiting TASc, by section

Figure 2. Each marker is one of the 17 main-text statements that state a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumors (inclusion rule under Method). Columns run in reading order from the Abstract to the Discussion, followed by the two figure titles. The Results FC column includes the subsection heading "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy". The upper row holds statements that depletion promotes tumorigenesis. The lower row holds statements that it inhibits tumor growth or improves outcome. Marker shape gives the intervention: circles for AAV8-GFAP-DTA, squares for PVT1 loss, diamonds for fluorocitrate, triangles for sentences that name both PVT1 and FC. Markers in the same cell are stacked. Only the two orange markers, in the Abstract and the Introduction, sit in the upper row. Both concern the DTA experiment, whose three other statements all sit in the lower row. The supplementary titles of figs. S4, S13 and S16 (not drawn) would all sit in the lower row. Directions and locations come from recompute.py.txt.

Why it matters

Explanations considered

  1. The sentences refer to a different experiment or setting. This does not hold. "Pancreatic-specific" depletion matches only the intrapancreatic AAV8-GFAP-DTA delivery in KPC mice (Fig. 2, fig. S4). Every other statement about that experiment, in the Results, the Fig. 2 title and the fig. S4 title, reports an antitumor effect. The paper reports no subgroup, time point or model in which depletion accelerated tumors.
  2. "Tumorigenesis" is used in an unusual sense. This does not hold. The paper uses the word in its ordinary sense throughout, for example "TASc promotes PDAC tumorigenesis" and "Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis".
  3. The increase in PanIN 1 lesions counts as "promoted". This does not hold. After depletion, the percentage of low-grade PanIN 1 rose while the share of high-grade PanIN 2/3 fell. The paper reports this shift in the same sentence as "inhibited tumor growth" and summarizes the panel under the title "Depletion of TASc attenuates pancreatic cancer tumorigenesis".
  4. The intended subject was TASc, not their depletion. "TASc promoted the tumorigenesis of PDAC", as in the Results heading, would be consistent with the data. So would "Depletion of TASc attenuated tumorigenesis", as in the Fig. 2 title. This explains how the wording could have arisen. It does not change the fact that the printed sentences state the opposite of the data.
  5. Conversion or archiving artifact. Ruled out. The publisher's typeset PDF in the PMC Open Access dataset (version 1, the only version; marked as the publisher's version, not an author manuscript) contains both sentences word for word. So does the Europe PMC full text. The conflict is in words ("promoted" against "attenuates"), not in symbols, signs or table cells that a conversion could alter.
  6. The supplement resolves it. It does not. The fig. S4 title ("Depletion of TASc inhibits PDAC tumorigenesis."), the fig. S13 title ("FC treatment sensitizes PDAC to immunotherapy.") and the title of the fig. S16 schematic ("Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of immunotherapy-refractory pancreatic cancer.") all support the Results. The supplement contains no sentence saying depletion promotes tumorigenesis.

None of these explanations reconciles the Abstract and Introduction sentences with the paper's results.

Status

As of 2026-10-23:

Suggested correction

Reverse the direction in the two sentences so that they match the Results, the Fig. 2 title and the fig. S4 title:

No change to the data, figures, statistics or conclusions is needed. The Fig. 2G legend could also give the WT group size, since it currently gives n for two of its three groups.

Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9891701 (Sci Adv 2023, doi:10.1126/sciadv.add6995)
# Abstract and Introduction vs Results / Fig. 2: what does depleting tumor-associated
# nonmyelinating Schwann cells (TASc) do to PDAC tumorigenesis?
#
# Run:  python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of an effect stated in words; there is no numeric
# value to recompute. The script therefore does three things:
#   1. holds a hand-curated list of every statement (sentence, Results subsection heading or
#      figure title; Materials and Methods excluded) that states a direction for the effect of
#      depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth,
#      tumorigenesis, survival or response to immunotherapy, each copied verbatim from the
#      article as archived in PubMed Central and labelled with the direction it states.
#      The list was built by reading every sentence of the Abstract, Introduction, Results and
#      Discussion, every Results subsection heading and every figure title (see INCLUSION below);
#   2. if run inside the /home/ubuntu/papers repo, re-checks each sentence against the
#      archived full text (src/show.py) and confirms it sits in the stated section;
#      skipped silently elsewhere;
#   3. tallies the directions and draws the two figures.
# Group sizes (n) are transcribed from the Fig. 2 legend.
#
# Optional checks of the publisher's typeset PDF and the Supplementary Materials PDF, both
# held in the PMC Open Access dataset (version 1 is the only version):
#   aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/PMC9891701.1.pdf ./
#   aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/sciadv.add6995_sm.pdf ./
#   pdftotext PMC9891701.1.pdf vor.txt ; pdftotext sciadv.add6995_sm.pdf sm.txt
#   VOR_TXT=./vor.txt SUPP_TXT=./sm.txt python recompute.py.txt
# Without those variables the two checks are reported as "not run", not as passed.

import os, re, subprocess
from collections import Counter, OrderedDict

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch

HERE = os.path.dirname(os.path.abspath(__file__))

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"

# ------------------------------------------------------------------ statements
# INCLUSION: subject = depleting / inhibiting / targeting TASc (by AAV8-GFAP-DTA, fluorocitrate
# or "a small-molecule" agent) or TASc-expressed PVT1; predicate = a direction for tumor growth,
# tumorigenesis, survival or response to immunotherapy. Left out: statements about the
# presence (not removal) of TASc or PVT1 (see CONVERSE), deletion of PVT1 nucleotides, RAF1 or
# TDO2, immune-cell or TASc-abundance readouts only, cytotoxicity, safety or specificity of FC,
# and statements that TASc are a therapeutic target without a stated tumor outcome.
# key: (section as archived, direction, experiment, verbatim statement)
#   direction "PRO"  = depletion promotes tumorigenesis (protumor)
#   direction "ANTI" = depletion inhibits tumor growth / improves outcome (antitumor)
#   experiment: "DTA" = AAV8-GFAP-DTA depletion of Schwann cells in KPC mice (Fig. 2)
#               "PVT1" = loss of PVT1 in TASc (Fig. 6), "FC" = fluorocitrate (Fig. 7)
STATEMENTS = OrderedDict([
    ("abs_pro", ("ABSTRACT", "PRO", "DTA",
        "Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.")),
    ("abs_pvt1", ("ABSTRACT", "ANTI", "PVT1",
        "Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
    ("abs_fc", ("ABSTRACT", "ANTI", "FC",
        "Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized "
        "PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.")),
    ("intro_pro", ("INTRODUCTION", "PRO", "DTA",
        "Depletion of TASc promoted the tumorigenesis of PDAC.")),
    ("intro_pvt1", ("INTRODUCTION", "ANTI", "PVT1",
        "Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
    ("intro_fc", ("INTRODUCTION", "ANTI", "FC",
        "Furthermore, using a small-molecule compound to inhibit TASc in vivo notably improved "
        "the efficacy of immune checkpoint inhibitors.")),
    ("res_dta_surv", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
        "KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group "
        "(Fig. 2C).")),
    ("res_dta_growth", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
        "Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased "
        "percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).")),
    ("res_pvt1", ("RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis", "ANTI", "PVT1",
        "on the contrary, depletion of Pvt1 in Schwann cells inhibited tumor growth (Fig. 6, A and B).")),
    ("res_fc_heading", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
        "ANTI", "FC",
        "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy")),   # subsection heading
    ("res_fc_retraction", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
        "ANTI", "FC",
        "Treatment with FC and control antibodies led to retraction of PDAC (Fig. 7, A to C).")),
    ("res_fc_combo", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
        "ANTI", "FC",
        "Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis in "
        "comparison with anti–PD-1 treatment alone (Fig. 7, A to C, and fig. S13A).")),
    ("res_fc_summary", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
        "ANTI", "FC",
        "Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the "
        "infiltration and cytotoxicity of CD8+ T cells.")),
    ("disc_restored", ("DISCUSSION", "ANTI", "PVT1+FC",
        "Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic "
        "microenvironment and improved the antitumor effects of immune checkpoint inhibitors.")),
    ("disc_therapy", ("DISCUSSION", "ANTI", "PVT1+FC",
        "Considering the critical role of TASc in immunosuppressive microenvironments, depleting TASc "
        "or TASc-expressed PVT1 in PDAC may be a promising therapeutic modality.")),
    ("fig2_title", ("Fig. 2 legend", "ANTI", "DTA",
        "Depletion of TASc attenuates pancreatic cancer tumorigenesis.")),
    ("fig7_title", ("Fig. 7 legend", "ANTI", "FC",
        "TASc inhibition sensitizes PDAC to immunotherapy.")),
])

KIND = {"res_fc_heading": "heading", "fig2_title": "figure title", "fig7_title": "figure title"}

# Statements about the presence (not depletion) of TASc; these imply that depletion should
# reduce tumorigenesis. Shown for context only, not part of the tally.
CONVERSE = OrderedDict([
    ("res_heading", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results subsection heading",
        "TASc promotes PDAC tumorigenesis")),
    ("res_s5", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results, same subsection, paragraph 2",
        "Together, these results suggested that the presence of TASc in PDAC facilitates "
        "tumorigenesis and resistance to immunotherapy.")),
])

# Supplementary Materials (sciadv.add6995_sm.pdf), figure titles that state a direction for
# depletion/inhibition/targeting of TASc. Counted separately from the main-text tally.
# fig. S16 is a schematic (graphic illustration) of the whole model; its title is counted too.
SUPP_TITLES = OrderedDict([
    ("figS4_title", ("fig. S4 title", "ANTI", "DTA", "Depletion of TASc inhibits PDAC tumorigenesis.")),
    ("figS13_title", ("fig. S13 title", "ANTI", "FC", "FC treatment sensitizes PDAC to immunotherapy.")),
    ("figS16_title", ("fig. S16 title (schematic)", "ANTI", "TASc targeting",
        "Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of "
        "immunotherapy-refractory pancreatic cancer.")),
])

# Fig. 2 legend, panels C, F and G (transcribed)
N_FIG2C = {"WT": 8, "AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13}   # "n = 8, 12, and 13 mice"
N_FIG2G = {"AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13}            # "n = 12 and 13 animals"
FIG2_LEGEND_SNIPPETS = [
    "Log-rank test; n = 8, 12, and 13 mice, respectively.",
    "Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.",
]


# ------------------------------------------------------------------ verification
def load_archived_text():
    repo = "/home/ubuntu/papers"
    py = "/home/ubuntu/papers-data/venv/bin/python"
    if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
        return None
    try:
        out = subprocess.run([py, "src/show.py", "PMC9891701", "--full"], cwd=repo,
                             capture_output=True, text=True, timeout=120).stdout
    except Exception:
        return None
    return out or None


def sections(txt):
    """Map section label -> text, using the --full layout of src/show.py."""
    secs, cur = {}, None
    for line in txt.splitlines():
        if line.startswith("ABSTRACT:"):
            cur = "ABSTRACT"; line = line[len("ABSTRACT:"):]
        elif line.startswith("## "):
            cur = line[3:].strip(); secs[cur] = secs.get(cur, "") + cur + "\n"; continue  # heading kept
        else:
            m = re.match(r"\[Fig\. (\d+)\.\]", line)
            if m:
                cur = f"Fig. {m.group(1)} legend"
        if cur:
            secs[cur] = secs.get(cur, "") + line + "\n"
    return secs


def verify(txt):
    secs = sections(txt)
    ok = True
    print("Verbatim check against the archived full text (src/show.py PMC9891701 --full):")
    for key, (sec, *_rest, s) in list(STATEMENTS.items()) + [
            (k, (v[0], v[2])) for k, v in CONVERSE.items()]:
        total = txt.count(s)
        in_sec = secs.get(sec, "").count(s)
        flag = "ok" if in_sec >= 1 else "MISSING"
        ok &= in_sec >= 1
        print(f"  [{flag}] {key:18s} found {total}x in full text, {in_sec}x in '{sec[:45]}'")
    leg2 = secs.get("Fig. 2 legend", "")
    for snip in FIG2_LEGEND_SNIPPETS:
        c = leg2.count(snip)
        ok &= c == 1
        print(f"  [{'ok' if c == 1 else 'MISSING'}] Fig. 2 legend: {snip}")
    print("  all checks passed" if ok else "  SOME CHECKS FAILED")
    return ok


def norm_pdf_text(t):
    t = re.sub(r"-\n", "-", t)           # keep hyphens split across lines
    return re.sub(r"\s+", " ", t)


def verify_pdfs():
    vor, supp = os.environ.get("VOR_TXT"), os.environ.get("SUPP_TXT")
    print("\nPublisher PDF (version of record) check:", end=" ")
    if vor and os.path.exists(vor):
        t = norm_pdf_text(open(vor, encoding="utf-8", errors="replace").read())
        print()
        for key in ("abs_pro", "intro_pro", "res_dta_surv", "res_dta_growth", "fig2_title"):
            q = STATEMENTS[key][3]
            n_full = t.count(q)
            if n_full:
                print(f"  {key:18s} found {n_full}x")
            else:  # the PDF text layer can interleave a figure caption inside a sentence
                tail = q[q.index(" ", 12) + 1:]
                print(f"  {key:18s} found 0x as one string; tail '{tail[:40]}...' found "
                      f"{t.count(tail)}x (sentence interrupted by a figure caption in the PDF text layer)")
    else:
        print("not run (set VOR_TXT)")
    print("Supplementary Materials check:", end=" ")
    if supp and os.path.exists(supp):
        t = norm_pdf_text(open(supp, encoding="utf-8", errors="replace").read())
        print()
        for key, (loc, d, exp, q) in SUPP_TITLES.items():
            print(f"  {key:18s} found {t.count(q)}x  ({loc}: {q})")
        n_pro = len(re.findall(r"(?i)deplet[^.]{0,80}promot[^.]{0,40}tumorigen", t))
        print(f"  sentences of the form 'deplet... promot... tumorigen...' in supplement: {n_pro}")
    else:
        print("not run (set SUPP_TXT)")


# ------------------------------------------------------------------ tally
def tally():
    print("\nDirection stated for the effect of depleting/inhibiting/targeting TASc or TASc-expressed PVT1:")
    for key, (sec, d, exp, s) in STATEMENTS.items():
        lab = "depletion PROMOTES tumor" if d == "PRO" else "depletion INHIBITS tumor"
        print(f"  {key:18s} {sec[:40]:40s} {KIND.get(key, 'sentence'):12s} {exp:8s} {lab}")
    c = Counter(d for _, d, _, _ in STATEMENTS.values())
    n = len(STATEMENTS)
    kinds = Counter(KIND.get(k, "sentence") for k in STATEMENTS)
    print(f"\nTotal directional statements (main text): {n}  ({dict(kinds)})")
    print(f"  depletion promotes tumorigenesis: {c['PRO']}  "
          f"({', '.join(k for k, v in STATEMENTS.items() if v[1] == 'PRO')})")
    print(f"  depletion inhibits tumor / improves outcome: {c['ANTI']}")
    dta = [(k, v) for k, v in STATEMENTS.items() if v[2] == "DTA"]
    cd = Counter(v[1] for _, v in dta)
    print(f"Statements about the AAV8-GFAP-DTA depletion experiment itself: {len(dta)} "
          f"({cd['PRO']} promotes [{', '.join(k for k, v in dta if v[1] == 'PRO')}], "
          f"{cd['ANTI']} inhibits [{', '.join(k for k, v in dta if v[1] == 'ANTI')}])")
    allst = list(STATEMENTS.values()) + list(SUPP_TITLES.values())
    ca = Counter(v[1] for v in allst)
    dta_all = [v for v in allst if v[2] == "DTA"]
    cda = Counter(v[1] for v in dta_all)
    print(f"Adding the supplementary figure titles ({', '.join(v[0] for v in SUPP_TITLES.values())}): "
          f"{len(allst)} statements, {ca['PRO']} promote, {ca['ANTI']} inhibit; "
          f"DTA experiment {len(dta_all)} statements, {cda['PRO']} promote, {cda['ANTI']} inhibit")
    nos16 = [v for k, v in SUPP_TITLES.items() if k != "figS16_title"]
    cn = Counter(v[1] for v in list(STATEMENTS.values()) + nos16)
    print(f"  (without the fig. S16 schematic title: {n + len(nos16)} statements, "
          f"{cn['PRO']} promote, {cn['ANTI']} inhibit)")
    fc = [v for v in STATEMENTS.values() if v[2] == "FC"]
    print(f"Fluorocitrate/small-molecule statements (main text): {len(fc)}, all "
          f"{'antitumor' if all(v[1] == 'ANTI' for v in fc) else 'MIXED'}")
    pvt = [v for v in STATEMENTS.values() if v[2] == "PVT1"]
    print(f"PVT1-loss statements (main text): {len(pvt)}, all "
          f"{'antitumor' if all(v[1] == 'ANTI' for v in pvt) else 'MIXED'}")
    both = [v for v in STATEMENTS.values() if v[2] == "PVT1+FC"]
    print(f"Statements naming both PVT1 loss and FC (main text): {len(both)}, all "
          f"{'antitumor' if all(v[1] == 'ANTI' for v in both) else 'MIXED'}")
    pro_secs = sorted({v[0] for v in STATEMENTS.values() if v[1] == "PRO"})
    print(f"Sections containing a 'promotes' statement: {pro_secs}")
    print(f"\nFig. 2C group sizes (log-rank): {N_FIG2C}; total mice = {sum(N_FIG2C.values())}")
    print(f"Fig. 2G group sizes (t test, tumor volume at 22 weeks): {N_FIG2G}; "
          f"total = {sum(N_FIG2G.values())}")
    return c, dta


# ------------------------------------------------------------------ figures
def style():
    plt.rcParams.update({
        "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
        "axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
        "svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
        "savefig.facecolor": BG,
    })


def panel(ax, x, y, w, h, color, title, sub, body):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
                                lw=2, ec=color, fc="white"))
    ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
    ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
    yy = y + h - 0.165
    for item in body:
        txt, kw = item if isinstance(item, tuple) else (item, {})
        ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
                fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
        yy -= kw.get("step", 0.12)


def fig1(n2c, n2g):
    fig = plt.figure(figsize=(7.2, 5.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    ax.text(0.03, 0.98, "Effect of TASc depletion on PDAC: two directions for one experiment",
            fontsize=11.5, fontweight="bold", va="top")

    panel(ax, 0.03, 0.255, 0.41, 0.66, ORANGE,
          "Abstract and Introduction",
          "summary of the depletion result",
          [("Abstract, sentence 4:", {"color": SEC, "size": 10, "step": 0.045}),
           ("“Depletion of pancreatic-specific\nTASc promoted the tumorigenesis\nof PDAC tumors.”",
            {"step": 0.17}),
           ("Introduction, final paragraph:", {"color": SEC, "size": 10, "step": 0.045}),
           ("“Depletion of TASc promoted the\ntumorigenesis of PDAC.”", {"step": 0.12}),
           ("Stated: depletion is protumor", {"color": ORANGE, "weight": "bold",
                                                         "size": 10.5})])

    panel(ax, 0.56, 0.255, 0.41, 0.66, BLUE,
          "Results and Fig. 2",
          "AAV8-GFAP-DTA depletion in KPC mice",
          [("Fig. 2C (log-rank; n = %d, %d, %d):" % tuple(n2c.values()),
            {"color": SEC, "size": 10, "step": 0.045}),
           ("“improved survival time compared\nto control group”", {"step": 0.095}),
           ("Fig. 2D–G (volumes: n = %d vs %d):" % tuple(n2g.values()),
            {"color": SEC, "size": 10, "step": 0.045}),
           ("“reduced PanIN 2/3, and inhibited\ntumor growth”", {"step": 0.095}),
           ("Fig. 2 title:", {"color": SEC, "size": 10, "step": 0.045}),
           ("“Depletion of TASc attenuates\npancreatic cancer tumorigenesis.”", {"step": 0.11}),
           ("Shown: depletion is antitumor", {"color": BLUE, "weight": "bold",
                                                         "size": 10.5})])

    arr = FancyArrowPatch((0.448, 0.585), (0.552, 0.585), arrowstyle="<|-|>", mutation_scale=16,
                          lw=2, color=TXT)
    ax.add_patch(arr)
    ax.text(0.5, 0.615, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
            fontweight="bold", linespacing=1.15)

    ax.text(0.03, 0.205,
            "The paper's only pancreas-specific depletion of TASc is this experiment: AAV8-GFAP-DTA\n"
            "delivered into the pancreas of KPC mice, compared with AAV8-GFAP-Blank controls.",
            fontsize=10, color=TXT, va="top", linespacing=1.35)
    ax.text(0.03, 0.10,
            "Elsewhere the paper agrees with the right panel: Results heading “TASc promotes PDAC\n"
            "tumorigenesis”; PVT1 loss in TASc and fluorocitrate depletion both reduce tumor growth.",
            fontsize=10, color=SEC, va="top", linespacing=1.35)
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
    plt.close(fig)


def fig2():
    order = ["ABSTRACT", "INTRODUCTION",
             "RESULTS > TASc promotes PDAC tumorigenesis",
             "RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis",
             "RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
             "DISCUSSION", "Fig. 2 legend", "Fig. 7 legend"]
    labels = ["Abstract", "Intro-\nduction", "Results\nDTA\n(Fig. 2)", "Results\nPVT1\n(Fig. 6)",
              "Results\nFC\n(Fig. 7)", "Discus-\nsion", "Fig. 2\ntitle", "Fig. 7\ntitle"]
    expmark = {"DTA": "o", "PVT1": "s", "FC": "D", "PVT1+FC": "^"}
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    fig.subplots_adjust(left=0.17, right=0.98, top=0.83, bottom=0.27)
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    counts = Counter()
    for key, (sec, d, exp, _) in STATEMENTS.items():
        xi = order.index(sec)
        yi = 1 if d == "PRO" else 0
        off = counts[(xi, yi)]; counts[(xi, yi)] += 1
        col = ORANGE if d == "PRO" else BLUE
        ax.plot(xi, yi + 0.13 * off - 0.0, marker=expmark[exp], ms=10, color=col,
                mec="white", mew=1, ls="none", zorder=3)
    ax.set_xticks(range(len(order))); ax.set_xticklabels(labels, fontsize=10)
    ax.set_yticks([0, 1])
    ax.set_yticklabels(["depletion\ninhibits\ntumor", "depletion\npromotes\ntumor"], fontsize=10)
    ax.tick_params(axis="y", length=0)
    ax.set_ylim(-0.35, 1.45); ax.set_xlim(-0.6, len(order) - 0.4)
    c = Counter(v[1] for v in STATEMENTS.values())
    ax.text(-0.5, 1.3, f"{c['PRO']} statements (orange): the Abstract and Introduction sentences",
            color=ORANGE, fontsize=10, fontweight="bold", va="center")
    ax.text(1.6, 0.62, f"{c['ANTI']} statements (blue), in every section",
            color=BLUE, fontsize=10, fontweight="bold", va="center")
    # marker legend for experiment
    from matplotlib.lines import Line2D
    h = [Line2D([], [], marker=m, color=SEC, ls="none", ms=8, label=l) for l, m in
         [("DTA depletion", "o"), ("PVT1 loss", "s"), ("fluorocitrate", "D"), ("PVT1 or FC", "^")]]
    ax.legend(handles=h, ncol=4, loc="lower left", bbox_to_anchor=(-0.2, 1.02), frameon=False,
              fontsize=10, handletextpad=0.3, columnspacing=1.0)
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
    plt.close(fig)


if __name__ == "__main__":
    txt = load_archived_text()
    if txt:
        verify(txt)
    else:
        print("(archived text not available here; verbatim check skipped)")
    verify_pdfs()
    tally()
    style()
    fig1(N_FIG2C, N_FIG2G)
    fig2()
    print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)

Binomial p for 12 of 17 complexes is 0.072, not the reported 0.0245

eLife (2024) · doi:10.7554/eLife.93333 · PMC10942632

Summary

The paper reports that 12 of 17 protein complexes had an ROC-AUC above 0.5 and calls this "a significant excess (binomial test, p=0.0245)" against a null expectation of one half. The exact one-sided binomial p-value for 12 or more successes out of 17 at p = 0.5 is 9402/131072 = 0.0717, and the two-sided value is 0.143, so neither is significant at 0.05. The reported 0.0245 equals P(X ≥ 13), the upper tail starting one success above the observed count. The second binomial test in the same paragraph shows the same pattern: 0.0012 is reported, while P(X ≥ 4 | n = 17, p = 0.05) is 0.0088. That result remains significant. The correction removes one of the paper's three complex-level arguments for a detectable physical-interaction signal in ERC, and the corrected value is now consistent with the paper's own non-significant global permutation test (p = 0.08). The headline conclusion in the title and abstract does not depend on this test.

What the paper reports

The first binomial test, its stated null, and the second binomial test:

"Of the 17 complexes, 12 had an ROC-AUC > 0.5. Since the random expectation would be that half would exceed 0.5, this is a significant excess (binomial test, p=0.0245). Moreover, five complexes had AUCs > 0.7 (Figure 4). These results indicate that physically interacting domains tend to have a higher ERC than nonphysically interacting domains in those complexes. Likewise, four complexes had their physically interacting domains ranked significantly higher than noninteracting domains at an alpha of 5% (one-tailed Mann–Whitney U test), which is also a significant excess of complexes overall (binomial test, p=0.0012)." (Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4, which opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5")

The count is restated in the figure legend:

"Of the 17 complexes, 12 have an ROC-AUC > 0.5. The SEC23/24 complex (bright green) has the highest ROC-AUC at 1, and the NUP84 complex (marigold) has the lowest AUC of 0.247." (Figure 4 legend)

The global test on the same AUCs:

"We compared the true average ROC-AUC from the 17 complexes (0.596) to the permuted distribution of average ROC-AUCs, which resulted in a permutation p-value of 0.08." (Results, same subsection, para 5, which opens "We then applied a third test for a general pattern")

The Discussion relies on the 12-of-17 result:

"Looking across all complexes in our study, a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings." (Discussion, para 2, which opens "We found that compensatory coevolution due to physical interaction contributes to ERC")

The abstract states the overall conclusion:

"We found no generalizable relationship between physical interaction and high ERC, although a few complexes ranked physical interactions higher than nonphysical interactions." (Abstract)

What the paper's own numbers show

Method. Under the null stated in the text, the number X of complexes with AUC > 0.5 is Binomial(n = 17, p = 0.5). The exact one-sided p-value for the observed 12 is P(X ≥ 12), which includes the observed outcome. It was computed by summing binomial coefficients exactly and checked with scipy.stats.binomtest. The same was done for the second test, X ~ Binomial(17, 0.05) with 4 observed. As an input check, the 17 per-complex AUCs printed in the version-of-record Figure 4 image were transcribed into recompute.py.txt. (The PMC text legend gives only the highest and lowest values.) They give 12 of 17 above 0.5, no values exactly at 0.5, and a mean of 0.596, matching the text. All numbers below are printed by recompute.py.txt when it is run. Paragraph numbers in this write-up are one-based, counted from the first paragraph of the named subsection, and each is given with its opening words.

Quantity Location Reported Recomputed
Complexes with ROC-AUC > 0.5 Results para 4 ("Of the 17 complexes, 12 had…"); Fig. 4 legend; Discussion para 2 12 of 17 12 of 17 (from the Fig. 4 AUCs)
Mean ROC-AUC Results para 5 ("We then applied a third test…") 0.596 0.596 (from the Fig. 4 AUCs)
Binomial p, 12 of 17 at p = 0.5, one-sided P(X ≥ 12) Results para 4 0.0245 9402/131072 = 0.0717
Same test, exact two-sided — — 0.1435
P(X ≥ 13) = P(X > 12) — — 3214/131072 = 0.02452 (equals the reported value)
Smallest count out of 17 with one-sided p < 0.05 — — 13
Binomial p, 4 of 17 significant at α = 0.05, one-sided P(X ≥ 4) Results para 4 0.0012 0.0088 (still < 0.05)
P(X ≥ 5) = P(X > 4) at p = 0.05 — — 0.00116 (equals the reported value)
Global permutation test on mean AUC Results para 5 0.08 not recomputable (needs per-complex ranked lists); consistent with the corrected 0.072
Complexes with AUC > 0.7 (side note) Results para 4 "five" 6 in the Fig. 4 AUCs (1.00, 0.821, 0.736, 0.723, 0.714, 0.713)
Complexes with physical > nonphysical mean ERC (side note) Results para 2 ("We split the proteins…") vs Table 1 12 of 17 14 of 17 in Table 1

Both reported binomial p-values equal the upper tail that starts one success above the observed count. That tail excludes the observed outcome, so it is not the p-value of the observation. For 12 of 17, the excluded term alone is P(X = 12) = 6188/131072 = 0.0472.

Figure

Reported and recomputed binomial p-values for the two complex-level tests

Figure 1. Reported p-values (orange) and exact binomial p-values recomputed from the paper's counts (blue), on one log-scaled axis; the dashed line marks α = 0.05. Top row: 12 of 17 complexes with ROC-AUC > 0.5 under the paper's stated null of one half. The reported 0.0245 falls below 0.05. The exact one-sided p (0.0717, filled blue) and two-sided p (0.143, open blue diamond) are both above it. Bottom row: 4 of 17 complexes individually significant at α = 0.05. The reported 0.0012 and the recomputed 0.0088 are both below 0.05, so only the size of that p-value changes. All values come from the paper's text or from recompute.py.txt.

Which tail of the Binomial(17, 0.5) distribution each p-value sums

Figure 2. The null distribution of the number of complexes with AUC > 0.5 out of 17, Binomial(17, 0.5). The blue bars (12 to 17) form the upper tail that includes the observed count, and they sum to the correct one-sided p = 0.0717 (blue bracket). The reported 0.0245 equals the sum over 13 to 17 only (orange bracket), leaving out the observed outcome, whose probability is 0.0472. This figure shows why a one-step shift in the tail boundary roughly triples the p-value here.

Why it matters

Results paragraphs 4 and 5 of the subsection use three complex-level tests to argue that physical coevolution contributes a detectable signal to ERC: the 12-of-17 sign test, the count of individually significant complexes, and the global permutation test. As reported, the first two are significant and the third is not, and the Discussion says the evidence "was tempered by a global permutation test". After correction, the 12-of-17 test (p = 0.072) is not significant either, so two of the three complex-wide tests fail to reach 0.05. The corrected value is in line with the permutation p of 0.08 rather than in tension with it.

Two statements depend on the 12-of-17 result as written. The Results paragraph calls it "a significant excess", and the Discussion's "a proportion higher than random chance ... this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5" cites it as evidence. At α = 0.05 the 12-of-17 result is not a proportion higher than chance. The Discussion's broader claim of a "non-negligible signal from physical interaction" is weakened but keeps some support. The 4-of-17 count remains significant (corrected p = 0.0088), and five complexes are reported as significant in the separate proportional-rank analysis.

The headline conclusion is that ERC does not reliably predict physical interactions ("no generalizable relationship"). The error does not undermine it. If anything the corrected value makes the evidence for a general physical-interaction signal weaker, which fits that conclusion. The error is not in the title or the abstract.

Explanations considered

The count is 13, not 12. Ruled out. The text, the Figure 4 legend and the Discussion all say 12 of 17. The 17 AUCs printed in the version-of-record Figure 4 image have exactly 12 above 0.5, and their mean (0.596) matches the value in the text, so the transcription is consistent.

A tie at AUC = 0.5 was dropped (n = 16). Does not account for it. No Figure 4 AUC equals 0.5 (the closest is SMC5/6 at 0.504). Even with n = 16, P(X ≥ 12 | 16, 0.5) = 0.0384, not 0.0245.

A two-sided test. Does not account for it. The two-sided exact p is 0.143, larger than the one-sided value.

A different null proportion. Ruled out by the text, which states the expectation as "half would exceed 0.5".

A convention that defines the test as strictly greater than the observed count. No such definition appears in the Methods, which describe only the per-complex Mann–Whitney tests and the permutations. P(X > k) is not a valid p-value for an observation of k, because it excludes the observed outcome. The value 0.0245 is what an upper-tail call returns when the observed count itself is passed as the cutoff, for example R's pbinom(12, 17, 0.5, lower.tail = FALSE), which returns P(X > 12). The correct call passes 11. R was not run here; recompute.py.txt evaluates the equivalent upper-tail function, scipy.stats.binom.sf, with both cutoffs (0.02452 for 12, 0.07173 for 11). The second test reproduces the same shift exactly (0.00116 vs 0.0088), which points to one consistent computational step rather than two separate typos.

A conversion or archiving artifact. Ruled out. The eLife version-of-record web page, the version-of-record PDF, the PMC page, the Europe PMC full-text XML and the reviewed preprint v2 all give the same numbers. The only markup difference is in the second test: in the Europe PMC XML its equals sign is wrapped in italic markup, while the eLife web page displays it as "p = 0.0012". The value is the same.

The analysis used different data. This cannot be settled from the paper and does not change the finding as published. The data deposit (see Status) contains a per-complex spreadsheet. An earlier audit stage recomputed AUCs from it and found values that differ slightly from Figure 4 (for example NUP84 0.260 vs 0.247), with 14 of 17 above 0.5. Table 1 also shows physical > nonphysical mean ERC in 14 of 17 complexes, where the text says 12. A count of 14 of 17 would give P(X ≥ 14) = 0.0064. But every statement in the paper, and the published Figure 4, reports 12 of 17, and the p-value is evaluated here for that reported count. If a different count was intended, the text, the figure and the Discussion would all need to change.

Status

As of 2026-10-23:

Sources: PMC10942632 (PubMed Central archive, read with src/show.py); https://elifesciences.org/articles/93333; https://cdn.elifesciences.org/articles/93333/elife-93333-v1.pdf; reviewed preprint and reviews at https://elifesciences.org/reviewed-preprints/93333/reviews; preprint doi:10.1101/2023.10.18.562970; data deposit https://github.com/nclark-lab/erc (subfolder physical_interaction_paper, named in Methods, "Calculating ERC").

Suggested correction

In Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4 (opening "Of the 17 complexes, 12 had an ROC-AUC > 0.5"), replace "this is a significant excess (binomial test, p=0.0245)" with "this excess is not statistically significant (one-sided exact binomial test, p=0.072)". Replace "(binomial test, p=0.0012)" with "(one-sided exact binomial test, p=0.0088)". In the Discussion, para 2 (opening "We found that compensatory coevolution due to physical interaction contributes to ERC"), revise "a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings" so that only the 4 individually significant complexes are cited as an excess over chance, and note that the 12-of-17 count is not significant. The authors could also:

Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC10942632 and regenerate fig1/fig2.

Run:  python recompute.py.txt   (writes fig1.svg/png and fig2.svg/png next to this file)

Inputs come from the article (eLife 2024, doi:10.7554/eLife.93333):
  * Results, "ERC does not distinguish physical interactions ...", para 4
    (opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5"):
    12 of 17 complexes with ROC-AUC > 0.5, "binomial test, p=0.0245";
    4 of 17 complexes significant at alpha = 5%, "binomial test, p=0.0012".
  * Figure 4 (version-of-record image): the 17 per-complex ROC-AUCs listed below.
  * Same subsection, para 5 (opens "We then applied a third test"):
    mean ROC-AUC 0.596, global permutation p = 0.08.
  * Same subsection, para 2 (opens "We split the proteins"): "12 had higher
    average ERC" for physically interacting domains.
  * Table 1: mean ERC of physically vs nonphysically interacting domains.
Paragraph numbers are one-based, counted from the first paragraph of the subsection.
"""
import os
from math import comb
from fractions import Fraction
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D

HERE = os.path.dirname(os.path.abspath(__file__))
N = 17

# ------------------------------------------------------------ reported values
REP_P1 = 0.0245   # 12 of 17 AUC > 0.5, null proportion 0.5
REP_P2 = 0.0012   # 4 of 17 significant, null proportion 0.05
REP_PERM = 0.08   # global permutation test on mean AUC (not recomputable here)

# Figure 4 (VOR image legend), per-complex ROC-AUC
AUC = {"SEC23/24": 1.00, "ESCRT-I": 0.821, "Exportin": 0.736, "CUL8": 0.723,
       "GET4_GET5": 0.714, "SWI_SNF": 0.713, "MCM": 0.666, "TREX": 0.600,
       "ORC": 0.565, "PAN1": 0.543, "COMA": 0.540, "SMC5/6": 0.504,
       "EXOCYST": 0.480, "ATG17/31/29": 0.478, "F1 ATPase": 0.467,
       "EIF3": 0.341, "NUP84": 0.247}
# Table 1: (physically interacting domain ERC, nonphysically interacting domain ERC)
TABLE1 = {"EIF3": (8.852, 11.125), "MCM": (10.283, 8.000), "NUP84": (1.541, 4.411),
          "ORC": (5.250, 4.035), "PAN1": (4.963, 4.716), "SMC5-6": (5.153, 4.110),
          "TREX": (4.157, 4.129), "EXOCYST": (6.221, 5.974), "COMA": (3.051, 2.869),
          "SWI/SNF": (4.121, 1.783), "CUL8": (2.611, 1.576), "GET4-GET5": (4.569, 3.302),
          "ATG17-31-29": (3.444, 1.584), "ESCRT-I": (4.041, 2.402),
          "F1 ATPase": (4.883, 5.654), "SEC23/24": (10.530, 7.754),
          "Exportin": (8.871, 6.326)}

# ------------------------------------------------------------ Figure 4 checks
k_auc = sum(v > 0.5 for v in AUC.values())
print("== Inputs from Figure 4 ==")
print(f"complexes listed: {len(AUC)}")
print(f"AUC > 0.5: {k_auc} of {len(AUC)}   (text: 12 of 17)")
print(f"AUC = 0.5 exactly (ties): {sum(v == 0.5 for v in AUC.values())}")
print(f"mean AUC: {sum(AUC.values())/len(AUC):.3f}   (text: 0.596)")
print(f"AUC > 0.7: {sum(v > 0.7 for v in AUC.values())}   (text: 'five complexes had AUCs > 0.7')")
print()

# ------------------------------------------------------------ test 1: 12 of 17, p0 = 0.5
k1 = 12
terms = [comb(N, k) for k in range(k1, N + 1)]
p_ge12 = Fraction(sum(terms), 2 ** N)
p_ge13 = Fraction(sum(comb(N, k) for k in range(k1 + 1, N + 1)), 2 ** N)
two_sided = binomtest(k1, N, 0.5).pvalue
print("== Test 1: 12 of 17 complexes with AUC > 0.5, null proportion 0.5 ==")
print(f"C(17,k), k=12..17: {terms}; sum = {sum(terms)}; 2^17 = {2**N}")
print(f"one-sided P(X>=12) = {sum(terms)}/{2**N} = {float(p_ge12):.4f}"
      f"   (scipy binomtest greater: {binomtest(k1, N, 0.5, alternative='greater').pvalue:.4f})")
print(f"two-sided exact p  = {two_sided:.4f}")
print(f"P(X>=13) = P(X>12) = {sum(terms[1:])}/{2**N} = {float(p_ge13):.5f}"
      f"   (reported: {REP_P1})")
print(f"P(X=12), the observed outcome left out of P(X>12) = {comb(N, 12)}/{2**N} = "
      f"{binom.pmf(12, N, 0.5):.4f}")
# R's pbinom(q, n, p, lower.tail = FALSE) returns P(X > q), i.e. scipy's binom.sf(q, n, p)
print(f"upper tail with the observed count as cutoff, P(X>12) "
      f"[R: pbinom(12, 17, 0.5, lower.tail = FALSE)] = {binom.sf(12, N, 0.5):.5f}")
print(f"upper tail with cutoff 11, P(X>11) = P(X>=12) "
      f"[R: pbinom(11, 17, 0.5, lower.tail = FALSE)] = {binom.sf(11, N, 0.5):.5f}")
print(f"tie excluded, P(X>=12 | n=16) = {binom.sf(11, 16, 0.5):.4f}")
print(f"smallest k out of 17 with P(X>=k) < 0.05: "
      f"{min(k for k in range(N + 1) if binom.sf(k - 1, N, 0.5) < 0.05)}")
print()

# ------------------------------------------------------------ test 2: 4 of 17, p0 = 0.05
k2 = 4
p2_ge4 = binom.sf(k2 - 1, N, 0.05)
p2_ge5 = binom.sf(k2, N, 0.05)
print("== Test 2: 4 of 17 complexes significant at alpha 0.05, null proportion 0.05 ==")
print(f"one-sided P(X>=4) = {p2_ge4:.4f}")
print(f"P(X>=5) = P(X>4)  = {p2_ge5:.5f}   (reported: {REP_P2})")
print()

# ------------------------------------------------------------ side check: Table 1
k_t1 = sum(a > b for a, b in TABLE1.values())
print("== Side check: Table 1 ==")
print(f"physical ERC > nonphysical ERC: {k_t1} of {len(TABLE1)}   (text: 12 of 17)")
print(f"for reference, a count of 14 of 17 would give P(X>=14 | 17, 0.5) = "
      f"{sum(comb(N, k) for k in range(14, N + 1))}/{2**N} = {binom.sf(13, N, 0.5):.4f}")
print()
print(f"Global permutation test (reported, not recomputable): p = {REP_PERM}")

# ------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": INK, "axes.labelcolor": INK,
                     "xtick.color": INK2, "ytick.color": INK2, "axes.edgecolor": INK2,
                     "svg.fonttype": "none", "font.family": "DejaVu Sans"})


def save(fig, name):
    for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
        fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
    plt.close(fig)


# Figure 1: reported vs recomputed p-values, one log axis, alpha = 0.05 reference
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
ax.set_facecolor(BG)
rows = [("12 of 17 complexes with ROC-AUC > 0.5\n(null proportion 0.5)",
         REP_P1, float(p_ge12), two_sided),
        ("4 of 17 complexes significant at α = 0.05\n(null proportion 0.05)",
         REP_P2, p2_ge4, None)]
ys = [1, 0]
for y, (lab, rep, rec, rec2) in zip(ys, rows):
    hi = max(rep, rec, rec2 or 0)
    ax.plot([min(rep, rec), hi], [y, y], color=GRID, lw=2, zorder=1)
    ax.plot(rep, y, "o", ms=10, color=ORANGE, zorder=3)
    ax.plot(rec, y, "o", ms=10, color=BLUE, zorder=3)
    ax.annotate(f"reported {rep}", (rep, y), xytext=(0, 11), textcoords="offset points",
                ha="center", color=INK, fontsize=10)
    ax.annotate(f"P(X≥{12 if y else 4}) = {rec:.4f}", (rec, y), xytext=(0, -18),
                textcoords="offset points", ha="center", color=INK, fontsize=10)
    if rec2 is not None:
        ax.plot(rec2, y, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
        ax.annotate(f"two-sided\n{rec2:.3f}", (rec2, y), xytext=(10, 0),
                    textcoords="offset points", ha="left", va="center", color=INK,
                    fontsize=10)
ax.plot([0.05, 0.05], [-0.6, 1.5], color=INK2, lw=1, ls="--", zorder=0)
ax.text(0.05, 1.55, "α = 0.05", ha="center", va="bottom", color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(5e-4, 0.8)
ax.set_ylim(-0.6, 1.75)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xticks([0.001, 0.003, 0.01, 0.03, 0.1, 0.3])
ax.set_xticklabels(["0.001", "0.003", "0.01", "0.03", "0.1", "0.3"])
ax.minorticks_off()
ax.set_xlabel("binomial p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
for s in ("top", "right", "left"):
    ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in paper"),
           Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
                  label="recomputed, one-sided"),
           Line2D([], [], marker="D", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
                  label="recomputed, two-sided")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.22, -0.25), ncol=3,
          frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")

# Figure 2: Binomial(17, 0.5) distribution; which tail each p-value sums
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
ks = list(range(N + 1))
pm = [binom.pmf(k, N, 0.5) for k in ks]
cols = ["#c9c8c3" if k < 12 else BLUE for k in ks]
ax.bar(ks, pm, width=0.8, color=cols, edgecolor=BG, lw=2, zorder=2)
ax.bar([12], [pm[12]], width=0.8, color=BLUE, edgecolor=BG, lw=2, zorder=2)
ytop = max(pm)
yb1, yb2 = pm[12] + 0.035, pm[12] + 0.07
ax.plot([12.6, 17.4], [yb1, yb1], color=ORANGE, lw=2)
for x in (12.6, 17.4):
    ax.plot([x, x], [yb1 - 0.006, yb1], color=ORANGE, lw=2)
ax.text(15, yb1 + 0.004, f"reported p = P(X≥13) = {float(p_ge13):.4f}", ha="center",
        va="bottom", fontsize=10, color=INK)
ax.plot([11.6, 17.4], [yb2, yb2], color=BLUE, lw=2)
for x in (11.6, 17.4):
    ax.plot([x, x], [yb2 - 0.006, yb2], color=BLUE, lw=2)
ax.text(14.5, yb2 + 0.004, f"correct p = P(X≥12) = {float(p_ge12):.4f}", ha="center",
        va="bottom", fontsize=10, color=INK)
ax.annotate(f"observed: 12 of 17\nP(X=12) = {binom.pmf(12, N, 0.5):.4f}",
            (12, pm[12]), xytext=(9.0, pm[12] + 0.02), ha="right", fontsize=10,
            color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_xticks(ks)
ax.set_xlabel("number of complexes with ROC-AUC > 0.5 (out of 17)")
ax.set_ylabel("probability under null (p = 0.5)")
ax.set_ylim(0, ytop * 1.08)
ax.set_xlim(-0.7, 17.7)
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
    ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))

Abstract reports the deductive-reasoning test as F(2,387), an error df larger than the whole sample

Brain (2025) · doi:10.1093/brain/awaf062 · PMC12073978

Summary

The Abstract reports the omnibus group comparison on the Deductive Reasoning Test (DRT) as F(2,387) = 18.491, P < 0.001. The Results report the same test as F(2,142) = 7.73, P < 0.001. The Results value fits the paper's own numbers exactly: Table 2 lists 48 + 55 + 43 = 146 participants, and an age-adjusted ANCOVA across three groups leaves 146 − 4 = 142 error degrees of freedom. An error df of 387 would need 391 participants, more than the 328 people recruited in total. Significance and direction are the same under either value, and the effect size each tuple implies is similar (partial η² 0.087 and 0.098). But the Abstract makes the evidence for the deductive-reasoning difference look as strong as that for the analogical-reasoning difference (F = 18.93) and implies a sample more than twice the size of the one analysed.

What the paper reports

ART and DRT performance was significantly impaired in patients with frontal lesions [ART: F(2,238) = 18.93; P < 0.001; Frontal group worse than Posterior group and healthy controls, both P < 0.001; DRT: F(2,387) = 18.491; P < 0.001; Frontal group worse than healthy controls, P < 0.01].

— Abstract

There was a significant difference between the Frontal, Posterior and HC groups for overall DRT performance [F(2,142) = 7.73; P < 0.001; see Table 2].

— Results, "Deductive reasoning task", first paragraph

HC(n = 48) | Frontal(n = 55) | Posterior(n = 43) | Left frontal(n = 24) | Right frontal(n = 31) | Left posterior(n = 19) | Right posterior(n = 24)

— Table 2, "Performance on the Deductive Reasoning Task", column headers. The table footer reads: "Groups are compared using analysis of covariance (ANCOVA), controlling for age."

Frontal, Posterior and HC groups were compared on demographic variables, using ANOVA and Fisher’s exact test, and performance on neuropsychological tests, ART and DRT, using analysis of covariance (ANCOVA), controlling for age.

— Methods, "Behavioural analysis"

Behavioural analyses were conducted on the 176 patients with lesions falling within frontal or posterior areas and HCs using SPSS version 29 (https://www.ibm.com/spss).

— Methods, "Behavioural analysis"

Two-hundred and forty-seven patients with unilateral, focal lesions were prospectively recruited from the National Hospital for Neurology and Neurosurgery

Eighty-one HCs, with no neurological or psychiatric history, were recruited to match patients as closely as possible for age, gender, years of education and National Adult Reading Test (NART) scores.

— Methods, "Participants"

What the paper's own numbers show

Method. In a one-way ANCOVA with k = 3 groups and one covariate (age), the F test for the group effect has k − 1 = 2 numerator df and N − k − 1 = N − 4 error df. The group sizes printed in the table headers therefore fix the error df. The same rule was checked against every other F test in the Results whose groups are tabulated. The age-adjusted F itself cannot be rebuilt from summary tables, because that needs the covariance of age with the scores. As a reference point only, the unadjusted one-way ANOVA F was computed from each table's n, means and SDs. Every number below is printed by recompute.py.txt (Python, numpy/scipy).

Quantity Location Reported Recomputed from the paper's numbers
DRT sample size Table 2 header HC 48, Frontal 55, Posterior 43 N = 146; subgroups agree (24 + 31 = 55; 19 + 24 = 43)
DRT error df, ANCOVA with age Results 142 146 − 4 = 142 (matches)
DRT error df Abstract 387 would need N = 391 (ANCOVA) or 390 (no covariate)
Largest possible sample Methods, Participants 247 patients + 81 HCs 328 in total, so a three-group ANCOVA on all of them has error df 324
Behavioural analysis set Methods, Behavioural analysis 176 patients + HCs 176 + 81 = 257
P for F(2,142) = 7.73 Results P < 0.001 6.5 × 10⁻⁴
P for F(2,387) = 18.491 Abstract P < 0.001 2.1 × 10⁻⁸
Partial η² implied by each tuple Results / Abstract — 0.098 / 0.087
Ratio of the two DRT F values Abstract vs Results 18.491 vs 7.73 2.39
Unadjusted DRT F from Table 2 n, means, SDs Table 2 — F(2,143) = 6.80, P = 0.0015 (Results F is 1.14 times this; Abstract F is 2.72 times)
Control: ART sample and error df Table 1; Abstract; Results n 72 + 100 + 70; df 238 N = 242; 242 − 4 = 238 (matches)
Control: unadjusted ART F from Table 1 Table 1 F(2,238) = 18.93 F(2,239) = 17.36 (reported is 1.09 times this)
Control: DRT indeterminate / determinate Results F(2,142) = 7.93 / 4.23 unadjusted 7.18 / 3.81 (ratios 1.10 / 1.11); df 142 matches
Control: ART progression / odd-one-out Results F(2,238) = 15.20 / 11.74 unadjusted 14.16 / 11.13 (ratios 1.07 / 1.06); df 238 matches
Control: pooled Frontal and Posterior DRT means Table 2 71.67 / 79.37 71.68 / 79.37 from the hemisphere subgroups
Other error dfs in Results (RAPM, S fluency, Stroop) Results 184, 151, 150 imply N = 188, 155, 154, all ≤ 257

Every ART and DRT F test in the Results has error df = N − 4 for its table, and each of the six tabulated ANCOVA Fs lies 6–14% above the unadjusted F from the same table. The Abstract's DRT tuple is the only one that fits neither pattern.

Figure

The Abstract's DRT statistic and the Results' DRT statistic side by side, with the sample size each implies

Figure 1. The two printed versions of the DRT omnibus test. Left: the Abstract's F(2,387) = 18.491 (orange). Under the ANCOVA described in Methods (three groups, age as covariate) an error df of 387 corresponds to N = 391 (gray, derived from the Abstract's df). Right: the Results' F(2,142) = 7.73 and the Table 2 group sizes (black, as printed), from which N = 146 and error df 146 − 4 = 142 are recomputed (blue). The study recruited 328 participants in total (247 patients and 81 healthy controls), so no analysis described in the paper could have 387 error df.

Reported error df for each group comparison in the paper against N − 4 from the corresponding table

Figure 2. Reported error df (orange circles) and the error df recomputed as N − 4 from the group sizes in Table 1 or Table 2 (blue diamonds) for every ART and DRT omnibus F test in the paper. The seven further ART tests (progression, odd-one-out, intra- and cross-dimensional and others) all report df 238 and are drawn as one row. The dotted line marks 324, the error df of a three-group ANCOVA with age on all 328 recruited participants. The Abstract's DRT value is the only one that departs from N − 4, and it lies beyond that ceiling.

Why it matters

The Abstract gives the DRT omnibus test as one of two headline behavioural results, set beside the ART result as evidence that "ART and DRT performance was significantly impaired in patients with frontal lesions". Read from the Abstract, the evidence for the two group differences looks about equally strong (F = 18.93 and F = 18.491, both with P near 2 × 10⁻⁸). By the paper's own Results, the evidence for the DRT difference is clearly weaker (F = 7.73, P = 6.5 × 10⁻⁴), and it rests on 146 participants, not the roughly 391 that the printed df implies. The DRT effect size is smaller than the ART one under either tuple (partial η² 0.087 or 0.098, against 0.137 for the ART).

The stated conclusions do not change. Both tuples give P < 0.001, the Frontal group is worse than healthy controls under either, and the implied partial η² is similar (0.087 and 0.098). The paper's main findings — the right-versus-left frontal differences on the ART and on DRT indeterminate questions, and the lesion-deficit network mapping — rest on other analyses.

The readers most likely to be affected are those who take test statistics or sample sizes from abstracts, such as reviewers judging sample size and precision, or meta-analysts. A meta-analyst who converts the Abstract tuple gets a slightly smaller partial η² (0.087 instead of 0.098) and a much larger N (391 instead of 146), which would give the DRT result far more weight than its sample supports. The Abstract also presents the DRT as a promising clinical test. For a reader weighing that claim, the Abstract overstates the strength of evidence and the sample size behind the DRT result, while the effect size it implies is close to the Results value.

Explanations considered

The Abstract's 18.491 is numerically close to the ART statistic, 18.93. That is an observation only; the paper's numbers do not show how the Abstract value arose.

Status

As of 2026-10-23:

Sources: doi.org/10.1093/brain/awaf062 · publisher article page · PMC12073978 · Europe PMC record · Crossref works record · scientific commentary

Suggested correction

The authors could check the DRT omnibus ANCOVA in the Abstract against the original output and align it with the Results. If the Results value is correct, the Abstract would read "DRT: F(2,142) = 7.73; P < 0.001; Frontal group worse than healthy controls, P < 0.01". If 18.491 comes from a different analysis, the Abstract should say which analysis it is and how many participants it included, since no analysis described in the paper has an error df of 387.

Recomputation script (python recompute.py.txt)
"""Recompute the Deductive Reasoning Test (DRT) omnibus statistics of PMC12073978
(Brain 2025, doi:10.1093/brain/awaf062) and regenerate the figures.

Run:   python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand_A ... cand_E, .svg/.png) into ./candidates/.

All inputs are copied from the paper (Abstract; Methods, Participants and
Behavioural analysis; Results; Tables 1 and 2).
Model stated in Methods: one-way ANCOVA across three groups (Frontal, Posterior,
healthy controls) with age as the single covariate, so
    group df = k - 1 = 2,  error df = N - k - 1 = N - 4.
The age-adjusted F cannot be rebuilt from the tables (it needs the age-score
covariance), so the unadjusted one-way ANOVA F is computed from the printed
group n, means and SDs as a reference point only.
"""
import os

import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")

# ---------------------------------------------------------------- inputs
N_PATIENTS_RECRUITED = 247          # Methods, Participants
N_PATIENTS_FP = 176                 # frontal 102 + posterior 74
N_HC = 81                           # Methods, Participants
DRT_N = {"HC": 48, "Frontal": 55, "Posterior": 43}         # Table 2 header
DRT_SUB = {"LF": 24, "RF": 31, "LP": 19, "RP": 24}
ART_N = {"HC": 72, "Frontal": 100, "Posterior": 70}        # Table 1 header

ABSTRACT_DRT = (2, 387, 18.491)     # Abstract
RESULTS_DRT = (2, 142, 7.73)        # Results, Deductive reasoning task, p0
ART_REPORTED = (2, 238, 18.93)      # Abstract and Results

# (label, ns, means, sds, reported F, reported df2, location)
TESTS = [
    ("DRT overall", list(DRT_N.values()), [82.79, 71.67, 79.37], [15.22, 15.90, 16.03], 7.73, 142, "Results"),
    ("DRT indeterminate", list(DRT_N.values()), [76.06, 55.89, 70.56], [25.21, 30.53, 27.60], 7.93, 142, "Results"),
    ("DRT determinate", list(DRT_N.values()), [84.79, 76.69, 82.14], [15.42, 15.12, 15.17], 4.23, 142, "Results"),
    ("ART overall", list(ART_N.values()), [79.40, 67.79, 77.38], [11.93, 15.51, 13.51], 18.93, 238, "Abstract; Results"),
    ("ART progression", list(ART_N.values()), [83.22, 71.94, 81.19], [12.86, 16.60, 14.52], 15.20, 238, "Results"),
    ("ART odd-one-out", list(ART_N.values()), [75.58, 63.71, 73.57], [15.09, 20.06, 17.08], 11.74, 238, "Results"),
]


def anova_from_summary(ns, means, sds):
    ns, m, s = (np.asarray(x, float) for x in (ns, means, sds))
    N, k = ns.sum(), len(ns)
    gm = (ns * m).sum() / N
    ssb = (ns * (m - gm) ** 2).sum()
    ssw = ((ns - 1) * s ** 2).sum()
    F = (ssb / (k - 1)) / (ssw / (N - k))
    return int(N), k, F, stats.f.sf(F, k - 1, N - k)


def p_of(F, d1, d2):
    return stats.f.sf(F, d1, d2)


def eta_p2(F, d1, d2):
    return F * d1 / (F * d1 + d2)


# ---------------------------------------------------------------- numbers
print("== Sample sizes ==")
n_drt = sum(DRT_N.values())
n_art = sum(ART_N.values())
print(f"DRT analysed (Table 2): 48 + 55 + 43 = {n_drt}")
print(f"  subgroup check: LF + RF = {DRT_SUB['LF'] + DRT_SUB['RF']} (Frontal 55); "
      f"LP + RP = {DRT_SUB['LP'] + DRT_SUB['RP']} (Posterior 43)")
print(f"ART analysed (Table 1): 72 + 100 + 70 = {n_art}")
max_behav = N_PATIENTS_FP + N_HC
all_recruited = N_PATIENTS_RECRUITED + N_HC
print(f"Behavioural analysis set (176 F/P patients + 81 HC) = {max_behav}")
print(f"All recruited (247 patients + 81 HC) = {all_recruited}")

print("\n== Error df under the stated ANCOVA (N - 3 groups - 1 covariate) ==")
print(f"DRT: {n_drt} - 4 = {n_drt - 4}  (Results reports 142)")
print(f"ART: {n_art} - 4 = {n_art - 4}  (Abstract and Results report 238)")
n_implied_abs = ABSTRACT_DRT[1] + 4
print(f"N implied by abstract F(2,387): ANCOVA {n_implied_abs}; ANOVA without covariate {ABSTRACT_DRT[1] + 3}")
print(f"  excess over DRT sample: {n_implied_abs - n_drt}; over all recruited: {n_implied_abs - all_recruited}")
print(f"N implied by Results F(2,142): {RESULTS_DRT[1] + 4}")
print("Within-subject pooling check: 146 participants x 2 question types = "
      f"{2 * n_drt} observations -> error df {2 * n_drt - 4}")

print("\n== P values and partial eta-squared of the reported tuples ==")
for lab, (d1, d2, F) in [("Abstract DRT", ABSTRACT_DRT), ("Results DRT", RESULTS_DRT), ("ART", ART_REPORTED)]:
    print(f"{lab:13s} F({d1},{d2}) = {F}: P = {p_of(F, d1, d2):.2e}, partial eta^2 = {eta_p2(F, d1, d2):.3f}")
print(f"Abstract DRT F / Results DRT F = {ABSTRACT_DRT[2] / RESULTS_DRT[2]:.2f}")
print(f"Abstract DRT F vs ART F: {ABSTRACT_DRT[2]} vs {ART_REPORTED[2]} (difference {ART_REPORTED[2] - ABSTRACT_DRT[2]:.3f})")

print("\n== Unadjusted one-way ANOVA from the tables' n, mean, SD (reference only) ==")
UNADJ = {}
for lab, ns, m, s, Frep, d2rep, loc in TESTS:
    N, k, F, p = anova_from_summary(ns, m, s)
    UNADJ[lab] = F
    print(f"{lab:18s} N={N} unadjusted F(2,{N - k}) = {F:5.2f}, P = {p:.2e} | "
          f"reported ANCOVA F(2,{d2rep}) = {Frep} ({loc}); ratio reported/unadjusted = {Frep / F:.2f}; "
          f"expected error df N-4 = {N - 4}")
print(f"Abstract DRT F / unadjusted DRT F = {ABSTRACT_DRT[2] / UNADJ['DRT overall']:.2f}")

print("\n== Pooled group means from the hemisphere subgroups (Table 2) ==")
print(f"DRT Frontal:   (24*75.88 + 31*68.42)/55 = {(24 * 75.88 + 31 * 68.42) / 55:.2f}  (reported 71.67)")
print(f"DRT Posterior: (19*77.05 + 24*81.21)/43 = {(19 * 77.05 + 24 * 81.21) / 43:.2f}  (reported 79.37)")

# Other error dfs reported in Results (implied N under the same model)
OTHER_DF = {"RAPM": 184, "S fluency": 151, "Stroop": 150}
print("\n== Other reported error dfs -> implied N (N = df + 4) ==")
for k_, v in OTHER_DF.items():
    print(f"{k_}: df {v} -> N {v + 4} (<= {max_behav})")

# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch

BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
    "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
    "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2


def save(fig, path_noext):
    os.makedirs(os.path.dirname(path_noext), exist_ok=True)
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def style_axis(ax):
    ax.grid(axis="x", color=GRID, lw=0.8)
    ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)


def panel(ax, x, y, w, h, title, lines, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.02",
                                fc="white", ec=edge, lw=1.6))
    ax.text(x + 0.03, y + h - 0.06, title, fontsize=11.5, weight="bold", va="top")
    yy = y + h - 0.17
    for txt, col, wt in lines:
        ax.text(x + 0.03, yy, txt, fontsize=10.5, color=col, weight=wt, va="top", family="DejaVu Sans")
        yy -= 0.085


# --- Candidate A: two text panels joined by an annotated connector
def cand_A(path):
    fig = plt.figure(figsize=(W, 3.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
    panel(ax, 0.03, 0.30, 0.40, 0.62, "Abstract", [
        ("“DRT: F(2,387) = 18.491;", ORANGE, "bold"),
        ("P < 0.001”", ORANGE, "bold"),
        ("", TXT, "normal"),
        ("error df 387 implies", SEC, "normal"),
        (f"N = 387 + 4 = {n_implied_abs} participants", SEC, "normal"),
    ], ORANGE)
    panel(ax, 0.57, 0.30, 0.40, 0.62, "Results  /  Table 2", [
        ("“F(2,142) = 7.73; P < 0.001”", TXT, "bold"),
        ("Table 2: n = 48 + 55 + 43", TXT, "normal"),
        ("", TXT, "normal"),
        (f"recomputed: N = {n_drt}", BLUE, "bold"),
        (f"error df = {n_drt} − 4 = {n_drt - 4} ✓", BLUE, "bold"),
    ], BLUE)
    ax.annotate("", xy=(0.565, 0.61), xytext=(0.435, 0.61),
                arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
    ax.text(0.50, 0.66, "same test", ha="center", fontsize=10, color=SEC)
    ax.text(0.50, 0.185, f"Conflict: the abstract's df needs {n_implied_abs} participants; the DRT sample is {n_drt},\n"
            f"and only {all_recruited} people (247 patients + 81 controls) were recruited in total.",
            ha="center", va="center", fontsize=10.5)
    ax.text(0.03, 0.025, "Orange: printed in the Abstract.  Black: printed in the Results and Table 2.\n"
            "Blue: recomputed from the paper's group sizes.  Gray: derived from the Abstract's df.",
            fontsize=10, color=SEC, va="bottom")
    save(fig, path)


# --- Candidate B: participants on one axis
def cand_B(path):
    rows = [
        ("Implied by abstract F(2,387)", n_implied_abs, ORANGE, "o", "reported df + 4"),
        ("All recruited (247 + 81)", all_recruited, SEC, "s", "Methods"),
        ("Behavioural set (176 + 81)", max_behav, SEC, "s", "Methods"),
        ("Implied by Results F(2,142)", RESULTS_DRT[1] + 4, TXT, "o", "reported df + 4"),
        ("DRT groups, Table 2 (48+55+43)", n_drt, BLUE, "D", "recomputed sum"),
    ]
    fig, ax = plt.subplots(figsize=(W, 3.6))
    for i, (lab, v, c, mk, note) in enumerate(rows[::-1]):
        ax.hlines(i, 0, v, color=GRID, lw=2)
        ax.plot(v, i, mk, color=c, ms=10, mfc=c if mk != "o" or c == ORANGE else "white", mew=2)
        ax.text(v + 8, i, f"{v}", va="center", fontsize=10.5, color=c, weight="bold")
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows[::-1]])
    ax.axvline(all_recruited, color=SEC, lw=1, ls=":")
    ax.text(all_recruited - 4, len(rows) - 0.45, "everyone recruited", ha="right", fontsize=10, color=SEC)
    ax.set_xlim(0, 440); ax.set_ylim(-0.6, len(rows) - 0.3)
    ax.set_xlabel("Participants")
    style_axis(ax)
    fig.text(0.01, 0.01, "Orange: from the abstract's df.  Blue: recomputed from Table 2.\nGray: counts in Methods.  Black open circle: from the Results df.",
             fontsize=10, color=SEC)
    fig.tight_layout(rect=(0, 0.11, 1, 1))
    save(fig, path)


# --- Candidate C: calculation flow
def cand_C(path):
    fig = plt.figure(figsize=(W, 3.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")

    def box(x, y, w, h, t, ec, col=TXT, wt="normal"):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
                                    fc="white", ec=ec, lw=1.6))
        ax.text(x + w / 2, y + h / 2, t, ha="center", va="center", fontsize=10.5, color=col, weight=wt)

    def arrow(x0, x1, y, t=None):
        ax.annotate("", xy=(x1, y), xytext=(x0, y), arrowprops=dict(arrowstyle="->", color=TXT, lw=2))
        if t:
            ax.text((x0 + x1) / 2, y + 0.11, t, ha="center", va="bottom", fontsize=10, color=SEC)

    # columns: x, width
    cols = [(0.02, 0.21), (0.30, 0.15), (0.53, 0.15), (0.76, 0.22)]
    ya, yb, h = 0.60, 0.17, 0.19
    ax.text(0.02, 0.92, "From the paper's group sizes (Table 2) to the Results", fontsize=11, weight="bold")
    box(*[cols[0][0], ya, cols[0][1], h], "Table 2 groups\n48 + 55 + 43", SEC)
    box(cols[1][0], ya, cols[1][1], h, f"N = {n_drt}", BLUE, BLUE, "bold")
    box(cols[2][0], ya, cols[2][1], h, f"error df\n= {n_drt - 4}", BLUE, BLUE, "bold")
    box(cols[3][0], ya, cols[3][1], h, "Results:\nF(2,142) = 7.73  ✓", TXT, TXT, "bold")
    arrow(0.235, 0.295, ya + h / 2, "sum")
    arrow(0.455, 0.525, ya + h / 2, "− 4")
    arrow(0.685, 0.755, ya + h / 2)

    ax.text(0.02, 0.49, "From the abstract's df back to a sample size", fontsize=11, weight="bold")
    box(cols[3][0], yb, cols[3][1], h, "Abstract:\nF(2,387) = 18.491", ORANGE, ORANGE, "bold")
    box(cols[2][0], yb, cols[2][1], h, "error df\n= 387", ORANGE, ORANGE, "bold")
    box(cols[1][0], yb, cols[1][1], h, f"N = {n_implied_abs}", ORANGE, ORANGE, "bold")
    box(cols[0][0], yb, cols[0][1], h, f"more than all\n{all_recruited} recruited  ✗", ORANGE, TXT)
    arrow(0.755, 0.685, yb + h / 2)
    arrow(0.525, 0.455, yb + h / 2, "+ 4")
    arrow(0.295, 0.235, yb + h / 2)
    ax.text(0.02, 0.04, "− 4 = three groups and one age covariate (ANCOVA, Methods).  Blue: recomputed from\n"
            "the paper's group sizes.  Orange: printed in the abstract or derived from it.",
            fontsize=10, color=SEC, va="bottom")
    save(fig, path)


# --- Candidate D: F statistics, reported vs unadjusted reference
def cand_D(path):
    rows = [("DRT overall (Abstract)", ABSTRACT_DRT[2], UNADJ["DRT overall"], True)]
    for lab, ns, m, s, Frep, d2, loc in TESTS:
        rows.append((f"{lab} ({loc.replace('; ', ', ')})", Frep, UNADJ[lab], False))
    rows = rows[::-1]
    fig, ax = plt.subplots(figsize=(W, 4.3))
    for i, (lab, Frep, Fu, hl) in enumerate(rows):
        ax.hlines(i, min(Frep, Fu), max(Frep, Fu), color=GRID, lw=2)
        ax.plot(Fu, i, "D", color=BLUE, ms=8)
        ax.plot(Frep, i, "s" if hl else "o", color=ORANGE, ms=9)
        if hl:
            ax.text((Frep + Fu) / 2, i + 0.17, f"{Frep} = {Frep / Fu:.1f} × the unadjusted F of {Fu:.2f}",
                    ha="center", va="bottom", fontsize=10, color=ORANGE, weight="bold")
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
    for t in ax.get_yticklabels():
        if t.get_text().startswith("DRT overall (Abstract"):
            t.set_weight("bold")
    ax.set_xlim(0, 25); ax.set_ylim(-0.5, len(rows) - 0.15)
    ax.set_xlabel("F statistic for the group effect")
    style_axis(ax)
    ax.plot([], [], "s", color=ORANGE, ms=9, label="reported F (Abstract; no stated analysis gives df 387)")
    ax.plot([], [], "o", color=ORANGE, ms=9, label="reported F (Results, ANCOVA with age)")
    ax.plot([], [], "D", color=BLUE, ms=8, label="unadjusted F recomputed from table n, mean, SD")
    ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=1, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


# --- Candidate E: error df audit, reported vs N - 4
def cand_E(path):
    rows = [
        ("DRT overall (Abstract)", 387, n_drt - 4),
        ("DRT overall (Results)", 142, n_drt - 4),
        ("DRT indeterminate (Results)", 142, n_drt - 4),
        ("DRT determinate (Results)", 142, n_drt - 4),
        ("ART overall (Abstract, Results)", 238, n_art - 4),
        ("ART, 7 further tests (Results)", 238, n_art - 4),
    ][::-1]
    fig, ax = plt.subplots(figsize=(W, 3.5))
    for i, (lab, rep, exp) in enumerate(rows):
        ax.hlines(i, min(rep, exp), max(rep, exp), color=GRID, lw=2)
        ax.plot(exp, i, "D", color=BLUE, ms=11)
        ax.plot(rep, i, "o", color=ORANGE, ms=8)
        if rep != exp:
            ax.text(exp + 12, i - 0.3, f"387 vs {exp}", ha="left", va="center",
                    fontsize=10, color=TXT, weight="bold")
    lim = all_recruited - 4
    ax.axvline(lim, color=SEC, lw=1, ls=":")
    ax.text(lim - 9, 2.5, f"largest error df for a\nthree-group ANCOVA on all\n{all_recruited} recruited: {lim}",
            ha="right", va="center", fontsize=10, color=SEC)
    ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
    ax.set_xlim(0, 420); ax.set_ylim(-0.5, len(rows) - 0.5)
    ax.set_xlabel("Error degrees of freedom")
    style_axis(ax)
    ax.plot([], [], "o", color=ORANGE, ms=8, label="reported error df")
    ax.plot([], [], "D", color=BLUE, ms=11, label="N − 4 recomputed from the table's group sizes")
    ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=2, frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)

cand_A(os.path.join(CAND, "cand_A_text_panels"))
cand_B(os.path.join(CAND, "cand_B_participants_axis"))
cand_C(os.path.join(CAND, "cand_C_calculation_flow"))
cand_D(os.path.join(CAND, "cand_D_F_statistics"))
cand_E(os.path.join(CAND, "cand_E_error_df_audit"))
# Provisional choice for the write-up until a design is picked: A as fig1, E as fig2.
cand_A(os.path.join(HERE, "fig1"))
cand_E(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)

Abstract calls a MELD–referral odds ratio of 1.13 "lower referral odds"

JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.2567 · PMC13003376

Summary

The study follows 325 patients hospitalized with severe alcohol-associated hepatitis at 5 US transplant centers and models referral for liver transplant evaluation. The abstract states that "higher MELD scores were associated with lower referral odds" and gives, in the same sentence, an odds ratio of 1.13 (95% CI 1.07–1.18; P < .001). An odds ratio above 1, with a confidence interval wholly above 1, means that higher MELD scores go with higher odds of referral. The Results text, Table 2, Table 1 (mean MELD 32.3 in referred vs 27.4 in non-referred patients) and Supplement 1 eTable 3 all show that direction. All the numbers agree with each other. The error is a single direction word in the abstract, which reverses the stated direction of the association for one of the study's primary outcomes.

What the paper reports

Abstract, Results:

In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).

Results, "Factors Associated With Referral for Transplant Evaluation", paragraph 1:

In multivariable logistic regression, adjusting for age, sex, race, ethnicity, educational level, insurance, marital status, living with children, MELD score, ADI, and site of care, a higher MELD score was associated with increased odds of referral (odds ratio [OR], 1.13 [95% CI, 1.07-1.18]; P < .001) (Table 2).

Table 2 (Multivariable Logistic Regression Analyses at Each Step in the Liver Transplant Cascade), referral column (n = 299):

MELD score (≤40)c | 1.13 (1.07-1.18) | <.001

Table 2, footnote c (the same definition appears in Table 1 and in the Methods):

Scores range from 6 to 40, with higher scores indicating severe liver disease.

Results, "Patient Characteristics", paragraph 3, and Table 1:

Compared with those who were not referred, patients who were referred for LT evaluation had significantly higher mean (SD) MELD scores (32.3 [7.7] vs 27.4 [6.9]; P < .001)

Discussion, paragraph 2:

As expected, the MELD score was associated with the transplant cascade.

Supplement 1, eTable 3 (referral model with an insurance-by-site interaction), row "Meld_cap": estimate 0.1432, standard error 0.0284, Wald χ² 25.4024, P < .0001.

All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC13003376 --grep. The eTable 3 values were read from the supplement PDF.

What the paper's own numbers show

Method. In a logistic model, OR = exp(β). An OR above 1 means a positive coefficient: the odds of the outcome (here, referral) rise as the covariate rises. I converted the printed OR and CI bounds to the log scale and took the standard error from the CI width to get a z statistic. I compared the abstract's numbers with the Results text and Table 2. I computed the OR that a "lower odds" statement would need, which is the reciprocal OR per one-point lower MELD. I converted the eTable 3 coefficient to an OR with a Wald CI. As a descriptive check, I compared the Table 1 MELD means by referral status with a Welch t test, and approximated the crude logistic slope with the normal-discriminant formula (difference in means divided by pooled variance). That approximation is unadjusted and assumes normal MELD distributions with equal variances. It is used only to check the sign. All calculations are in recompute.py.txt.

Quantity Location Reported Recomputed
MELD → referral, adjusted OR Abstract, Results 1.13 (1.07–1.18), P < .001, described as "lower referral odds" ln OR = +0.122; ln CI bounds +0.068 and +0.166, both > 0; z = 4.90, p ≈ 1.0 × 10⁻⁶: higher odds
MELD → referral, adjusted OR Results, paragraph 1 1.13 (1.07–1.18), P < .001, "increased odds" Same numbers as the abstract; direction word consistent with them
MELD → referral, adjusted OR Table 2, referral column 1.13 (1.07–1.18), P < .001 Geometric CI midpoint 1.124, consistent with 1.13
Size of the effect derived — +13% odds per MELD point (CI +7% to +18%); 1.13¹⁰ = 3.39 (1.97–5.23) per 10 points
OR that "lower odds" would require derived not reported anywhere 1/1.13 = 0.885 (0.847–0.935), per 1-point lower MELD
Sensitivity referral model Supplement 1, eTable 3, Meld_cap β = 0.1432, SE 0.0284, χ² 25.40, P < .0001 (β/SE)² = 25.42; OR = 1.154 (1.091–1.220): higher odds
Mean MELD, referred vs not referred Table 1; Results, Patient Characteristics 32.3 (7.7) vs 27.4 (6.9), P < .001; SMD 0.68 Difference +4.9 points; Welch t = 5.75 (df 228), p = 2.9 × 10⁻⁸; SMD 0.67
Crude OR per MELD point (approximation) from Table 1 — Pooled SD 7.21; slope 0.0944; OR ≈ 1.099 (> 1)
Referral rate Abstract; Results 120 (36.9%) 120/325 = 36.9%

Every number in the abstract sentence matches the Results and Table 2. The adjusted estimates, the sensitivity model and the descriptive data all point the same way: patients with higher MELD scores were more likely to be referred. Only the word "lower" in the abstract disagrees.

Figure

The abstract and the Results give the same odds ratio with opposite direction words

Figure 1. On the left is the abstract's sentence and on the right the Results text and Table 2 row. Both are short excerpts, and both printed estimates are shown in orange: OR 1.13 (95% CI 1.07–1.18), P < .001. The double arrow marks the conflict: the numbers are identical, but the abstract says "lower" and the Results say "increased". The blue box gives values recomputed from the printed numbers. The log OR and both log CI bounds are positive (z = 4.9), which contradicts "lower" and supports "increased". The Table 1 means (32.3 in referred vs 27.4 in non-referred patients) point the same way.

Odds ratio for referral per MELD point: printed values, recomputed values and the reciprocal that "lower" would require

Figure 2. Odds ratios for referral per 1-point higher MELD score, on a log axis with a thin reference line at OR = 1. Orange: the estimate as printed in the abstract and in the Results and Table 2. It is identical in both places and lies entirely to the right of 1. Blue: values recomputed from the paper's own numbers. These are the eTable 3 sensitivity model, exp(0.1432 ± 1.96 × 0.0284) = 1.154 (1.091–1.220), and the crude normal-discriminant approximation from the Table 1 means (1.099, point estimate only), both above 1. The hollow blue marker is the reciprocal, 1/1.13 = 0.885 (0.847–0.935). This is the estimate a "lower referral odds" statement would require. The paper does not report it.

Why it matters

The abstract is the part of a paper that most readers see, and the part that indexing services, evidence summaries and citing articles most often take statements from. Referral is one of the study's primary outcomes, and MELD is one of its two named exposures. As printed, the abstract sentence tells a reader that sicker patients were less likely to be referred for transplant evaluation. That would be a clinically surprising result, and it is the opposite of what the paper's analysis shows. A reader who knows how to read an odds ratio can see that the sentence contradicts itself. A reader who takes in only the words, or a tool that extracts the claim from the text, would record the reversed direction.

The consequences are limited. No number is wrong, and the significance and size of the estimate are as printed: each additional MELD point goes with about 13% higher adjusted odds of referral. The title, Key Points and abstract Conclusions do not mention the MELD main effect. The Results, Table 2, Table 1, the Discussion and the supplement are all consistent with each other. The paper's headline finding is unaffected. That finding is the MELD × ADI interaction at intermediate MELD scores (P for interaction = .01) and the referral probabilities by ADI band, and neither depends on this sentence. The ADI, wait-listing, transplant and mortality results are also unaffected.

Explanations considered

The evidence fits a single wrong direction word in the abstract's Results paragraph. Nothing in the paper shows how it arose.

Status

As of 2026-10-23:

Sources: publisher full text; PMC13003376; Europe PMC REST full-text XML and supplementary files for PMC13003376; Crossref REST API for doi:10.1001/jamanetworkopen.2026.2567.

Suggested correction

Abstract, Results. Replace "In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001)." with:

In multivariable analysis, higher MELD scores were associated with higher referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).

The numbers stay as printed, and they match the Results text, Table 2 and Supplement 1. No other part of the paper needs to change.

Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13003376 write-up and regenerate fig1/fig2.

Run:  python recompute.py.txt
All inputs are values printed in the article (Abstract, Results, Tables 1-2)
or in Supplement 1 (eTable 3). Nothing is fitted to patient-level data.
"""
import math
import os

import numpy as np
from scipy import stats

HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)

def se_log(lo, hi):
    """SE of log(OR) implied by a 95% CI."""
    return (math.log(hi) - math.log(lo)) / (2 * Z)

# ------------------------------------------------------------------ inputs
OR, LO, HI = 1.13, 1.07, 1.18          # Abstract = Results p0 = Table 2 (referral, MELD <=40)
N_REFMODEL = 299                       # Table 2 referral column header
N_ALL, N_REF = 325, 120                # Abstract / Results
M1, S1, N1 = 32.3, 7.7, 120            # Table 1: with referral
M0, S0, N0 = 27.4, 6.9, 205            # Table 1: without referral
B_E3, SE_E3, CHI_E3 = 0.1432, 0.0284, 25.4024   # Supplement 1 eTable 3, Meld_cap

print("=== 1. Direction implied by the reported OR (Abstract, Results, Table 2) ===")
b = math.log(OR); se = se_log(LO, HI); z = b / se
print(f"  log OR = ln(1.13) = {b:+.3f}")
print(f"  log CI bounds: ln(1.07) = {math.log(LO):+.3f}, ln(1.18) = {math.log(HI):+.3f}  (both > 0: {math.log(LO) > 0})")
print(f"  SE(log OR) from CI = {se:.4f};  z = {z:.2f};  two-sided p = {2*stats.norm.sf(abs(z)):.1e}  (reported P < .001)")
print(f"  geometric CI midpoint = {math.sqrt(LO*HI):.3f}  (reported OR {OR})")
print(f"  percent change in odds per +1 MELD point: {100*(OR-1):.0f}% (CI {100*(LO-1):.0f}% to {100*(HI-1):.0f}%)")
print(f"  OR per +10 MELD points = 1.13^10 = {OR**10:.2f} (CI {LO**10:.2f} to {HI**10:.2f})")

print("\n=== 2. What 'lower referral odds' would require (reciprocal, not reported) ===")
print(f"  OR per 1-point LOWER MELD = 1/1.13 = {1/OR:.3f}  (CI {1/HI:.3f} to {1/LO:.3f})")

print("\n=== 3. Supplement 1 eTable 3 (sensitivity referral model, insurance x site interaction) ===")
print(f"  Meld_cap coefficient {B_E3:+.4f}, SE {SE_E3:.4f}")
print(f"  Wald chi2 = (b/SE)^2 = {(B_E3/SE_E3)**2:.2f}  (printed {CHI_E3})")
lo3, hi3 = math.exp(B_E3 - Z*SE_E3), math.exp(B_E3 + Z*SE_E3)
OR3 = math.exp(B_E3)
print(f"  OR = exp(b) = {OR3:.3f}  (Wald 95% CI {lo3:.3f} to {hi3:.3f})")

print("\n=== 4. Descriptive cross-check, Table 1 MELD by referral status ===")
diff = M1 - M0
t = diff / math.sqrt(S1**2/N1 + S0**2/N0)
dfw = (S1**2/N1 + S0**2/N0)**2 / ((S1**2/N1)**2/(N1-1) + (S0**2/N0)**2/(N0-1))
print(f"  mean difference = {M1} - {M0} = {diff:.1f} MELD points (referred higher)")
print(f"  Welch t = {t:.2f}, df = {dfw:.0f}, p = {2*stats.t.sf(t, dfw):.1e}  (reported P < .001)")
sp2 = ((N1-1)*S1**2 + (N0-1)*S0**2) / (N1+N0-2)
smd = diff / math.sqrt((S1**2 + S0**2)/2)
print(f"  SMD (average-variance) = {smd:.2f}  (Table 1 SMD 0.68)")
beta_crude = diff / sp2
OR_crude = math.exp(beta_crude)
print(f"  pooled SD = {math.sqrt(sp2):.2f}; normal-discriminant crude logistic slope = {beta_crude:.4f}"
      f" -> crude OR per MELD point ~ {OR_crude:.3f}  (approximation; > 1)")

print("\n=== 5. Counts ===")
print(f"  referral rate = {N_REF}/{N_ALL} = {100*N_REF/N_ALL:.1f}%  (reported 36.9%)")
print(f"  referral model n (Table 2 header) = {N_REFMODEL}")

# ------------------------------------------------------------------ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D

BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
                     "xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
                     "svg.fonttype": "none"})

def save(fig, name):
    fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
    fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
    plt.close(fig)

# ---- Figure 1: abstract vs Results/Table 2 text diagram
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 70); ax.axis("off")

def panel(x, title, lines, word, after, numbers):
    ax.add_patch(FancyBboxPatch((x, 30), 38, 34, boxstyle="round,pad=0.4,rounding_size=1.5",
                                fc="white", ec=SEC, lw=1.2))
    ax.text(x + 2, 61.5, title, fontsize=11.5, weight="bold", va="top")
    ax.text(x + 2, 55.5, lines, fontsize=10.5, style="italic", va="top", linespacing=1.35)
    ax.text(x + 2, 44.6, word, fontsize=12, weight="bold", color=TXT, va="center",
            bbox=dict(boxstyle="round,pad=0.25", fc="#fde6dc", ec=ORA, lw=1.5))
    ax.text(x + 2 + 1.45*len(word) + 2.2, 44.6, after, fontsize=10.5, style="italic", va="center")
    ax.text(x + 2, 37.3, numbers, fontsize=11, color=ORA, weight="bold", va="center", linespacing=1.3)
    ax.text(x + 2, 32.4, "as printed in the paper", fontsize=10, color=SEC, va="center")

panel(2, "Abstract, Results",
      "“In multivariable analysis, higher\nMELD scores were associated with",
      "lower", "referral odds”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")
panel(60, "Results text and Table 2",
      "“a higher MELD score was\nassociated with",
      "increased", "odds of referral”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")

ax.add_patch(FancyArrowPatch((41.0, 44.6), (59.5, 44.6), arrowstyle="<|-|>", mutation_scale=13,
                             lw=2, color=TXT, shrinkA=1, shrinkB=1))
ax.text(50.5, 53.5, "Conflict:\nsame OR,\nCI and P;\nopposite\ndirection", ha="center", va="center",
        fontsize=10, linespacing=1.15)

# recomputed box
ax.add_patch(FancyBboxPatch((14, 4), 72, 18, boxstyle="round,pad=0.4,rounding_size=1.5",
                            fc="#e8f0fb", ec=BLU, lw=2))
ax.text(50, 19.6, "Recomputed from the printed OR and CI", ha="center", va="top", fontsize=11,
        weight="bold", color=BLU)
ax.text(50, 14.6,
        f"ln(1.13) = {b:+.3f};  ln(1.07) = {math.log(LO):+.3f},  ln(1.18) = {math.log(HI):+.3f}  (all > 0)\n"
        f"z = {z:.1f}:  each higher MELD point goes with HIGHER referral odds\n"
        f"Table 1 agrees: mean MELD {M1} referred vs {M0} not referred",
        ha="center", va="top", fontsize=10.2, linespacing=1.4)
ax.add_patch(FancyArrowPatch((28, 23.6), (22, 29.6), arrowstyle="-|>", mutation_scale=12,
                             lw=1.5, color=BLU))
ax.add_patch(FancyArrowPatch((72, 23.6), (78, 29.6), arrowstyle="-|>", mutation_scale=12,
                             lw=1.5, color=BLU))
ax.text(23, 26.6, "contradicts “lower”", fontsize=10, color=SEC, va="center", ha="right")
ax.text(77, 26.6, "supports “increased”", fontsize=10, color=SEC, va="center", ha="left")
leg = [Line2D([], [], color=ORA, lw=0, marker="s", ms=9, label="Reported in the paper"),
       Line2D([], [], color=BLU, lw=0, marker="s", ms=9, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.045), ncol=2, frameon=False,
          fontsize=10)
save(fig, "fig1")

# ---- Figure 2: dot-and-interval plot on a log axis
rows = [
    ("Abstract (says “lower”)", OR, LO, HI, ORA, "o", True),
    ("Results and Table 2 (“increased”)", OR, LO, HI, ORA, "o", True),
    ("eTable 3 sensitivity model", OR3, lo3, hi3, BLU, "s", True),
    ("Crude, from Table 1 means", OR_crude, None, None, BLU, "D", True),
    ("1/1.13: what “lower” would need\n(not reported)", 1/OR, 1/HI, 1/LO, BLU, "o", False),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, o, lo, hi, col, mk, filled), y in zip(rows, ys):
    if lo is not None:
        ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
        txt = f"{o:.3f} ({lo:.3f}–{hi:.3f})" if col == BLU else f"{o:.2f} ({lo:.2f}–{hi:.2f})"
    else:
        txt = f"{o:.3f} (point only)"
    ax.plot(o, y, marker=mk, ms=9, color=col, mfc=col if filled else BG, mew=2, lw=0)
    ax.text(o, y + 0.33, txt, va="center", ha="center", fontsize=10, color=col)
ax.axvline(1.0, color=SEC, lw=0.9)
ax.set_xscale("log")
ax.set_xlim(0.80, 1.30)
ticks = [0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
    ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.set_xlabel("Odds ratio for referral per 1-point higher MELD score (log scale)")
ax.text(0.985, ys[0] + 0.8, "← lower odds", ha="right", va="center", fontsize=10, color=SEC)
ax.text(1.015, ys[0] + 0.8, "higher odds →", ha="left", va="center", fontsize=10, color=SEC)
ax.set_ylim(-0.5, ys[0] + 1.1)
leg = [Line2D([], [], color=ORA, lw=2, marker="o", ms=8, label="Reported in the paper"),
       Line2D([], [], color=BLU, lw=2, marker="s", ms=8, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png")

Abstract reports P < 0.0001 for 3/170 vs 3/172 serious adverse events; the table gives P > 0.999

Nat Med (2025) · doi:10.1038/s41591-025-03674-y · PMC12092238

Summary

The abstract of this randomized trial (SUPPRESS-AF) says that procedure-related serious adverse events did not differ between PVI plus low-voltage-area ablation and PVI alone, and gives "1.7% versus 1.8%, P < 0.0001". The paper's own Extended Data Table 4 gives the counts behind those percentages, 3 of 170 and 3 of 172, and prints P > 0.999 for the same comparison with the test the Methods name (two-sided Fisher's exact test). Recomputed from those counts, Fisher's exact test gives P = 1.000. With six events in total, no split between the arms could give a two-sided Fisher P below 0.014. The words "no difference" are correct and the table is correct. The P value in the abstract is not, and a reader of the abstract alone sees a highly significant safety difference that the data do not show.

What the paper reports

Abstract:

There was no difference in the procedure-related serious adverse events between the two groups (1.7% versus 1.8%, P < 0.0001).

Extended Data Table 4, "Safety endpoints". This table is an image in the archived article; the text below was read from the publisher's image file (41591_2025_3674_Tab4_ESM.jpg). Columns: PVI+LVA-ABL n=170 | PVI-alone n=172 | p:

Procedure-related serious adverse events, n (%) | 3 (1.7) | 3 (1.8) | >0.999

Extended Data Table 4, footnote:

p for comparison between groups using the two-sided Fisher's exact test.

Methods, "Statistical analysis":

Safety endpoints were analyzed using Fisher's exact test. All tests were conducted at a two-sided α level of 0.05.

Results, "Safety endpoints":

There was no difference in procedure-related serious adverse events between the two groups. Case presentations of procedure-related serious adverse events are shown in Extended Data Table 6.

Discussion, fourth paragraph:

Serious, procedure-related, adverse event rates were not increased with the addition of LVA ablation.

Results, "Patients and follow-up" (the safety population):

A total of 342 patients were randomly assigned to either the LVA-ABL group (170 patients) or the PVI-alone group (172 patients) and safety outcomes were assessed.

Extended Data Table 6 (an image, read from 41591_2025_3674_Tab6_ESM.jpg) describes six cases: in PVI-alone, one cardiac tamponade, one stroke and one major bleeding; in PVI+LVA-ABL, one esophageal fistula and two major bleedings.

What the paper's own numbers show

Method. I took the counts and denominators from Extended Data Table 4 and checked them against the case list in Extended Data Table 6. I ran a two-sided Fisher's exact test on the 2×2 table [[3, 167], [3, 169]], the test that the Methods and the table footnote name. As robustness checks I also ran one-sided Fisher tests, chi-square tests with and without continuity correction, and Fisher's test with the efficacy-population denominator (171) in place of 172. Because Fisher's test conditions on the total number of events, I listed every possible split of the six events and its P value, to find the smallest P these margins allow. I also found how many events the LVA arm would need, against 3/172, for P < 0.0001. Finally I recomputed the other rows of Extended Data Table 4 with the same test to confirm that the method reproduces the paper's printed values. Every number below is printed by recompute.py.txt.

Quantity Location Reported Recomputed from the paper's numbers
Serious AEs, PVI+LVA-ABL Abstract; Ext. Data Table 4 3 (1.7%) of 170 3/170 = 1.76%
Serious AEs, PVI-alone Abstract; Ext. Data Table 4 3 (1.8%) of 172 3/172 = 1.74% (3/171 = 1.75%)
Case count Ext. Data Table 6 6 cases 3 PVI+LVA-ABL, 3 PVI-alone
P, serious AEs Abstract P < 0.0001 Fisher two-sided P = 1.000
P, serious AEs Ext. Data Table 4 >0.999 Fisher two-sided P = 1.000 (odds ratio 1.012)
P with denominator 171 robustness – 1.000
One-sided Fisher P robustness – 0.652 (greater) / 0.663 (less)
Chi-square P robustness – 1.000 (Yates); 0.988 (uncorrected)
Risk difference robustness – 0.02 percentage points (95% CI −2.76 to 2.80)
Lowest two-sided Fisher P over all splits of 6 events robustness – 0.014 (all six in PVI-alone); 0.030 (all six in PVI+LVA-ABL)
LVA-arm events needed for P < 0.0001 vs 3/172 robustness – 21/170 (12.4%), P = 8.7×10⁻⁵
Bleeding events, 6 vs 2 Ext. Data Table 4 P = 0.173 P = 0.1727
Death from any cause, 2 vs 2 Ext. Data Table 4 >0.999 P = 1.000
Periprocedural AEs, 11 vs 4 Ext. Data Table 4; Discussion P = 0.069; OR 2.91 (0.91–9.31) P = 0.0692; OR 2.91 (Woolf CI 0.91–9.31)
Symptomatic stroke, 4 vs 2 Ext. Data Table 4 2 (1.7%), P = 0.723 2/172 = 1.16%, P = 0.447; with 3/172: 1.74%, P = 0.723

The same procedure reproduces the printed P values for bleeding, death and periprocedural events, and the table's own >0.999 for serious adverse events. Only the abstract's value disagrees.

Two smaller points, which do not bear on the main finding. First, 3/170 = 1.76% and 3/172 = 1.74%, so ordinary rounding would give 1.8% and 1.7%; the printed 1.7% and 1.8% appear in swapped order, in both the abstract and the table. Second, in the symptomatic stroke row the PVI-alone entry "2 (1.7)" is internally inconsistent: 2/172 is 1.2%, and Fisher's test on 4 vs 2 gives P = 0.447, whereas 3/172 gives both the printed 1.7% and the printed P = 0.723. The count may be 3 rather than 2; the paper does not say which entry is correct.

Figure

Abstract and Extended Data Table 4 give different P values for the same counts

Figure 1. The two places the paper reports the comparison of procedure-related serious adverse events. The left panel quotes the abstract and the right panel reproduces the relevant row of Extended Data Table 4: the same percentages, the same counts and a footnote naming the two-sided Fisher exact test. The printed P values (orange) are P < 0.0001 and P > 0.999. The blue box gives the two-sided Fisher exact P recomputed from the table's counts, 3/170 vs 3/172, which is 1.000 and agrees with the table.

All possible splits of six events and their Fisher P values

Figure 2. Given the arm sizes (170 and 172) and six events in total, these are the seven possible splits of the events. Bar heights are the probability of each split if the arms do not differ (hypergeometric distribution), and each bar is labelled with its two-sided Fisher exact P. The reported split, 3 vs 3, is the most likely split and has P = 1.000. The most extreme splits give P = 0.030 and 0.014, so no arrangement of six events in these arms could yield the P < 0.0001 printed in the abstract (orange text). All values are computed in recompute.py.txt.

Why it matters

The abstract is the part of the paper most readers see, and it is what indexing services and summaries reproduce. Read alone, "P < 0.0001" signals a highly significant difference in serious procedural harm between the two ablation strategies, the opposite of what the counts show and of what the same sentence says in words. A reader who notices the clash cannot tell from the abstract which part is wrong.

The effect on the paper's conclusions is small. The Results, Discussion and Extended Data Table 4 all report no difference, the table's P value is correct, and the primary efficacy result (61% vs 50% freedom from AF or AT, log-rank P = 0.127) does not depend on this value. The correct P (about 1.0) leaves the paper's safety reading unchanged: adding LVA ablation did not increase procedure-related serious adverse events in this trial. The trial is small for this endpoint, however (six events in total, risk-difference 95% CI about −2.8 to +2.8 percentage points), and the Discussion itself notes a tendency toward more periprocedural adverse events with LVA ablation (11 vs 4, P = 0.069). That point is separate and is not affected by this error.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

  1. In the abstract, replace "(1.7% versus 1.8%, P < 0.0001)" with the value in Extended Data Table 4, for example "(3/170 (1.8%) versus 3/172 (1.7%), P > 0.999)".
  2. Check the order of the percentages for this row in the abstract and in Extended Data Table 4: 3/170 rounds to 1.8% and 3/172 to 1.7%.
  3. Check the symptomatic stroke row of Extended Data Table 4: the PVI-alone entry "2 (1.7)" with P = 0.723 is consistent with 3 events, not 2. Correct either the count or the percentage and P value.
  4. Optional: in the Discussion, the periprocedural adverse event percentages are labelled "95% CI" ("11 (95% CI = 6.5%) versus 4 (95% CI = 2.3%) patients"); they are proportions, not confidence intervals.
Recomputation script (python recompute.py.txt)
"""Recompute the procedure-related serious adverse event P value of PMC12092238
(Nat Med 2025, doi:10.1038/s41591-025-03674-y, SUPPRESS-AF).

Run:  python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
  fig1.svg/png  (abstract vs Extended Data Table 4, text-panel diagram)
  fig2.svg/png  (all possible splits of the six events and their Fisher P values)
  candidates/cand_A..cand_E.svg/png  (five candidate designs for review; A = fig1, E = fig2)

Inputs are copied from the paper:
  Abstract: "(1.7% versus 1.8%, P < 0.0001)"
  Extended Data Table 4 (image 41591_2025_3674_Tab4_ESM.jpg), safety population
    PVI+LVA-ABL n = 170, PVI-alone n = 172, P by two-sided Fisher's exact test
  Extended Data Table 6: six case descriptions (3 PVI+LVA-ABL, 3 PVI-alone)
  Discussion: OR = 2.91 (95% CI 0.91-9.31) for periprocedural adverse events
"""
import math
import os

import numpy as np
from scipy import stats

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (from the paper)
N_LVA, N_PVI = 170, 172          # Extended Data Table 4 headers (safety population)
N_PVI_EFFICACY = 171             # abstract / efficacy population
SAE = (3, 3)                     # Procedure-related serious AEs, Extended Data Table 4
SAE_PRINTED_PCT = (1.7, 1.8)     # Abstract and Extended Data Table 4
ABSTRACT_P = "P < 0.0001"
TABLE_P = ">0.999"
# Other Extended Data Table 4 rows: (a, b, printed % a, printed % b, printed P)
ROWS = {
    "Bleeding events": (6, 2, 3.5, 1.2, "0.173"),
    "Symptomatic stroke": (4, 2, 2.4, 1.7, "0.723"),
    "Death from any cause": (2, 2, 1.2, 1.2, ">0.999"),
    "Periprocedural adverse events": (11, 4, 6.5, 2.3, "0.069"),
    "Procedure-related serious AEs": (3, 3, 1.7, 1.8, ">0.999"),
}
# Extended Data Table 6 case list
TABLE6 = [("PVI-alone", "Cardiac tamponade"), ("PVI-alone", "Stroke"),
          ("PVI+LVA-ABL", "Esophageal fistula"), ("PVI+LVA-ABL", "Major bleeding"),
          ("PVI-alone", "Major bleeding"), ("PVI+LVA-ABL", "Major bleeding")]

# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
    "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
    "svg.fonttype": "none", "lines.linewidth": 1.5,
})
LW = 1.5  # 1.5 pt = 2 px at 96 px/in
MS = 8    # marker size in points (>= 8 px)


def fisher2(a, n1, b, n2, alt="two-sided"):
    return stats.fisher_exact([[a, n1 - a], [b, n2 - b]], alternative=alt)[1]


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg")
    fig.savefig(path_noext + ".png", dpi=200)
    plt.close(fig)


def cp_ci(k, n, a=0.05):
    lo = stats.beta.ppf(a / 2, k, n - k + 1) if k > 0 else 0.0
    hi = stats.beta.ppf(1 - a / 2, k + 1, n - k) if k < n else 1.0
    return lo, hi


# ---------------------------------------------------------------- 1. the serious-AE comparison
a, b = SAE
print("=== 1. Procedure-related serious AEs, Extended Data Table 4 ===")
print(f"PVI+LVA-ABL {a}/{N_LVA} = {100*a/N_LVA:.2f}%  (printed {SAE_PRINTED_PCT[0]})")
print(f"PVI-alone   {b}/{N_PVI} = {100*b/N_PVI:.2f}%  (printed {SAE_PRINTED_PCT[1]})")
print(f"PVI-alone with efficacy denominator {b}/{N_PVI_EFFICACY} = {100*b/N_PVI_EFFICACY:.2f}%")
t = np.array([[a, N_LVA - a], [b, N_PVI - b]])
or_, p2 = stats.fisher_exact(t)
P_FISHER = p2
print(f"Fisher exact two-sided P = {p2:.4f}  (odds ratio {or_:.3f})   reported: abstract {ABSTRACT_P}, table {TABLE_P}")
print(f"Fisher exact two-sided P with n = 171 = {fisher2(a, N_LVA, b, N_PVI_EFFICACY):.4f}")
P_GT = fisher2(a, N_LVA, b, N_PVI, "greater")
P_LT = fisher2(a, N_LVA, b, N_PVI, "less")
print(f"Fisher one-sided P: greater = {P_GT:.3f}, less = {P_LT:.3f}")
P_YATES = stats.chi2_contingency(t)[1]
P_CHI = stats.chi2_contingency(t, correction=False)[1]
print(f"chi-square P: Yates = {P_YATES:.3f}, uncorrected = {P_CHI:.4f}")
pA, pB = a / N_LVA, b / N_PVI
d = pA - pB
se = math.sqrt(pA * (1 - pA) / N_LVA + pB * (1 - pB) / N_PVI)
print(f"risk difference = {100*d:.3f} pp, Wald 95% CI {100*(d-1.96*se):.2f} to {100*(d+1.96*se):.2f} pp")
for m in (0.03, 0.05):  # hypothetical non-inferiority margins (the paper describes none)
    print(f"hypothetical one-sided non-inferiority test, margin {100*m:.0f} pp: P = {stats.norm.sf((m-d)/se):.1e}")
CI_LVA, CI_PVI = cp_ci(a, N_LVA), cp_ci(b, N_PVI)
print(f"Clopper-Pearson 95% CI: LVA {100*CI_LVA[0]:.2f}-{100*CI_LVA[1]:.2f}%, PVI {100*CI_PVI[0]:.2f}-{100*CI_PVI[1]:.2f}%")
cnt = {"PVI+LVA-ABL": 0, "PVI-alone": 0}
for g, _ in TABLE6:
    cnt[g] += 1
print(f"Extended Data Table 6 case count: {cnt}")

# ---------------------------------------------------------------- 2. every split of the six events
print("\n=== 2. Every possible split of the six events (margins fixed: 170, 172, 6) ===")
TOT = a + b
hg = stats.hypergeom(N_LVA + N_PVI, TOT, N_LVA)   # population, successes, draws
SPLIT = []
for k in range(TOT + 1):
    pk = hg.pmf(k)
    pf = fisher2(k, N_LVA, TOT - k, N_PVI)
    SPLIT.append((k, pk, pf))
    print(f"  {k} vs {TOT-k}: probability under no difference = {pk:.4f}, two-sided Fisher P = {pf:.4f}")
PMIN = min(s[2] for s in SPLIT)
print(f"smallest attainable two-sided Fisher P with six events = {PMIN:.4f}")

# ---------------------------------------------------------------- 3. what would P < 0.0001 require
print("\n=== 3. Events needed in the LVA arm (vs 3/172) for two-sided Fisher P < 0.0001 ===")
CURVE = [(x, fisher2(x, N_LVA, b, N_PVI)) for x in range(0, 31)]
X_NEED = next(x for x, p in CURVE if x > b and p < 1e-4)
print(f"  {X_NEED}/{N_LVA} = {100*X_NEED/N_LVA:.1f}%  (P = {dict(CURVE)[X_NEED]:.2e}); "
      f"{X_NEED-1}/{N_LVA}: P = {dict(CURVE)[X_NEED-1]:.2e}")
X05 = next(x for x, p in CURVE if x > b and p < 0.05)
print(f"  first count with P < 0.05: {X05}/{N_LVA} (P = {dict(CURVE)[X05]:.4f})")

# ---------------------------------------------------------------- 4. other rows of Extended Data Table 4
print("\n=== 4. Other rows of Extended Data Table 4 ===")
for name, (x, y, ra, rb, rp) in ROWS.items():
    print(f"  {name:31s} {x}/{N_LVA} = {100*x/N_LVA:.2f}% (printed {ra}); "
          f"{y}/{N_PVI} = {100*y/N_PVI:.2f}% (printed {rb}); Fisher P = {fisher2(x, N_LVA, y, N_PVI):.4f} (printed {rp})")
print(f"  stroke row if PVI-alone count were 3: 3/172 = {300/172:.2f}%, Fisher P (4 vs 3) = {fisher2(4, N_LVA, 3, N_PVI):.3f}")
o = (11 * (N_PVI - 4)) / ((N_LVA - 11) * 4)
s = math.sqrt(1/11 + 1/(N_LVA-11) + 1/4 + 1/(N_PVI-4))
print(f"  periprocedural AEs OR = {o:.2f}, Woolf 95% CI {math.exp(math.log(o)-1.96*s):.2f}-{math.exp(math.log(o)+1.96*s):.2f} (Discussion: 2.91, 0.91-9.31)")


# ================================================================= figures
def panel(ax, x, y, w, h, title, edge):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
                                fc="white", ec=edge, lw=LW, transform=ax.transAxes))
    ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
            fontweight="bold", va="top", color=INK)


def fig_A(path):
    """Text panels: abstract vs Extended Data Table 4, with recomputed value."""
    fig = plt.figure(figsize=(7.2, 4.5))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    r = fig.canvas.get_renderer()
    # left: abstract
    panel(ax, 0.03, 0.42, 0.44, 0.54, "Abstract", INK2)
    ax.text(0.05, 0.82, "\u201cThere was no difference in the\nprocedure-related serious adverse\n"
            "events between the two groups\n(1.7% versus 1.8%,", transform=ax.transAxes,
            fontsize=10.5, va="top", color=INK, linespacing=1.35)
    tp = ax.text(0.05, 0.575, "P < 0.0001", transform=ax.transAxes, fontsize=13, fontweight="bold",
                 color=ORANGE, va="center")
    bb = tp.get_window_extent(r).transformed(ax.transAxes.inverted())
    ax.text(bb.x1 + 0.003, 0.575, ").\u201d", transform=ax.transAxes, fontsize=10.5, va="center")
    # right: Extended Data Table 4
    panel(ax, 0.53, 0.42, 0.44, 0.54, "Extended Data Table 4", INK2)
    ax.text(0.55, 0.82, "Procedure-related serious\nadverse events, n (%)", transform=ax.transAxes,
            fontsize=10.5, va="top", linespacing=1.35)
    for xx, lab, val in [(0.55, "PVI+LVA-ABL\nn = 170", "3 (1.7)"), (0.705, "PVI-alone\nn = 172", "3 (1.8)")]:
        ax.text(xx, 0.68, lab, transform=ax.transAxes, fontsize=10, color=INK2, va="top")
        ax.text(xx, 0.565, val, transform=ax.transAxes, fontsize=10.5, va="center")
    ax.text(0.855, 0.68, "P", transform=ax.transAxes, fontsize=10, color=INK2, va="top")
    ax.text(0.855, 0.565, ">0.999", transform=ax.transAxes, fontsize=13, fontweight="bold",
            color=ORANGE, va="center")
    ax.text(0.55, 0.465, "footnote: two-sided Fisher\u2019s exact test", transform=ax.transAxes,
            fontsize=10, color=INK2, va="center")
    # connector between the two P values, below the panels
    arr = FancyArrowPatch((0.13, 0.415), (0.90, 0.415), transform=ax.transAxes,
                          connectionstyle="arc3,rad=0.16", arrowstyle="<->", mutation_scale=12,
                          lw=LW, color=INK2)
    ax.add_patch(arr)
    ax.text(0.5, 0.275, "Conflict: same counts and same comparison, but P < 0.0001 vs P > 0.999",
            transform=ax.transAxes, ha="center", fontsize=10.5, color=INK, fontweight="bold")
    # recomputed
    ax.add_patch(FancyBboxPatch((0.08, 0.075), 0.84, 0.15, boxstyle="round,pad=0,rounding_size=0.015",
                                fc="white", ec=BLUE, lw=LW, transform=ax.transAxes))
    ax.text(0.5, 0.185, "Recomputed from the table\u2019s counts", transform=ax.transAxes,
            ha="center", fontsize=10, color=INK2, va="center")
    ax.text(0.5, 0.118, f"Fisher\u2019s exact test, 3/170 vs 3/172: two-sided P = {P_FISHER:.3f}",
            transform=ax.transAxes, ha="center", fontsize=11, color=BLUE, fontweight="bold", va="center")
    ax.text(0.5, 0.025, "Orange: values printed in the paper.  Blue: recomputed from the paper\u2019s counts.",
            transform=ax.transAxes, ha="center", fontsize=10, color=INK2)
    save(fig, path)


def fig_B(path):
    """P values on one log axis."""
    rows = [
        ("Abstract (printed)", 1e-4, ORANGE, "lt", "P < 0.0001"),
        ("Ext. Data Table 4 (printed)", 0.999, ORANGE, "gt", "P > 0.999"),
        ("Fisher, two-sided (stated test)", P_FISHER, BLUE, "", f"{P_FISHER:.3f}"),
        ("Chi-square, uncorrected", P_CHI, BLUE, "", f"{P_CHI:.3f}"),
        ("Fisher, one-sided (each direction)", min(P_GT, P_LT), BLUE, "", f"{P_GT:.3f} / {P_LT:.3f}"),
        ("Lowest P any split of 6 events allows", PMIN, BLUE, "", f"{PMIN:.3f}"),
    ]
    fig, ax = plt.subplots(figsize=(7.2, 3.6))
    ys = np.arange(len(rows))[::-1]
    for y, (lab, v, c, kind, txt) in zip(ys, rows):
        mk = "o" if c == BLUE else "D"
        if lab.startswith("Fisher, one-sided"):
            ax.plot([P_LT, P_GT], [y, y], color=c, lw=LW)
            ax.plot([P_GT, P_LT], [y, y], mk, color=c, ms=MS, mec="white", mew=1)
        else:
            ax.plot(v, y, mk, color=c, ms=MS, mec="white", mew=1)
        if kind == "lt":
            ax.annotate("", xy=(1.6e-6, y), xytext=(1e-4, y),
                        arrowprops=dict(arrowstyle="->", color=c, lw=LW))
        ax.text(1.03, y, txt, color=INK, fontsize=10, ha="left", va="center",
                transform=ax.get_yaxis_transform())
    ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
    ax.text(0.05 * 1.12, len(rows) - 0.45, "α = 0.05", fontsize=10, color=INK2)
    ax.set_xscale("log"); ax.set_xlim(1e-6, 2.2); ax.set_ylim(-0.6, len(rows) - 0.2)
    ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
    ax.set_xlabel("P value (log scale)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
    h2, = ax.plot([], [], "o", color=BLUE, ms=MS, label="recomputed from the paper\u2019s counts")
    fig.legend(handles=[h1, h2], loc="upper center", ncol=2, frameon=False, fontsize=10)
    fig.tight_layout(rect=(0, 0, 0.87, 0.92))
    save(fig, path)


def fig_C(path):
    """Fisher P as a function of the LVA-arm count."""
    xs = np.array([c[0] for c in CURVE]); ps = np.array([c[1] for c in CURVE])
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    ax.plot(xs, ps, color=BLUE, lw=LW, marker="o", ms=5, mec="white", mew=0.6,
            label="two-sided Fisher P, k/170 vs 3/172 (recomputed)")
    ax.axhline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
    ax.text(30.3, 0.05, "0.05", va="center", fontsize=10, color=INK2)
    ax.axhline(1e-4, color=ORANGE, lw=LW, ls=(0, (5, 2)), label="abstract: P < 0.0001 (printed)")
    ax.text(30.3, 1e-4, "0.0001", va="center", fontsize=10, color=INK2)
    ax.plot(3, P_FISHER, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
    ax.annotate(f"reported counts 3 vs 3:\nP = {P_FISHER:.3f}", (3, P_FISHER), xytext=(0.6, 2e-3),
                fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    pn = dict(CURVE)[X_NEED]
    ax.plot(X_NEED, pn, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
    ax.annotate(f"{X_NEED}/170 ({100*X_NEED/N_LVA:.1f}%) would be\nneeded for P < 0.0001",
                (X_NEED, pn), xytext=(22.5, 3e-3), fontsize=10,
                arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
    ax.set_yscale("log"); ax.set_ylim(1e-8, 2.5); ax.set_xlim(-0.5, 30)
    ax.set_xlabel("Events in the PVI+LVA-ABL arm (of 170), PVI-alone fixed at 3 of 172")
    ax.set_ylabel("Two-sided P (log scale)")
    ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right"):
        ax.spines[s].set_visible(False)
    ax.legend(loc="lower left", frameon=False, fontsize=10)
    fig.tight_layout()
    save(fig, path)


def fig_D(path):
    """Per-arm event rates with exact 95% CIs."""
    fig, ax = plt.subplots(figsize=(7.2, 2.9))
    arms = [("PVI+LVA-ABL (3/170)", a / N_LVA, CI_LVA, SAE_PRINTED_PCT[0]),
            ("PVI-alone (3/172)", b / N_PVI, CI_PVI, SAE_PRINTED_PCT[1])]
    for y, (lab, p, ci, pr) in zip([1, 0], arms):
        ax.plot([100 * ci[0], 100 * ci[1]], [y, y], color=BLUE, lw=LW)
        ax.plot(100 * p, y, "o", color=BLUE, ms=MS, mec="white", mew=1)
        ax.plot(pr, y + 0.22, "D", color=ORANGE, ms=MS, mec="white", mew=1)
        ax.text(100 * ci[1] + 0.15, y, f"{100*p:.2f}% (95% CI {100*ci[0]:.2f}–{100*ci[1]:.2f})",
                va="center", fontsize=10)
        ax.text(pr + 0.15, y + 0.22, f"printed {pr}%", va="center", fontsize=10, color=INK2)
    ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in arms])
    ax.set_xlim(0, 9.5); ax.set_ylim(-1.0, 1.6)
    ax.set_xlabel("Procedure-related serious adverse events, % of patients")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
    h2, = ax.plot([], [], "o-", color=BLUE, ms=MS, lw=LW, label="recomputed, exact 95% CI")
    ax.legend(handles=[h1, h2], loc="lower right", ncol=2, frameon=False, fontsize=10)
    ax.set_title(f"Abstract prints P < 0.0001; these counts give Fisher P = {P_FISHER:.3f}",
                 fontsize=11, loc="left")
    fig.tight_layout()
    save(fig, path)


def fig_E(path):
    """All seven possible splits of six events, with their two-sided Fisher P."""
    ks = [s[0] for s in SPLIT]; pk = [s[1] for s in SPLIT]; pf = [s[2] for s in SPLIT]
    fig, ax = plt.subplots(figsize=(7.2, 3.8))
    cols = [BLUE if k == a else "#bcd3ee" for k in ks]
    ax.bar(ks, pk, width=0.72, color=cols, edgecolor=BG, lw=2)
    for k, p, f in zip(ks, pk, pf):
        ax.text(k, p + 0.008, f"P = {f:.3f}", ha="center", fontsize=10, color=INK)
        ax.text(k, -0.045, f"{k} vs {TOT-k}", ha="center", fontsize=10, color=INK2)
    ax.text(a, pk[a] + 0.035, "reported split", ha="center", fontsize=10, color=BLUE, fontweight="bold")
    ax.text(-0.5, 0.35, "Abstract: P < 0.0001 (printed)", ha="left", va="top", fontsize=10.5,
            color=ORANGE, fontweight="bold")
    ax.text(-0.5, 0.315, f"No split of six events can give a\ntwo-sided Fisher P below {PMIN:.3f}",
            ha="left", va="top", fontsize=10, color=INK)
    ax.set_xticks([]); ax.set_ylim(-0.07, 0.37); ax.set_yticks(np.arange(0, 0.36, 0.05)); ax.set_xlim(-0.6, 6.6)
    ax.axhline(0, color=INK2, lw=0.8)
    ax.set_ylabel("Probability if the arms do not differ")
    ax.set_xlabel("Split of the six events (PVI+LVA-ABL vs PVI-alone); bar labels: two-sided Fisher P", labelpad=4)
    ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
    ax.spines["left"].set_bounds(0, 0.35)
    for s in ("top", "right", "bottom"):
        ax.spines[s].set_visible(False)
    fig.tight_layout()
    save(fig, path)


for name, fn in [("cand_A", fig_A), ("cand_B", fig_B), ("cand_C", fig_C), ("cand_D", fig_D), ("cand_E", fig_E)]:
    fn(os.path.join(CAND, name))
fig_A(os.path.join(HERE, "fig1"))
fig_E(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/cand_A..E (.svg, .png) in", HERE)

Abstract reports 389 animals; the Methods list 116, and 389 sums per-context subject counts

eLife (2023) · doi:10.7554/eLife.87008 · PMC10547472

Summary

The abstract says facial behavior was coded "in 389 animals". The Methods list the study subjects by species and sex, and they add up to 116 adults (32 rhesus, 46 Barbary, 38 crested macaques). Table 2 agrees with the Methods: no per-context subject count is larger than the Methods headcount for its species. The figure 389 is exactly the sum of Table 2's "N subjects" column over all twelve species-by-context rows, so it counts each animal once for every context it was coded in. The study's analyses do not use this number and its conclusions are unaffected, but anyone who reads only the abstract will take the sample to be 3.35 times as large as it is.

What the paper reports

Abstract:

We coded facial behavior in >3000 social interactions across three social contexts (aggressive, submissive, affiliative) in 389 animals, using the Facial Action Coding System for macaques (MaqFACS).

Methods, "Study subjects and data collection", first paragraph:

Behavioral data and video recordings were collected on one adult male and 31 adult female rhesus macaques (M. mulatta), on 18 adult male and 28 adult female Barbary macaques (M. sylvanus), and 17 adult male and 21 adult female crested macaques (M. nigra).

Methods, "Facial behavior and social context coding":

We coded facial behavior of adult individuals but included their interactions with any other group member regardless of age or sex.

Table 2 shows the number of social interactions per species and context from which FACS codes were made.

Table 2, "Total number of social interactions per species and social context that were MaqFACS coded":

Species Context N interactions N subjects Duration (s)
Rhesus Affiliative 193 29 1197
Rhesus Aggressive 413 32 2050
Rhesus Submissive 318 31 1262
Rhesus Unclear 121 30 802
Barbary Affiliative 683 43 4897
Barbary Aggressive 585 44 2128
Barbary Submissive 529 34 1890
Barbary Unclear 603 45 3500
Crested Affiliative 241 35 1918
Crested Aggressive 62 23 284
Crested Submissive 25 18 115
Crested Unclear 107 25 684

Discussion, on the comparison that limits inference:

Finally, effectively, our comparison is limited to three species which is a small sample.

What the paper's own numbers show

Method. I added the Methods counts by species and sex to get the number of distinct adults. For each species I summed the four Table 2 "N subjects" cells and found the largest of them. I then compared the sums with the abstract's 389, and the largest cell with the Methods headcount. As a separate check I summed the Table 2 interaction counts against the abstract's ">3000". Last, I counted animals in the authors' public data file (the file the paper's data availability statement points to): distinct animals per species, and the number of contexts each animal was coded in. Every number below is printed by recompute.py.txt. The data-file rows need the file itself: run python recompute.py.txt --data <path or URL>, where the URL is maqfacs.txt at the root of the GitHub repository the paper cites (Methods, "Specificity bias correction"), in raw form https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt with <owner> taken from that citation. Its sha256 is printed and quoted under Status.

Quantity Location Reported Recomputed from the paper's numbers
Animals in the study Abstract 389 animals 116 distinct adults (Methods)
Rhesus adults Methods 1 male + 31 female 32
Barbary adults Methods 18 male + 28 female 46
Crested adults Methods 17 male + 21 female 38
Total distinct adults Methods not stated 32 + 46 + 38 = 116
Rhesus, sum of N subjects over four contexts Table 2 29, 32, 31, 30 122 (3.81 × the 32 adults)
Barbary, sum of N subjects over four contexts Table 2 43, 44, 34, 45 166 (3.61 × the 46 adults)
Crested, sum of N subjects over four contexts Table 2 35, 23, 18, 25 101 (2.66 × the 38 adults)
Sum of all 12 N-subjects cells Table 2 — 122 + 166 + 101 = 389, identical to the abstract
Same sum without the Unclear rows Table 2 — 289, so 389 is not a three-context total
Largest per-context N subjects, by species Table 2 32 / 45 / 35 at most the Methods headcount (32 / 46 / 38), as it must be
Abstract count relative to headcount Abstract vs Methods 389 389 / 116 = 3.35; 273 more animals than were studied
Interactions, three named contexts Abstract vs Table 2 >3000 924 + 1797 + 328 = 3049 (3880 with Unclear); consistent
Distinct animals in the authors' data file Data file maqfacs.txt — 32 rhesus, 46 Barbary, 40 crested = 118. Crested: 17 male, 22 female, 1 with no sex recorded (Methods: 17 male, 21 female). Rhesus and Barbary match the Methods by sex
Per-context distinct subjects in the data file Data file — Reproduces all 12 Table 2 N-subjects cells and durations (to 0.5 s) once the file's "agonistic" label is merged into "unclear"; they sum to 389
Contexts per animal in the data file Data file — Rhesus: 27 animals in 4 contexts, 4 in 3, 1 in 2. Barbary: 29, 16, 1. Crested: 10 in 4, 13 in 3, 5 in 2, 12 in 1. Sum over animals of contexts = 389

The arithmetic leaves one reading. The abstract's 389 is the total of the "N subjects" column, a column that counts each animal once per context. Taken as a headcount, the paper's own Methods and Table 2 give 116 adults (118 animals in the published data file).

Figure

Abstract and Methods counts side by side, with Table 2's per-context subject counts

Figure 1. On the left is the abstract's sentence giving the sample as 389 animals (orange: the value the paper reports). On the right is the Methods list of adults by species and sex, which sums to 116 (blue: recomputed from the paper's own counts). The lower panel gives Table 2's four per-context "N subjects" values for each species. Each value is no larger than the Methods headcount for its species, which is consistent with each one counting distinct animals, and together they add up to 122 + 166 + 101 = 389. The authors' data file settles the reading: it reproduces all 12 cells, and the number of contexts per animal sums to 389 over 118 animals. The abstract's figure is therefore the number of subject-context pairs, not the number of animals. All values are taken from the paper or printed by recompute.py.txt.

Per-species dot plot of Table 2 context counts, Methods headcount and their sum

Figure 2. For each species, the open circles are the four Table 2 per-context subject counts, stacked vertically (affiliative, aggressive, submissive, unclear, top to bottom) so that close values such as 29 to 32 do not hide one another. The blue vertical line is the Methods headcount (32, 46, 38) and the orange circle is the sum of the four context counts (122, 166, 101; together 389). Every context count lies at or below the headcount, which is consistent with each one counting distinct animals (the data-file check confirms it). Their sum is 2.7 to 3.8 times the headcount (3.81 rhesus, 3.61 Barbary, 2.66 crested) because most animals appear in more than one context. The axis starts at zero.

Why it matters

The error is in the abstract, the part of a paper that is read, indexed and copied most widely. A reader, reviewer or meta-analyst who takes the sample size from the abstract or from a database built from it will record N = 389. The correct figure is about 116 (118 in the published data file), so the study would look 3.35 times as large as it is. That matters most when studies are weighed or compared by number of individuals, and in particular for the rhesus sample: 32 animals from one breeding group, 31 of them female, a limitation the Methods acknowledge.

The inferential results are not affected. Entropy, context specificity and the random-forest classifier are computed from 500 ms time blocks within interactions, and the species comparison rests on three species, as the Discussion states. The headcount enters none of these calculations, so every reported estimate and its direction stand. The abstract's other count, ">3000 social interactions", is consistent with Table 2 (3049 in the three named contexts).

Explanations considered

None of these explains the number. Adding up the per-context subject counts reproduces it exactly.

Status

As of 2026-10-23:

Suggested correction

Abstract, change "in 389 animals" to "in 116 adult animals" (or the authors' deduplicated count, 118 if the data file's 40 crested individuals are the correct figure). If a per-context total is wanted, it could be stated as such, for example: "116 adults (389 subject-context combinations)". Reconciling the crested count, 38 in the Methods and 40 in the data file, would make the Methods and the data file agree.

Recomputation script (python recompute.py.txt)
"""Recompute the sample size stated in the abstract of PMC10547472
(eLife 2023;12:RP87008, doi:10.7554/eLife.87008).

Run:  python recompute.py.txt                          (paper's own numbers + figures)
      python recompute.py.txt --data <path or URL>     (also check the authors' data file)
The data file is maqfacs.txt at the root of the GitHub repository whose URL the paper
gives in Methods, "Specificity bias correction" (https://github.com/<owner>/macaque-facial-complexity);
its raw URL is https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt.
Needs numpy and matplotlib; pandas only for the optional data-file check.
Prints every recomputed number used in writeup.md and writes, next to this file:
  fig1.svg/png, fig2.svg/png               (figures used in writeup.md)
  candidates/cand_A..cand_E.svg/png        (five alternative designs for fig1)

Inputs are copied from the paper as archived in PubMed Central (identical in the
eLife version of record):
  Abstract: ">3000 social interactions ... in 389 animals"
  Methods, "Study subjects and data collection": adults per species and sex
  Table 2: N interactions, N subjects and duration per species x context
Optional: the authors' data file maqfacs.txt (file at the root of the public GitHub
repository given in the paper's data availability statement; sha256 printed below),
used only to count distinct animals and contexts per animal. Candidate E needs it.
"""
import os
import sys
import urllib.request

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle  # noqa: E402

HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)

# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_ANIMALS = 389
ABSTRACT_INTERACTIONS_FLOOR = 3000                      # ">3000"
SPECIES = ("Rhesus", "Barbary", "Crested")
METHODS = {"Rhesus": (1, 31), "Barbary": (18, 28), "Crested": (17, 21)}  # (male, female) adults
CONTEXTS = ("Affiliative", "Aggressive", "Submissive", "Unclear")
T2 = {  # Table 2: (N interactions, N subjects, duration s)
    "Rhesus": {"Affiliative": (193, 29, 1197), "Aggressive": (413, 32, 2050),
               "Submissive": (318, 31, 1262), "Unclear": (121, 30, 802)},
    "Barbary": {"Affiliative": (683, 43, 4897), "Aggressive": (585, 44, 2128),
                "Submissive": (529, 34, 1890), "Unclear": (603, 45, 3500)},
    "Crested": {"Affiliative": (241, 35, 1918), "Aggressive": (62, 23, 284),
                "Submissive": (25, 18, 115), "Unclear": (107, 25, 684)},
}

# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, NEUTRAL = "#eb6834", "#2a78d6", "#b9b7b1"
plt.rcParams.update({
    "font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
    "axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
    "ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
    "savefig.facecolor": BG, "svg.fonttype": "path", "lines.linewidth": 2,
})
W = 7.2


def save(fig, path_noext):
    fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
    fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
    plt.close(fig)


# ---------------------------------------------------------------- recomputation
print("== Methods: adults per species ==")
adults = {}
for sp in SPECIES:
    m, f = METHODS[sp]
    adults[sp] = m + f
    print(f"{sp:8s} {m} male + {f} female = {adults[sp]}")
total_adults = sum(adults.values())
print(f"Total distinct adults (Methods): {total_adults}")

print("\n== Table 2: N subjects per context ==")
subj_sum, subj_max = {}, {}
for sp in SPECIES:
    vals = [T2[sp][c][1] for c in CONTEXTS]
    subj_sum[sp] = sum(vals)
    subj_max[sp] = max(vals)
    print(f"{sp:8s} {' + '.join(map(str, vals))} = {subj_sum[sp]};  max {subj_max[sp]}"
          f" <= Methods {adults[sp]}: {subj_max[sp] <= adults[sp]};"
          f"  sum / adults = {subj_sum[sp] / adults[sp]:.2f}")
total_subj = sum(subj_sum.values())
print(f"Sum over all 12 species x context cells: "
      f"{' + '.join(str(subj_sum[s]) for s in SPECIES)} = {total_subj}")
print(f"Equals the abstract's {ABSTRACT_ANIMALS}: {total_subj == ABSTRACT_ANIMALS}")
no_unclear = sum(T2[s][c][1] for s in SPECIES for c in CONTEXTS if c != "Unclear")
print(f"Sum excluding the Unclear rows: {no_unclear}")
print(f"Ratio {ABSTRACT_ANIMALS} / {total_adults} = {ABSTRACT_ANIMALS / total_adults:.2f}")
print(f"Overstatement: {ABSTRACT_ANIMALS - total_adults} animals")

print("\n== Table 2: interactions ==")
ni_all = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS)
ni_3 = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS if c != "Unclear")
for sp in SPECIES:
    print(f"{sp:8s} three named contexts {sum(T2[sp][c][0] for c in CONTEXTS[:3])};"
          f" all four {sum(T2[sp][c][0] for c in CONTEXTS)}")
print(f"Three named contexts: {ni_3} (> {ABSTRACT_INTERACTIONS_FLOOR}: "
      f"{ni_3 > ABSTRACT_INTERACTIONS_FLOOR}); all four contexts: {ni_all}")

# ---------------------------------------------------------------- optional: authors' data file
data = None
DATA_SRC = None
if "--data" in sys.argv:
    i = sys.argv.index("--data")
    if i + 1 < len(sys.argv) and not sys.argv[i + 1].startswith("--"):
        DATA_SRC = sys.argv[i + 1]
    else:
        print("\n[--data needs an argument: a path or URL to maqfacs.txt]")
if DATA_SRC is None:
    print("\n[data-file check not run: pass --data <path or URL to maqfacs.txt>]")
else:
    try:
        import hashlib
        import io

        import pandas as pd
        if DATA_SRC.startswith("http"):
            raw = urllib.request.urlopen(DATA_SRC, timeout=60).read()
        else:
            with open(DATA_SRC, "rb") as fh:
                raw = fh.read()
        print("\n== Authors' data file maqfacs.txt ==")
        print(f"sha256 {hashlib.sha256(raw).hexdigest()}  ({len(raw)} bytes)")
        df = pd.read_csv(io.BytesIO(raw), sep="\t")
        print(f"rows (500 ms blocks): {len(df)}")
        print("context labels:", {k: int(v) for k, v in df["context"].value_counts().items()})
        # Table 2's 'Unclear' row = the file's 'unclear' + 'agonistic' labels
        df["ctx"] = df["context"].replace({"agonistic": "unclear"})
        df["species"] = df["species"].str.capitalize()
        ids = df.drop_duplicates(["species", "subject"])
        n_animals = ids.groupby("species").size()
        sex = ids.groupby(["species", "sex"], dropna=False).size()
        print("distinct animals per species:", {k: int(v) for k, v in n_animals.items()}, "total", int(n_animals.sum()))
        print("by sex (NaN = sex missing):")
        print(sex.to_string())
        g = df.groupby(["species", "ctx"]).agg(nsub=("subject", "nunique"), blocks=("block", "size"))
        g["dur_s"] = g["blocks"] / 2
        print("per species x context: distinct subjects, duration (s) = blocks / 2")
        ok = True
        for sp in SPECIES:
            for c in CONTEXTS:
                r = g.loc[(sp, c.lower())]
                t = T2[sp][c]
                match = int(r.nsub) == t[1] and abs(r.dur_s - t[2]) <= 0.5
                ok &= match
                print(f"  {sp:8s} {c:12s} N subjects {int(r.nsub):3d} (Table 2 {t[1]:3d})"
                      f"  duration {r.dur_s:7.1f} (Table 2 {t[2]:5d})  match: {match}")
        print(f"All 12 N-subjects cells and durations reproduced (to 0.5 s): {ok}")
        print(f"Sum of per-context distinct subjects in the data: {int(g.nsub.sum())}")
        k = df.groupby(["species", "subject"])["ctx"].nunique()
        dist = k.groupby(level=0).value_counts().unstack(fill_value=0)
        print("animals by number of contexts they were coded in:")
        print(dist.to_string())
        print(f"sum over animals of contexts = {int(k.sum())}; distinct animals = {len(k)}")
        data = {sp: sorted(k.loc[sp].tolist(), reverse=True) for sp in SPECIES}
    except Exception as e:  # network or pandas missing: figures from the paper still build
        print(f"\n[data-file check skipped: {e!r}]")


# ---------------------------------------------------------------- figure helpers
def text_panel(ax, x, y, w, h, title, body, edge, title_color=INK):
    ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
                                fc="white", ec=edge, lw=2, transform=ax.transAxes))
    ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, ha="left", va="top",
            fontsize=10.5, color=title_color, fontweight="bold")
    ax.text(x + 0.02, y + h - 0.15, body, transform=ax.transAxes, ha="left", va="top",
            fontsize=10, color=INK, linespacing=1.45)


# ---- Candidate A: two text panels joined by an annotated connector (Abstract vs Methods/Table 2)
def cand_A(path):
    fig = plt.figure(figsize=(W, 4.4))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
    text_panel(ax, 0.01, 0.40, 0.38, 0.58, "Abstract  (reported)",
               "“We coded facial behavior in\n>3000 social interactions across\n"
               "three social contexts (aggressive,\nsubmissive, affiliative) in\n"
               "389 animals”", ORANGE)
    rows = "\n".join(f"{sp:<8s} {METHODS[sp][0]:>2d} M + {METHODS[sp][1]:>2d} F = {adults[sp]:>3d}"
                     for sp in SPECIES)
    text_panel(ax, 0.55, 0.40, 0.44, 0.58, "Methods, Study subjects",
               rows + f"\n{'':23s}───\n{'Adults':<21s}= {total_adults:>3d}",
               BLUE)
    ax.texts[-1].set_family("DejaVu Sans Mono")
    ax.annotate("", xy=(0.55, 0.67), xytext=(0.39, 0.67), xycoords="axes fraction",
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
    ax.text(0.47, 0.76, "conflict", ha="center", va="bottom", fontsize=10.5, color=INK,
            fontweight="bold", transform=ax.transAxes)
    ax.text(0.47, 0.58, f"389 vs {total_adults}\n= {ABSTRACT_ANIMALS / total_adults:.2f}×",
            ha="center", va="top", fontsize=10, color=INK2, transform=ax.transAxes)
    t2 = "\n".join(f"{sp:<8s} {' + '.join(f'{T2[sp][c][1]:2d}' for c in CONTEXTS)} = {subj_sum[sp]}"
                    for sp in SPECIES)
    ax.add_patch(FancyBboxPatch((0.01, 0.01), 0.98, 0.33, boxstyle="round,pad=0,rounding_size=0.012",
                                fc="white", ec=NEUTRAL, lw=2, transform=ax.transAxes))
    ax.text(0.03, 0.31, "Table 2, ‘N subjects’ per context\n(affil. + aggr. + subm. + unclear)",
            fontsize=10.5, fontweight="bold", va="top", transform=ax.transAxes, linespacing=1.35)
    ax.text(0.03, 0.17, t2, fontsize=10, va="top", transform=ax.transAxes, family="DejaVu Sans Mono",
            linespacing=1.35)
    ax.text(0.50, 0.29, f"122 + 166 + 101 = {total_subj}, the abstract's figure.\n"
            "Each cell is ≤ the Methods headcount for its\nspecies; the authors' data file confirms "
            "that the\nsum counts each animal once per context\nit was coded in (up to four times).",
            fontsize=10, color=INK2, va="top", transform=ax.transAxes, linespacing=1.45)
    save(fig, path)


# ---- Candidate B: two stacked bars on one count axis, segmented by species
def cand_B(path):
    fig, ax = plt.subplots(figsize=(W, 2.9))
    bars = [("Abstract: “389 animals”\n(= sum of Table 2 N subjects)", subj_sum, ORANGE),
            ("Methods: adults enrolled\n(distinct animals; same order)", adults, BLUE)]
    for i, (lab, vals, col) in enumerate(bars):
        y, left = 1 - i, 0
        for sp in SPECIES:
            v = vals[sp]
            ax.barh(y, v - 2, left=left + 1, height=0.52, color=col, edgecolor="none")
            ax.text(left + v / 2, y, f"{v}", ha="center", va="center", fontsize=10.5,
                    color="white", fontweight="bold")
            if i == 0:
                ax.text(left + v / 2, y - 0.34, sp, ha="center", va="top", fontsize=10, color=INK2)
            left += v
        ax.text(left + 6, y, f"{left}", ha="left", va="center", fontsize=11, fontweight="bold")
    ax.set_yticks([1, 0], [b[0] for b in bars], fontsize=10)
    ax.set_xlim(0, 420); ax.set_ylim(-0.75, 1.45)
    ax.set_xlabel("Count (starts at zero)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.set_title(f"The abstract's count is {ABSTRACT_ANIMALS / total_adults:.2f}× the number of "
                 "animals listed in the Methods", fontsize=11, loc="left", color=INK)
    save(fig, path)


# ---- Candidate C: per-species dot plot, contexts vs headcount vs sum
# Context circles are stacked vertically (affiliative, aggressive, submissive, unclear,
# top to bottom) so that close values such as 29-32 or 43-45 do not hide one another.
OFFS = (0.33, 0.11, -0.11, -0.33)


def cand_C(path):
    fig, ax = plt.subplots(figsize=(W, 3.6))
    for i, sp in enumerate(SPECIES):
        y = 2 - i
        vals = [T2[sp][c][1] for c in CONTEXTS]
        ax.plot([0, subj_sum[sp]], [y, y], color=GRID, lw=2, zorder=1)
        ax.scatter(vals, [y + o for o in OFFS], s=60, facecolor="white", edgecolor=INK2, lw=1.5,
                   zorder=3, label="Table 2 N subjects, one circle per context\n"
                   "(stacked: affiliative, aggressive, submissive, unclear)" if i == 0 else None)
        ax.plot([adults[sp]] * 2, [y - 0.44, y + 0.44], color=BLUE, lw=2.5, zorder=4,
                solid_capstyle="butt",
                label="Methods: adults (distinct animals)" if i == 0 else None)
        ax.scatter([subj_sum[sp]], [y], s=110, marker="o", color=ORANGE, zorder=4, ec=BG, lw=1.5,
                   label="Sum of the four context counts (adds to the abstract's 389)" if i == 0 else None)
        ax.text(adults[sp] + 2, y + 0.30, f"{adults[sp]}", ha="left", va="center", fontsize=10,
                color=BLUE, fontweight="bold")
        ax.text(subj_sum[sp], y + 0.22, f"{subj_sum[sp]}", ha="center", va="bottom", fontsize=10,
                color=INK)
    ax.set_yticks([2, 1, 0], SPECIES)
    ax.set_xlim(0, 180); ax.set_ylim(-0.55, 2.6)
    ax.set_xlabel("Number of animals (starts at zero)")
    ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
    for s in ("top", "right", "left"):
        ax.spines[s].set_visible(False)
    ax.tick_params(axis="y", length=0)
    ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=1, frameon=False, fontsize=10)
    ax.set_title("Every context count fits under the headcount; their sum does not",
                 fontsize=11, loc="left")
    save(fig, path)


# ---- Candidate D: calculation-flow diagram, two parallel chains
def cand_D(path):
    fig = plt.figure(figsize=(W, 3.6))
    ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 100); ax.set_ylim(1, 60)

    def box(x, y, w, h, txt, ec, bold=False, fs=10):
        ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.8",
                                    fc="white", ec=ec, lw=2))
        ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs,
                fontweight="bold" if bold else "normal")

    def arrow(x0, y0, x1, y1):
        ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
                    arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.6, mutation_scale=12))

    ax.text(1, 57, "Table 2: N subjects per species × context (12 cells)", fontsize=10.5,
            fontweight="bold", va="top")
    ax.text(1, 27, "Methods: adults per species and sex", fontsize=10.5, fontweight="bold", va="top")
    for i, sp in enumerate(SPECIES):
        y = 44 - i * 7
        box(1, y, 40, 5.5, f"{sp}: " + " + ".join(str(T2[sp][c][1]) for c in CONTEXTS), NEUTRAL)
        arrow(41.5, y + 2.75, 49.5, y + 2.75)
        box(50, y, 12, 5.5, str(subj_sum[sp]), NEUTRAL)
        arrow(62.5, y + 2.75, 72.5, 39.75)
        y2 = 15 - i * 7
        box(1, y2, 40, 5.5, f"{sp}: {METHODS[sp][0]} M + {METHODS[sp][1]} F", NEUTRAL)
        arrow(41.5, y2 + 2.75, 49.5, y2 + 2.75)
        box(50, y2, 12, 5.5, str(adults[sp]), NEUTRAL)
        arrow(62.5, y2 + 2.75, 72.5, 10.75)
    box(73, 36, 26, 7.5, f"{total_subj}\nsubject-context pairs", ORANGE, bold=True)
    box(73, 7, 26, 7.5, f"{total_adults}\ndistinct adults", BLUE, bold=True)
    ax.text(86, 47, "Abstract: “in 389 animals” (reported)", ha="center", fontsize=10,
            color=INK)
    ax.text(86, 3.2, "Methods headcount (recomputed)", ha="center", fontsize=10, color=INK2)
    ax.annotate("", xy=(86, 15.2), xytext=(86, 35.3),
                arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.6))
    ax.text(87.5, 25.2, f"{ABSTRACT_ANIMALS / total_adults:.2f}×", fontsize=10.5,
            fontweight="bold", va="center")
    save(fig, path)


# ---- Candidate E: unit chart from the authors' data file, one column per animal
def cand_E(path):
    if data is None:
        print("[candidate E skipped: needs the authors' data file]")
        return False
    fig = plt.figure(figsize=(W, 3.3))
    ax = fig.add_axes([0.0, 0.14, 1.0, 0.68])
    cs, gap = 1.0, 0.18
    for i, sp in enumerate(SPECIES):
        y0 = (2 - i) * 6.2
        for j, k in enumerate(data[sp]):
            for r in range(k):
                ax.add_patch(Rectangle((j * cs + gap / 2, y0 + r * cs + gap / 2), cs - gap, cs - gap,
                                       fc=ORANGE, ec="none"))
        n = len(data[sp])
        ax.plot([0, n * cs], [y0 - 0.35, y0 - 0.35], color=BLUE, lw=2.5)
        ax.text(-1, y0 + 2, sp, ha="right", va="center", fontsize=10.5)
        ax.text(n * cs + 1, y0 + 2.6, f"{sum(data[sp])} squares", ha="left", va="center",
                fontsize=10, color=INK)
        ax.text(n * cs + 1, y0 + 1.0, f"{n} animals", ha="left", va="center", fontsize=10,
                color=INK2)
    ax.set_xlim(-10, 62); ax.set_ylim(-1.0, 16.6); ax.axis("off")
    fig.text(0.02, 0.97, "Authors' data file: each column is one animal (blue rule spans the animals);\n"
             "each orange square is one context that animal was coded in (1–4).", fontsize=10,
             va="top", linespacing=1.4)
    tot_sq = sum(sum(v) for v in data.values())
    tot_an = sum(len(v) for v in data.values())
    fig.text(0.02, 0.03, f"Total: {tot_an} animals; {tot_sq} squares, which is the abstract's "
             "“389 animals”", fontsize=10.5, fontweight="bold", va="bottom")
    save(fig, path)
    return True


builders = {"A": cand_A, "B": cand_B, "C": cand_C, "D": cand_D, "E": cand_E}
for key, fn in builders.items():
    fn(os.path.join(CAND, f"cand_{key}"))

# Figures used in writeup.md (provisional choice: A as fig1, C as fig2)
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)

Registered primary outcome, door-to-balloon time, was replaced after completion; main text omits this

JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.3729 · PMC10980970 · NCT05415085

Summary

This single-center randomized trial compared culprit lesion PCI before complete coronary angiography with complete angiography first in STEMI. The first ClinicalTrials.gov version of NCT05415085 (v0, 2022-06-08) registered one primary outcome, "Reduction in door-to-balloon time". The second and current version (v1, 2023-10-17) replaced it with "Needle-to-balloon time of 10 minutes or less". v1 was posted 474 days after the registered primary completion date (2022-06-30) and 164 days before publication. The paper reports needle-to-balloon time of 10 minutes or less as "the primary outcome of this study". Its main text does not say that the primary outcome was changed, and it describes the trial as "prospectively registered", although the record was first submitted about 13 months after enrollment began. The only disclosure is one undated sentence in the protocol supplement. The originally registered outcome appears in a single Results sentence as a non-significant difference (61.6 vs 73 minutes; P = .12), and the paper does not identify it as the registered primary.

What was registered

ClinicalTrials.gov NCT05415085 has two versions. The record history lists the modules each version changed, and its lastUpdateVersions field shows that the primary outcomes were last changed in v1 and the secondary outcomes in v0, so the secondary outcomes never changed.

v0, first version. First submitted 2022-04-27, QC 2022-06-08, first posted 2022-06-10. Status RECRUITING; study start 2021-05-01 (actual); primary completion 2022-06-30 (estimated); enrollment 250 (estimated). Primary outcome:

Reduction in door-to-balloon time | Change in door-to-balloon time in minutes | time frame: through study completion, an average of 1 year

The brief summary in v0 reads: "The aim of this study is to assess the impact of culprit-first versus culprit-last percutaneous coronary intervention on the door to balloon time and clinical outcomes in patients with ST-elevation myocardial infarction." The keywords include "door-to-balloon time".

v1, current version. Submitted and QC 2023-10-17, posted 2023-10-19. Changed modules: Study Status, Study Design, Outcome Measures, Contacts/Locations. Status COMPLETED; primary completion 2022-06-30 (actual); study completion 2023-04-13 (actual); enrollment 106 (actual). Primary outcome:

Needle-to-balloon time of 10 minutes or less | Change in needle-to-balloon time in minutes | time frame: through study completion, an average of 1 year

The brief summary and keywords were not changed: the current record still gives the aim as the impact "on the door to balloon time and clinical outcomes" and still lists "door-to-balloon time" as a keyword.

Secondary outcomes, identical in v0 and v1:

Need for hemodynamic (mechanical/medical) support during PCI | Number of patients that needed mechanical support during PCI

Need for respiratory support during PCI | Number of patients that needed respiratory support during PCI

Rate of failed PCI | As determined by the operator

All three have the time frame "through study completion, an average of 1 year".

What the paper reports

Abstract, "Main Outcomes and Measures":

A needle-to-balloon time of 10 minutes or less.

Methods, "Study Protocol and End Point":

The primary outcome of this study was a needle-to-balloon time of 10 minutes or less. Secondary outcomes included the need for hemodynamic support (ie, mechanical and/or medical) during PCI, the need for invasive or noninvasive ventilation during PCI, and the rate of failed PCI.

Methods, "Power Calculation":

Based on previously published data, the primary outcome of a needle-to-balloon time of 10 minutes or less was used for sample size calculation.

Methods, "Study Design and Population", the only statement about the registration:

This study was prospectively registered in Clinical Trials.gov (NCT05415085).

The same paragraph gives the enrollment period: "The study included patients with a diagnosis of STEMI who underwent primary PCI between April 1, 2021, and August 31, 2022."

Results, "Primary and Secondary Outcomes", which holds the paper's only result for door-to-balloon time:

The primary outcome of a needle-to-balloon time of 10 minutes or less was achieved in 46 of the 90 patients (51.1%) who underwent culprit lesion PCI before CAG and in 18 of the 94 patients (19.1%) who underwent CAG followed by culprit lesion PCI (odds ratio, 4.4 [95% CI, 2.2-9.1]; P < .001).

There was no significant difference in the mean (SD) door-to-balloon time (61.6 [45.8] vs 73 [59.8] minutes; P = .12).

The main text contains neither "amend" nor "changed". Door-to-balloon time appears in it only as background (Introduction and Discussion, citing earlier studies) and in the Results sentence above. The Abstract, Key Points and Conclusions do not mention it.

Supplement 1 ("Study Protocol and Statistical Analysis Plan", jamanetwopen-e243729-s001.pdf), section "3. Endpoints", is the only place the change is mentioned:

a. Primary endpoint: o Reduction in Door-to-Balloon (D2B) time Protocol amendment: The investigators ask the IRB for a study protocol amendment due to reconsideration of study protocol and primary outcome: o Revised primary outcome: Needle-to-balloon time within 10 minutes.

The protocol text carries no date for the amendment or for its approval. Its sample-size paragraph uses the revised outcome, in the same words as the paper's Power Calculation section. Its "Study Cohort" section gives an "Estimated number of 300 patients (~150 each group)". The case report form appended to it is labelled "February 2021" and "Ver 1.4". That label belongs to the form; it does not date the amendment.

The discrepancy

Outcome Registered (version, date) In the paper Status
Door-to-balloon time ("Reduction in door-to-balloon time"; change in minutes) Sole primary in v0 (2022-06-08); removed in v1 (2023-10-17) Not named as an outcome. One Results sentence: mean 61.6 vs 73 min, P = .12. Not in the Abstract, Key Points or Conclusions demoted
Needle-to-balloon time of 10 minutes or less Absent from v0; sole primary in v1 (2023-10-17, after primary completion) Primary outcome: 46/90 (51.1%) vs 18/94 (19.1%), OR 4.4 (95% CI 2.2-9.1). Also the basis of the power calculation matched to v1; added relative to v0
Needle-to-balloon time in minutes (the v1 description, "Change in needle-to-balloon time in minutes") Description text of the v1 primary Reported as mean 11.4 vs 17.3 min, P < .001, not labelled primary matched (wording difference: the registry title is a threshold, its description is continuous)
Hemodynamic support during PCI Secondary, v0 and v1 Secondary matched
Respiratory support during PCI Secondary, v0 and v1 Secondary ("invasive or noninvasive ventilation") matched
Rate of failed PCI Secondary, v0 and v1 Secondary matched

The main text gives no account of the change. A reader of the article alone would see a primary outcome that matches the current registry record and a statement that the trial was prospectively registered.

Figures

Timeline of NCT05415085: study start, first registration, the registry version that changed the primary outcome, primary completion and publication

Figure 1. Registry events (blue, above the axis) and paper events (orange, below the axis) on one calendar axis. The paper's enrollment period (April 1, 2021 to August 31, 2022) is the orange bar. The light-grey band marks the period after the registered study start, and the darker band the period after the registered primary completion date (2022-06-30). The record was first submitted on 2022-04-27, 361 days after the registered start and 391 days after the paper's enrollment start, and its first version (v0, QC 2022-06-08) named door-to-balloon time as the primary outcome. The circled event, v1 on 2023-10-17, replaced the primary outcome with needle-to-balloon time of 10 minutes or less and changed enrollment from 250 (estimated) to 106 (actual). It falls after primary completion, after study completion (2023-04-13) and 164 days before publication on 2024-03-29.

Registered primary outcomes by version compared with the outcomes reported in the paper

Figure 2. Left column: the primary outcome in each registry version, quoted from the record (v0, 2022-06-08; v1, 2023-10-17). Right column: how the paper treats each one. The v1 outcome matches the paper's primary outcome, which carries the headline result (51.1% vs 19.1%; OR 4.4). The v0 outcome, door-to-balloon time, is not named as an outcome in the paper. Its only result is one Results sentence reporting no significant difference (61.6 vs 73 minutes; P = .12). The connector labels state the relation (REPLACED, DEMOTED, MATCHED). The bottom panel summarises the disclosure: none in the main text, one undated sentence in Supplement 1. The secondary outcomes are the same in both registry versions and in the paper.

Why it matters

A registered primary outcome lets readers judge whether the outcome that carries a trial's conclusions was chosen before the results were known. Here the first registered primary outcome, door-to-balloon time, is the one the trial's registry summary still names as its aim. It was replaced in the registry on 2023-10-17, after the registered primary completion date and after study completion. The paper's headline findings rest entirely on the replacement outcome: the Key Points finding (51.1% vs 19.1%), the abstract's odds ratio of 4.4, and the conclusion that culprit lesion PCI before complete CAG "resulted in shorter reperfusion times".

For the originally registered primary outcome the paper reports a mean (SD) door-to-balloon time of 61.6 (45.8) vs 73 (59.8) minutes, "no significant difference", P = .12. That result appears once, in the Results text. The Abstract, Key Points and Conclusions do not report it, and nothing in the paper tells the reader that it was the registered primary outcome.

The registry does not show when the decision to change the outcome was made. The protocol supplement records that an amendment was requested but gives no date. Without that date, a reader cannot tell whether the change came before enrollment, during it, or after outcome data were available. Each of these calls for a different reading of the result, and CONSORT 2010 item 6b asks trial reports to state any changes to trial outcomes after the trial commenced, with reasons.

The registration timing adds to this. The record was first submitted on 2022-04-27, 391 days after the paper's enrollment start (April 1, 2021) and 361 days after the registered start (2021-05-01). Under the ICMJE and WHO definition, which requires registration before the first participant is enrolled, the registration was retrospective. The paper's statement that the study was "prospectively registered" therefore does not match the record. The door-to-balloon primary was itself first recorded after about a year of enrollment.

Explanations considered

Separately, three other registry-paper differences are not part of the outcome change and are noted for completeness only. The registry gives enrollment as 106 (actual), while the paper reports 214 randomized in Results (106 and 108 per arm) and 216 in the Abstract. 106 is the number the paper gives for the culprit-lesion-first arm. The registry lists masking as "DOUBLE" (participant, care provider), while the paper describes an "open-label" trial. The registered primary completion date (2022-06-30) precedes the end of enrollment reported in the paper (August 31, 2022).

Status

As of 2026-10-23:

Suggested correction

  1. In Methods, state that the primary outcome registered at ClinicalTrials.gov on 2022-06-08 was door-to-balloon time, that it was changed to needle-to-balloon time of 10 minutes or less, and give the date of the protocol amendment and its IRB approval, the reason, and whether the change was made before or after any outcome data were seen. The registry update date (2023-10-17) should also be stated.
  2. Report door-to-balloon time as the originally registered primary outcome, with its effect estimate and confidence interval, in the Results and in the Abstract.
  3. Replace "This study was prospectively registered" with the registration dates: first submitted 2022-04-27 and first posted 2022-06-10, after enrollment began on April 1, 2021.
  4. Minor: reconcile the number randomized (216 in the Abstract, 214 in Results). Ask the sponsor to update the registry's enrollment figure, masking field and brief summary so that they agree with the trial as reported.

Paper reports dose-limiting toxicity as primary endpoint; the only registered primary outcome is response rate

Cell Rep Med (2026) · doi:10.1016/j.xcrm.2026.102967 · PMC13522775 · NCT05459441

Summary

The ClinicalTrials.gov record for this trial has had one version (2022-07-12), and its sole primary outcome is "Clinical response rate", the percentage of patients with complete or partial response, at 24 months. The published report states that "The primary endpoint was the incidence of dose-limiting toxicities (DLTs)". DLT incidence appears nowhere in the registry, and the paper places radiographic response among "secondary or exploratory endpoints", reported as best overall response rather than at 24 months. The paper cites the registration number but does not mention that the registered primary outcome differs, and it gives no protocol amendment or reason for the difference. The paper reports that "There were no dose-limiting toxicities observed during dose escalation". The registered primary outcome is also reported, in Table 3 and outside the abstract: 2 of 17 efficacy-evaluable patients (11.8%) had a partial response and none had a complete response.

What was registered

The registry history (ClinicalTrials.gov record-history service, checked 2026-10-22 and 2026-10-23) lists exactly one version, version 0, dated 2022-07-12. The service reports outcomesUpdateCount: 0, and the current public record (API v2) has the same outcomes. The record was first submitted on 2022-06-24 and first posted on 2022-07-15. Its last update was posted on 2022-07-15. It gives the study start as 2022-01-01 (actual), so the trial was registered 174 days after it started, and no version exists from before the start date. The phase is "Early Phase 1". The brief title begins "Efficacy and Safety Evaluation of Percutaneous Ommaya Capsule Injection of Autologous Bi-dimensional Specific T Cells…".

Version (date) Type Measure Time frame Description (verbatim)
v0 (2022-07-12) Primary "Clinical response rate" "24 months" "The percentage of patients with CR and PR in the total number of patients in the same period"
v0 (2022-07-12) Secondary "Overall survival (OS)" "24 months" "Frome the starting date of the enrollment until the date of the first documented disease progression or the date of the death from any cause,whichever comes first"
v0 (2022-07-12) Secondary "Safty(adverse events)" "12 months" "Number of participants with treatment-related adverse events as assessed by CTCAE v5.0"

The other registered fields were never updated. The estimated primary completion date is 2023-06-30 and the estimated completion date is 2023-12-31. Enrolment is "30 (ESTIMATED)". The overall status is now "UNKNOWN", with last known status "RECRUITING" and status verified 2022-02. "Dose-limiting toxicity" does not appear anywhere in the record. The record's submission-tracking data show a results submission dated 2026-08-06, released and withdrawn the same day, and another released on 2026-08-08. As of the check, no results are posted and the submitted content is not public.

What the paper reports

The primary endpoint, from STAR Methods, "Objectives and endpoints":

The primary objective of this study was to evaluate the safety and tolerability of intracranial administration of non-chimeric autologous expanded T cells combined with pemetrexed. The primary endpoint was the incidence of dose-limiting toxicities (DLTs), assessed during a prespecified DLT evaluation window defined as inside the intracranial treatment cycle (once weekly for 5 weeks).

Secondary or exploratory endpoints included overall survival (OS), progression-free survival (PFS), and disease control rate (DCR).

Tumor response was determined based on the best overall response from treatment initiation until disease progression or last adequate tumor assessment.

The DLT definition, from STAR Methods, "Dose-escalation design":

DLT was defined as the occurrence of ≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events within the predefined DLT evaluation period.

From the Introduction:

The primary objective was to assess safety and tolerability, with comprehensive monitoring of both systemic and intracranial AEs.

Clinical outcomes, including radiographic response, progression-free survival (PFS), and overall survival (OS), were assessed as secondary or exploratory endpoints to characterize potential signals of therapeutic activity.

The result for this primary endpoint, from Results, "Safety of SCOPE-T treatment":

There were no dose-limiting toxicities observed during dose escalation; therefore, the highest dose tested (5 × 108 RAK cells) was selected for dose expansion and designated as the recommended phase 2 dose (RP2D).

(The "108" is 10^8 as rendered in the PMC text.) The response results appear in Results under the heading "Exploratory radiographic and clinical assessments", and in Table 3 ("Summary of clinical responses in the PPS and EES"):

Objective response rate | 2 (10.5%) | 2 (11.8%)

Benefit beyond imaging evaluationa | 19 (100.0%) | 17 (100.0%)

Clinical benefit rate | 19 (100.0%) | 17 (100.0%)

(The trailing "a" marks a footnote listing clinical and laboratory improvements.) The Results attribute clinical benefit to RANO assessment: "Using RANO-BM and RANO-LM criteria, all 17 patients in the EES demonstrated clinical benefit."

The sample size is discussed in STAR Methods, "Study design and patients" ("The selected sample size (n = 25) is consistent with typical phase 1/1b trials designed to evaluate safety and tolerability."), and in "Statistical analysis" ("No formal hypothesis testing or power calculation was prespecified.").

The paper refers to the registration only by its NCT number (Abstract, Introduction) and in the line in STAR Methods, "Clinical trial registration":

This trial was registered with https://clinicaltrials.gov/study/NCT05459441.

The paper never mentions that response rate was the registered primary outcome, or any protocol amendment or change of endpoints. It does not give a registration date. Apart from the per-protocol analysis set (defined in STAR Methods, "Analysis populations", as "patients who completed at least one treatment cycle without major protocol deviations"), the study protocol is referred to only in the Table 4 footer ("OS and PFS: According to protocol, data cutoff as of July 6, 2024."). The same footer adds "OS∗: Updated data follow-up as of May 01, 2026." for a separate survival column. Neither a protocol nor a statistical analysis plan is published with the article: the supplement is Document S1 (Figures S1–S9, Tables S1–S3), Data S1 (xlsx) and Document S2 (the article plus supplement).

Every quotation above was checked character for character against the article as archived in PubMed Central (src/show.py PMC13522775 --grep), and each occurs exactly once. The endpoint sentences, the registration line and the Table 3 rows also appear unchanged in the publisher's typeset PDF (Document S2). make_figures.py.txt reruns the PMC check, and the registry check against API v2.

The discrepancy

Outcome Registered (version, date) In the paper Status
Clinical response rate (CR + PR / total patients) Primary, 24 months (v0, 2022-07-12) Radiographic response among "secondary or exploratory endpoints". Not in the named list of secondary/exploratory endpoints (OS, PFS, DCR). Reported as best overall response in Table 3: ORR 2/19 (10.5%) in the PPS and 2/17 (11.8%) in the EES Demoted; time point changed (best overall response, not 24 months)
Incidence of dose-limiting toxicities Not registered at any level (v0) Primary endpoint, with a 5-week DLT window Added
Treatment-related adverse events (CTCAE v5.0) Secondary, 12 months (v0) Reported (Table 2; Table S3; abstract: "no treatment-related adverse events greater than grade 2"), but not listed as an endpoint. No time frame stated Reported; not named an endpoint; time frame not stated
Overall survival Secondary, 24 months (v0). The registered description reads like a progression-or-death definition Secondary or exploratory: Kaplan-Meier median and 1-year rate Matched by name; 24-month time frame not used
Progression-free survival Not registered (v0) Secondary or exploratory Added
Disease control rate (CR + PR + SD) Not registered (v0) Secondary or exploratory. Table 3 has no row named DCR, but lists PR and SD counts and a "Clinical benefit rate" Added
Enrolment (context) 30, estimated (v0) 25 enrolled; "The selected sample size (n = 25)…" Differs; not explained

Figures

Timeline from 2021 to 2026 with registry events above the axis and trial and paper events below. The single registry version is shown as a bar running from 2022-07-12 to the check date.

Figure 1. Timeline of the trial and its registration. Blue circles are registry events: first submission (2022-06-24), version 0 (2022-07-12; the only version), first posting (2022-07-15) and the registered estimated primary completion date (2023-06-30, never updated), all on the axis. The results submissions (2026-08-06 and 2026-08-08; not posted as of 2026-10-23) share one blue circle, drawn on its stem above the version bar because on the axis it would sit under the publication marker one to three days earlier. Orange squares are trial and paper events taken from the article: study start and first enrolment (2022-01-01), last enrolment (2024-03-30), data cutoff (2024-07-06), journal submission (2025-11-28) and online publication (2026-08-05). The shaded band marks the 174 days between study start and registration. The blue bar shows that version 0 has been in force from 2022-07-12 to the check date (2026-10-23). No registry version changed the primary outcome, either before or after primary completion. The difference arises between the registry and the paper, not within the registry. The registered primary completion date is shown only as the registry's own, never-updated estimate; it is not a measure of reporting time (enrolment ended 2024-03-30 and the paper's data cutoff was 2024-07-06).

Two-column diagram. The left column shows registered outcomes from version 0 and the right column shows outcomes in the paper, joined by labelled connectors: ADDED, DEMOTED, REPORTED, MATCHED, ADDED.

Figure 2. Registered outcomes compared with the outcomes in the paper. The left column (blue) shows the outcomes in version 0 (2022-07-12) with their registered type and time frame. The right column (orange) shows how the paper designates and reports each one. Every connector carries a text label. DLT incidence is labelled ADDED: it is the paper's primary endpoint and is absent from the registry. Its result box quotes the paper: no DLTs were observed during dose escalation. The registered primary, clinical response rate, is labelled DEMOTED and time point changed: the paper treats it as secondary or exploratory and reports it as best overall response, with the result from Table 3 (ORR 2/17, 11.8%, exact 95% CI 1.5–36.4%, in the EES; 2/19, 10.5%, in the PPS; no CR). The registered adverse-event outcome is labelled REPORTED. Overall survival is labelled MATCHED by name. PFS and DCR are labelled ADDED. The footnote gives the response rate over all 25 enrolled patients (2/25 = 8%, exact 95% CI 1.0–26.0%), computed here from Tables 3–4 because the registered description uses "the total number of patients" as its denominator; the paper does not report this figure. Exact intervals are Clopper-Pearson, the method the paper states for categorical variables.

Why it matters

On severity: safety as the primary endpoint is conventional for a first-in-human phase 1 study, the paper repeatedly calls its efficacy findings exploratory, and the registered primary result appears in Table 3. The safety conclusion therefore does not depend on the discrepancy. What is missing is any statement that the registered primary outcome was different.

Explanations considered

  1. Is this a secondary analysis or a partial report? No. The paper is the primary report of the trial. It covers all 25 patients enrolled between 1 January 2022 and 30 March 2024, at both sites, with safety, response and survival. The registry has a single primary outcome, so it cannot have been split across publications. The other NCT numbers in the paper (NCT02208362, NCT04185038, NCT04196413) belong to other trials and are cited to justify sample size and dosing.
  2. Does the paper disclose the change? No. Its only registration statement is "This trial was registered with https://clinicaltrials.gov/study/NCT05459441." No passage refers to the registered outcomes, an amendment, a change of endpoints or a reason for one. Searches of the text and the supplement for "primary outcome", "amend", "registered" and "response rate" found nothing relevant. The only "amend" hit is an acknowledgment of help with "amendments to enhance the manuscript".
  3. Was there a protocol amendment? The Table 4 footer refers to a protocol ("According to protocol, data cutoff as of July 6, 2024.") and the per-protocol set is defined by the absence of "major protocol deviations", but no protocol, amendment history or statistical analysis plan is published. The registry was never updated after 2022-07-12, so it records no amendment either. A change in the protocol may have happened, but there is no public record of one.
  4. Are these the same outcome under different wording? No. CR + PR response rate measures efficacy. DLT incidence ("≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events" within the DLT window) measures safety. The registered safety outcome, the number of participants with treatment-related AEs (CTCAE v5.0) at 12 months, is a secondary outcome. It is also a different measure from DLT incidence, with a different time window.
  5. Is a safety primary endpoint standard for phase 1? Yes. It is a legitimate reason a team might name DLT incidence as primary, and the paper's design (dose escalation guided by DLTs) is consistent with it. It does not account for the registry stating otherwise, though. The registered title begins "Efficacy and Safety Evaluation", and the registered primary outcome is efficacy. The paper does not explain the difference.
  6. Is the registry simply out of date? The record has not been updated since July 2022: the status is "UNKNOWN", and the completion dates and enrolment are still estimates. A stale record could explain why the registry was never aligned with a later protocol. It does not make the change disclosed. The registry is the only public prespecification, and the paper calls its DLT window "prespecified" without saying that the registry lists a different primary outcome.
  7. Did the registration timing play a part? The trial was registered 174 days after it started, so even the registered outcomes were entered after enrolment had begun. The paper neither claims prospective registration nor gives a registration date. This affects how the registration can be interpreted, but it does not explain the outcome difference: the registry and the paper still disagree.
  8. Is it a conversion or typesetting artifact? No. The same endpoint sentences and the same registration line appear in the PMC-archived text and in the publisher's typeset PDF (Document S2).

None of these accounts for the absence of disclosure. The issue documented here is that the difference is not acknowledged. It may well have had a legitimate reason.

Status

As of 2026-10-23:

Suggested correction

  1. A correction notice, or an added paragraph in "Objectives and endpoints". It should state that the ClinicalTrials.gov record (NCT05459441, registered 2022-06-24, after the 2022-01-01 start) lists "Clinical response rate" (CR + PR, 24 months) as the sole primary outcome and does not list DLT incidence. It should also give when and why the primary endpoint became DLT incidence (protocol version and date), or state that the registration was inaccurate from the outset.
  2. The protocol and statistical analysis plan, with version dates, added as supplementary material, so that the "prespecified" DLT window and the endpoint hierarchy can be checked.
  3. The registered primary outcome reported as registered, or a statement of why it cannot be. That means CR + PR over all enrolled patients (2/25 from the paper's Tables 3–4) alongside the EES and PPS figures, and the response status at 24 months (for instance from the May 2026 follow-up the paper already uses for survival), or a note that follow-up did not allow it. The abstract could give the objective response rate next to the disease-stabilization statement.
  4. The ClinicalTrials.gov record updated: outcome measures (with the change recorded in the registry's history), actual enrolment (25), actual primary completion and completion dates, and recruitment status. The results submitted in August 2026, not posted as of 2026-10-23, would let readers check the registered outcomes once they are public.

Registered 6-week re-intervention count reported instead as a 6-month total procedure count, without disclosure

Gut (2018) · doi:10.1136/gutjnl-2017-315335 · PMC6582745 · NCT02685865

Summary

This paper is the primary report of a single-centre randomised trial comparing lumen-apposing metal stents (LAMS) with plastic stents for EUS-guided drainage of walled-off necrosis (WON). In every ClinicalTrials.gov version (v0, 2016-02-18, to v7, 2019-02-08) the primary outcome has a 6-week time frame, and its description defines the counted events as re-interventions: repeat EUS-guided drainage, direct endoscopic necrosectomy (DEN) or a percutaneous catheter "due to inadequate treatment response". v0 to v3 title it "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success". v4 (2017-09-13) changed "re-interventions" to "interventions" in the measure title and in the first sentence of the description, but left the definition, the hypothesis and the time frame unchanged; v4 to v7, including the current record, carry this wording. The paper's primary outcome is instead the "total number of procedures needed to achieve treatment success at 6-month follow-up". This count also includes the index procedure, ERCP, enteral feeding tubes and procedures for adverse events, so it differs from the registered outcome in both time point and construct. The paper does not mention the difference, and it reports no per-patient result or between-group test for the registered outcome.

What was registered

The record has eight versions, v0 to v7 (indices from the ClinicalTrials.gov history API). The record was first submitted on 2015-11-24. The earliest version, v0, is dated 2016-02-18 (first posted 2016-02-19), six days after the actual study start of 2016-02-12. v0 gave the start only as "2016-02", and from v3 (2017-04-21) the start is "2016-02-12" (actual). No version exists on or before the study start.

v0, 2016-02-18 (status Not yet recruiting; estimated enrolment 62). One primary outcome:

Measure: "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")

Description: "The primary objective of this study is to compare the number of re-interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up. Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage. As FCSEMS have larger diameter than plastic stents, the investigators hypothesize that the no. of re-interventions performed will be lower in the FCSEMS group compared to the plastic stent group."

The registered secondary outcome for treatment success was:

"Treatment success" (time frame: "6 week"): "Decrease in the size of WON to <33% of the original size and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement"

The other registered secondaries were technical success (day 0), treatment failure (6 months), recurrence (6 months), procedure duration (day 0), adverse events (6 months), duration of hospitalisation (6 months) and cost analysis (6 months).

v1 (2016-04-12), v2 (2017-02-13), v3 (2017-04-21). No change to the outcome measures. v1 changed the Study Identification module (brief title from "Trial Comparing FCSEMS and Plastic Stents" to "Comparison of FCSEMS and Plastic Stents") and the status, description, conditions and contacts modules; v2 and v3 changed the status module (status and dates) only.

v4, 2017-09-13 (status Active, not recruiting; actual enrolment 60; primary completion "2017-07-10", actual). The module labels include "Outcome Measures". The primary measure title became:

"The number of interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")

In the first sentence of the description, "re-interventions" became "interventions" ("...to compare the number of interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up."). The rest of the description was not changed: the definition still begins "Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage", and the hypothesis still refers to "the no. of re-interventions". The secondary treatment-success description became "Resolution of WON on CT scan and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement", still at "6 week". v4 also made wording edits to two other secondary-outcome descriptions, with no change to their time frames: treatment failure gained the words "or underlying diseaseor underlying disease." (as written), and procedure duration now starts when the "endoscope [is] inserted into the gastrointestinal lumen to perform transmural drainage" instead of "inserted into the oropharynx". This edit was made after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received (2017-09-22).

v5, 2018-05-30. This is the last version before online publication (2018-06-01). The outcomes are the same as in v4.

v6, 2018-07-31, and v7, 2019-02-08. No outcome change. v6 changed the actual primary completion date from 2017-07-10 to 2018-07-10. v7 set the status to Completed. The current record (fetched from /api/v2/studies/NCT02685865 on 2026-10-23; last update posted 2019-02-11) matches v7, and it still gives the primary time frame as "6 weeks".

No version gives a 6-month time frame for the primary outcome. None defines the primary outcome as a count that includes the index procedure, ERCP, enteral feeding tubes or procedures for adverse events.

What the paper reports

The primary outcome, Abstract (Design):

"Primary outcome was comparing total number of procedures to achieve treatment success defined as symptom relief in conjunction with WON resolution on CT at 6 months."

Introduction, final paragraph:

"The primary objective of this randomised trial was to evaluate the effectiveness of LAMS by comparing the total number of procedures required to achieve treatment success between patients undergoing LAMS and plastic stent placement at 6-month follow-up."

Methods > Outcome measures:

"The main outcome measure was to compare the total number of procedures needed to achieve treatment success at 6-month follow-up."

Methods > Definitions:

"The total number of procedures required to achieve treatment success included the index endoscopic intervention, endoscopic reinterventions, percutaneous drain placement, ERCP, enteral feeding tube placement and any procedures performed for the management of adverse events or disease-specific events in each cohort during the 6-month follow-up period."

"Treatment success was defined as resolution of WON on CT scan in association with clinical resolution of symptoms at 6-month follow-up."

Methods > Sample size calculation:

"A two-sided sample size calculation was performed to detect a difference of one in the total number of procedures needed to achieve treatment success (assuming one for LAMS and two for plastic stents)"

Methods > Patient follow-up. This is the only role the 6-week visit has in the paper:

"On initial outpatient follow-up at 6 weeks postintervention, all patients underwent CT to assess treatment response prior to stent removal and an ERCP to assess for pancreatic duct integrity."

The primary result, Abstract (Results) and Table 3 ("Total no. of procedures for treatment success": mean 2.8 (SD 1.2) vs 3.2 (SD 1.5); median 2 vs 3; range 2–7 vs 2–7; p=0.192):

"There was no significant difference in total number of procedures performed (median 2 (range 2–7) LAMS vs 3 (range 2–7) plastic, p=0.192)"

The paper's statements about the registration and the protocol:

The online supplement (gutjnl-2017-315335supp001.pdf, retrieved from the Europe PMC supplementary-files archive) has four tables: costs; "Procedures performed to achieve treatment success"; Poisson regression; and linear regression. Supplementary Table 2 lists procedure totals by type (LAMS vs plastic): "Repeat EUS-guided drainage 2 / 5", "Endoscopic necrosectomy 4 / 6", "Percutaneous drain placement 2 / 5", "Enteral feeding tube placement 7 / 13", "ERCP for pancreatic duct integrity 31 / 29" and "Management of adverse events 14 / 3". The supplement says nothing about the registration or any outcome change.

The discrepancy

Outcome Registered (version, date) In the paper Status
Number of re-interventions (repeat EUS drainage with the same stent type, DEN or percutaneous catheter, for inadequate response) to treatment success Primary in v0–v3 (2016-02-18 to 2017-04-21). From v4 (2017-09-13) the title says "interventions", but the description keeps the re-intervention definition (through v7, 2019-02-08) Not an outcome. The Results text and Supp. Table 2 give only procedure totals by type (LAMS 2+4+2 = 8; plastic 5+6+5 = 16), with no per-patient summary and no test not reported
Time point of the primary outcome "6 weeks" in every version, v0–v7; description "at 6 week follow-up" "at 6-month follow-up" (Abstract, Introduction, Methods) time point changed
Total number of procedures to treatment success (index procedure, endoscopic reinterventions, percutaneous drains, ERCP, enteral feeding tubes, procedures for adverse or disease-specific events) In no version Primary outcome and basis of the sample size calculation; median 2 vs 3, p=0.192 added (as primary)
Treatment success Secondary, "6 week"; v0: "<33% of the original size" plus symptoms; v4 (2017-09-13): "Resolution of WON on CT scan" plus symptoms Secondary, "at 6-month follow-up"; 29 (93.5%) vs 28 (96.6%), p=0.999; also the end point of the primary count time point changed
Technical success, procedure duration, adverse events, length of stay, costs, recurrence, treatment failure Secondary, v0–v7 (day 0 or 6 months) Secondary outcomes or definitions in Methods; reported in Table 3 and the text matched

Figures

Timeline of registration, registry versions, enrolment and publication for NCT02685865

Figure 1. Timeline of NCT02685865 and the paper. Blue marks are registry events (above the axis). Filled circles are the first submission (2015-11-24), v0 (2016-02-18, six days after the actual study start of 2016-02-12) and v4. The diamond is the registered actual primary completion (2017-07-10). Open squares are registry versions that did not change the outcomes (v1–v3, v5, v6, v7); v5 (2018-05-30) sits beside the publication marker. The ringed marker is v4 (2017-09-13), the only version that edited the outcome measures. It falls after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received. Orange marks are the paper (below the axis): the randomisation period it reports (February 2016 to March 2017), receipt (2017-09-22; its axis marker sits under the v4 marker nine days before it, and its stem leads down to the label), online publication (2018-06-01) and print (2019-07). The paper's 6-month primary outcome appears at no point in the registry. v6 (2018-07-31), after publication, moved the actual primary completion date to 2018-07-10. Source: make_figures.py.txt.

Registered versus reported primary outcomes

Figure 2. On the left, in blue, are the registered outcomes. On the right, in orange, are what the paper reports. Each connector carries a label for its status. Top row: the registered primary (re-interventions to treatment success at 6 weeks) was replaced by a 6-month total procedure count. That count adds the index procedure, ERCP, feeding tubes and procedures for adverse or disease-specific events. Its result was median 2 vs 3, p=0.192, with a minimum of 2 in each arm. The v4 title edit is noted in grey. Middle row (dashed outline: an outcome with no reported result): for the registered primary as defined, the paper gives only procedure totals for its three components (8 vs 16), with no per-patient summary, no test and no 6-week split. Bottom row: the registered secondary, treatment success at 6 weeks, is reported at 6 months. The footer lists the other procedure types in the paper's count (Supplementary Table 2) and the paper's only registration statement and only protocol amendment.

Why it matters

The paper reports no result for the registered primary outcome. The registered measure is a per-patient count of re-interventions done for inadequate treatment response, reaching treatment success at 6 weeks, and the paper gives no between-group comparison of it at any time point. The closest data are procedure totals for the three registered components: 8 in the LAMS arm (2 extra transmural tracts, 4 necrosectomies, 2 percutaneous drains) against 16 in the plastic arm (5, 6 and 5). These come from Results > LAMS cohort and Plastic stent cohort ("Additional procedures undertaken due to suboptimal treatment response at 72 hours (for ongoing SIRS) included creation of additional transmural tracts (n=2), direct endoscopic necrosectomy (n=4) and percutaneous drainage catheter placement (n=2)" and "...(n=5), direct endoscopic necrosectomy (n=6) and percutaneous drainage catheter placement (n=5)") and Supplementary Table 2. These are event totals, not patient-level data. They are not split at 6 weeks and are not tested, so the registered comparison cannot be reconstructed from the paper.

The substituted outcome is built differently in ways that bear on its result:

So the paper's null primary result (p=0.192) and its "non-superiority" headline concern a different end point from the one registered. The totals for the registered components are lower in the LAMS arm (8 vs 16), but no test is given and these are not per-patient data. Whether the registered comparison would have been null or not cannot be judged from what is published. The registered hypothesis itself ("the no. of re-interventions performed will be lower in the FCSEMS group") is never tested as stated.

Two further points make the published primary harder to interpret:

This is not a case where a null registered result became a positive headline: the published primary is itself null, and both outcomes are procedure counts. But the paper presents the 6-month total count as the trial's primary outcome. Readers, reviewers and meta-analysts who take it at face value would not know that the registered primary, measured at 6 weeks and restricted to re-interventions, is not reported.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The authors and the journal could add a statement to Methods > Outcome measures, along these lines:

"The primary outcome registered at ClinicalTrials.gov (NCT02685865; first submitted 24 November 2015) was the number of re-interventions (repeat EUS-guided drainage using the same stent type, direct endoscopic necrosectomy, or percutaneous catheter insertion for inadequate treatment response) needed to achieve treatment success at 6-week follow-up. The primary outcome reported here, the total number of procedures (including the index procedure, ERCP, enteral feeding tubes and procedures for adverse events) needed to achieve treatment success at 6-month follow-up, differs from the registered outcome in its definition and time point. [Date of and reason for the change.]"

The correction could also report the registered primary outcome as prespecified: the per-patient number of re-interventions to treatment success at 6 weeks, by arm, with the planned between-group test, or say why it cannot be computed. Treatment success at 6 weeks, the registered secondary, could be reported too. If a protocol amendment or statistical analysis plan set up the 6-month total count, making that document and its date available would let readers check when the end point was fixed relative to unblinded data. The registry record could be updated to give the outcome that was analysed, with a note on when it changed.

Trial report names an unregistered primary outcome and omits registered primary HIV risk behavior

JAMA Intern Med (2014) · doi:10.1001/jamainternmed.2014.2556 · PMC4811188 · NCT00987961

Summary

ClinicalTrials.gov record NCT00987961 has listed the same two primary outcomes, "opioid use" and "HIV risk behavior", in every version from the first (2009-09-30) to the last (2015-11-10). The published primary report calls two outcomes its "prespecified primary outcome variables": entry into buprenorphine treatment and length of illicit opioid use. Illicit opioid use matches the registered "opioid use" outcome. Entry into treatment, which gives the abstract its headline result (72.2% vs 11.9%), is not a registered primary or secondary outcome, and the registered primary outcome HIV risk behavior is not reported or mentioned anywhere in the paper. The paper cites the registration only by its identifier and does not mention these differences.

What was registered

The registry record has 11 versions (v0 to v10). The ClinicalTrials.gov history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true, outcomesUpdateCount: 0 and lastUpdateVersions.primaryOutcomes: 0. The outcomes were therefore set in v0 and never edited. Versions v1 to v10 changed only the Study Status, Sponsor/Collaborators, Contacts/Locations or Study Design modules.

Version Submitted Primary outcomes (registry wording) Time frame
v0 (first version) 2009-09-30 (first posted 2009-10-01) "opioid use"; "HIV risk behavior" "baseline, 1-month, 3-months, 6-months" (both)
v1-v6 2010-03-23 to 2013-07-08 unchanged unchanged
v7 (last before the print issue) 2014-07-16 unchanged unchanged
v8-v10 (after publication) 2015-05-21 to 2015-11-10 unchanged unchanged

The registered secondary outcomes, also unchanged since v0, are "reduction in injection-related medical conditions" and "reduction in emergency department and hospital utilization", with the same time frame.

The descriptive fields say the same thing. The Detailed Description ends: "Primary outcomes are opioid use and HIV risk behavior." The Brief Summary says the study "tests whether starting Suboxone (buprenorphine) during a medical hospitalization, and then providing an appointment (a "link") for after discharge to maintenance buprenorphine in an outpatient setting will reduce HIV risk behavior in individuals who inject opioids." "HIV risk behavior" is also one of the record's three keywords, in v0 and in the current version.

Timing: the registry gives the study start as 2009-09. The paper says patients were identified "From August 1, 2009", so the first registration came about two months after recruitment began. No registry version exists from before enrollment started, but the outcomes in v0 were entered during recruitment, years before the data were complete. Primary completion is given as 2014-03 (actual); v0 estimated 2013-12.

What the paper reports

The paper's primary-outcome definition (Methods, "Primary Outcome Variables"):

The prespecified primary outcome variables were entry into buprenorphine treatment (in-person intake [anytime between study enrollment and 6 months after enrollment]) at the hospital-associated OAT program, confirmed by OAT electronic medical record review, and length of illicit opioid use (number of days of reported opioid use in the 30 days before the 1-, 3-, and 6-month interviews using a standard 30-day time-line follow-back method).

The Introduction (final paragraph) sets out the same two goals, treatment entry first:

This study had 2 primary goals. We sought to determine whether offering hospitalized, opioid-dependent patients initiation and linkage to office-based opioid addiction treatment with buprenorphine would facilitate entry into and increase persistence in buprenorphine treatment. We then examined whether this treatment initiation and linkage would decrease illicit opioid use at 6 months.

The Abstract, Main Outcomes and Measures:

Entry and sustained engagement with buprenorphine OAT at 1, 3, and 6 months (medical record verified) and prior 30-day use of illicit opioids (self-report).

Secondary outcomes (Methods, "Secondary Outcome Variables"): "Secondary outcome variables included time to entry into the buprenorphine program (days to in-person intake at the hospital-associated OAT program, confirmed by electronic medical record review) and OAT days".

The sample-size statement (Methods, Statistical Analysis) concerns opioid use only: "Specifically, the study was powered (1 – β > .8) to detect a 33% between-group difference in the rate of opioid use during follow-up (days of use per 30 follow-up days)." The Discussion's limitations paragraph refers to a single primary outcome: "our primary outcome measure (past 30-day illicit opioid use) also depended on self-report".

On the registration, the paper's only statement is the back-matter line "TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00987961". Searches of the archived text found no occurrence of "HIV" (the only related mention is "human immunodeficiency virus disease", once as background in the Introduction and once as a discharge diagnosis in Results), no occurrence of "risk behavior", "registered" or "amend", and no use of "protocol" except for the treatment protocol. The paper cites no trial protocol, statistical analysis plan or supplement (Europe PMC: no supplementary files).

The discrepancy

Outcome Registered (version, date) In the paper Status
Opioid use / illicit opioid use Primary: "opioid use", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) Primary: days of illicit opioid use in the 30 days before the 1-, 3- and 6-month interviews; IRR 0.60 (95% CI 0.46-0.73) matched
HIV risk behavior Primary: "HIV risk behavior", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) Not named, defined or reported not reported
Entry into buprenorphine treatment Not registered as primary or secondary in any version "Prespecified" primary: in-person intake at the hospital-associated OAT program by 6 months, record-verified; 52/72 (72.2%) vs 8/67 (11.9%) added
Injection-related medical conditions Secondary (v0, 2009-09-30; unchanged) Not reported as an outcome not reported
Emergency department and hospital utilization Secondary (v0, 2009-09-30; unchanged) Not reported as an outcome; the Discussion cites a prior study linking injection drug use to such use not reported
Time to treatment entry; OAT days Not registered Secondary outcomes added

Figures

Timeline of registry versions, recruitment, primary completion and publication for NCT00987961

Figure 1. Timeline of the ClinicalTrials.gov record and the trial. The filled blue marker is v0 (2009-09-30), the only version that defined the outcomes. The hollow markers are v1 to v10, none of which edited the primary or secondary outcomes, before or after the registry's actual primary completion date (2014-03). The orange lane shows dates from the paper: recruitment from 2009-08-01 to 2012-10-31 (so v0 came about two months after recruitment began), presentation of the results at two meetings in April and June 2013, and the print issue of 2014-08-01. The Crossref DOI record was created on 2014-07-01; the online-first date could not be confirmed from the publisher. Study start and primary completion are month-level registry dates, drawn mid-month.

Registered primary outcomes compared with the primary outcomes reported in the paper

Figure 2. Registered primary outcomes (left; v0, 2009-09-30, unchanged in all later versions) against the outcomes the paper calls its prespecified primary outcomes (right; Methods, "Primary Outcome Variables"). Registered "opioid use" matches the paper's days of illicit opioid use, with the same 1-, 3- and 6-month time points; the paper reports IRR 0.60 (95% CI 0.46-0.73) for it. Registered "HIV risk behavior" has no counterpart in the paper. Entry into buprenorphine treatment appears as a primary outcome in the paper but in no version of the registration, and it carries the abstract's headline result. The note at the foot covers secondary outcomes: neither registered secondary is reported, and the paper's two secondaries were not registered.

Why it matters

The abstract's first and largest result is for an outcome the registration does not list: "linkage participants were more likely to enter buprenorphine OAT than those in the detoxification group (52 [72.2%] vs 8 [11.9%], P < .001)". The abstract conclusion that linkage "is an effective means for engaging medically hospitalized patients who are not seeking addiction treatment" rests on this outcome. Calling it "prespecified" leads readers to treat it as a confirmatory primary result, but the public record does not show that it was prespecified as an outcome at all. Entry into the linked program is also an outcome that follows closely from the intervention itself (the linkage arm was given an intake appointment and a bridging prescription), which makes the choice of primary outcome especially relevant to how the result is read.

For the registered primary outcomes, the paper reports one and not the other:

The registered secondary outcomes (injection-related medical conditions; emergency department and hospital use) are also not reported. The Discussion suggests the reduction in opioid use "is likely to translate into improved health outcomes", and the Conclusions say future work "should evaluate whether decreased illicit opioid use and increased use of OAT in the 6 months after hospital discharge could have health benefits and prevent subsequent hospitalizations". Emergency department and hospital use was, however, a registered secondary outcome of this trial, with a time frame of baseline to 6 months.

The paper does acknowledge multiplicity in general terms ("Given multiple outcomes, readers concerned about overall type I error rates may consider P < .01 a conservative standard to evaluate statistical significance."), but a reader cannot tell from it how many outcomes were planned or which ones were dropped.

Explanations considered

Status

As of 2026-10-23:

Sources: ClinicalTrials.gov NCT00987961 (current record, version history and v0, v6, v7 records); PMC4811188 as archived in PubMed Central; Crossref and Europe PMC metadata for doi:10.1001/jamainternmed.2014.2556.

Suggested correction

A correction or clarification notice could:

  1. State the primary and secondary outcomes as registered at ClinicalTrials.gov (NCT00987961, first submitted 2009-09-30 and unchanged since): opioid use and HIV risk behavior at baseline, 1, 3 and 6 months; secondary outcomes injection-related medical conditions and emergency department and hospital use.
  2. Say when and why entry into buprenorphine treatment became a primary outcome and HIV risk behavior was dropped, and point to the document (protocol, grant application or analysis plan) in which the outcomes described as "prespecified" were set out, ideally making that document available.
  3. Report the results for HIV risk behavior and for the registered secondary outcomes, or say where they are reported or why they cannot be.
  4. Label entry into buprenorphine treatment in the abstract and Methods as an outcome not listed in the trial registration, and make the abstract's description of the illicit opioid use IRR match the Results.

The registry record could also be updated with a note describing the outcomes as analysed, so that readers of either source can see the relationship between them.

Registered primary outcome is overall survival; the final report uses melanoma-specific survival without comment

N Engl J Med (2014) · doi:10.1056/NEJMoa1310460 · PMC4058881 · NCT00275496

Summary

Every version of the ClinicalTrials.gov record for MSLT-I (NCT00275496), from the first on 2006-01-10 to the current one dated 2015-09-01, gives one primary outcome: whether sentinel-node biopsy with selective lymphadenectomy "will effectively prolong overall survival", with a 10-year time frame added in 2009. The final trial report states instead that "The primary end point was melanoma-specific survival (survival until death from melanoma)." The paper does not mention the registered outcome or explain the difference, and it reports no overall-survival estimate, hazard ratio or curve. The only all-cause mortality information in the paper is a set of crude death counts in the Fig. 1B flow diagram. The trial was registered about 12 years after it started, so the registry entry may not reflect the original protocol. Even so, the registered primary outcome had been public for eight years before publication and the paper does not reconcile the two.

What was registered

ClinicalTrials.gov has 10 versions of the record. All of them postdate the study start (1993-11) and the end of enrollment (2002). The primary outcome measure text is the same, character for character, in all 10:

To determine whether wide excision of the primary with intraoperative lymphatic mapping (LM) followed by selective lymphadenectomy will effectively prolong overall survival compared to wide excision of the primary melanoma alone.

Version Date Primary outcome What changed in Outcome Measures
v0 2006-01-10 (first submitted; first posted 2006-01-12) text above, no time frame first registration
v1 2006-01-20 unchanged secondary only: "Sentinel node metastases in WEX arm; Incidence of nodal metastases" became "Incidence of Sentinel Node Metastases (biopys) vs clinical metastases (observation)"
v2, v3 2006-11-29, 2007-11-02 unchanged none (Study Status, References)
v4 2009-04-30 unchanged; time frame "10 years" added "10 years" added to primary and secondary; "biopys" corrected to "biopsy"
v5, v6 2011-11-07, 2012-07-30 unchanged none (v6 records primary completion 2012-06, actual)
v7 2012-11-19 (last version before publication) unchanged, "10 years" none (Study Status, References)
v8, v9 2014-05-13, 2015-09-01 unchanged, "10 years" none (Study Status only)

The registered secondary outcomes in v4 to v9 are: "Disease-free survival; Incidence, timing, and anatomic distribution of distant metastases; Morbidity of procedures; Significance of TA90 levels; Incidence of Sentinel Node Metastases (biopsy) vs clinical metastases (observation); Accuracy of LM" (time frame "10 years"). Melanoma-specific survival does not appear as a primary or secondary outcome in any version.

Other registry dates: study start 1993-11; primary completion 2012-06 (actual); last update posted 2015-09-02. The current record lists this paper (PMID 24521106) as a "derived" citation and three "result" citations: the 2006 third interim report (N Engl J Med 2006 Sep 28, PMID 17005948); a 2010 Ann Surg Oncol paper on morbidity and length of stay after early versus delayed completion lymphadenectomy (PMID 20614193); and a 2012 Ann Surg Oncol paper on metastasectomy for distant metastases in MSLT-I patients (PMID 22648554). The two Ann Surg Oncol citations first appear in v7. The registry records masking as None (open label) in every version.

What the paper reports

The primary end point, in Methods, Statistical Analysis (first paragraph):

The primary end point was melanoma-specific survival (survival until death from melanoma). The secondary end points, described previously,11,12 included disease-free survival, survival with tumor-positive or tumor-negative sentinel nodes, and the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases.

References 11 and 12 are the 2006 third interim report in the same journal and a 1999 Ann Surg report on the accuracy of lymphatic mapping (PMC1420894). Neither is a protocol or a statistical analysis plan.

The registration appears once, at the end of the abstract:

(Funded by the National Cancer Institute, National Institutes of Health, and the Australia and New Zealand Melanoma Trials Group; ClinicalTrials.gov number, NCT00275496.)

The main results for the paper's own primary end point (abstract; Results, Survival Rates):

No significant treatment-related difference in the 10-year melanoma-specific survival rate was seen in the overall study population

hazard ratio for death from melanoma in the biopsy group, 0.84; 95% confidence interval [CI], 0.64 to 1.09; P = 0.18

This is for patients with intermediate-thickness melanomas (1.20 to 3.50 mm), the primary study group.

The analysis methods name only melanoma-specific and disease-free survival (Methods, Statistical Analysis, third and fourth paragraphs):

The Kaplan–Meier method was used to estimate mean 5-year and 10-year rates of melanoma-specific and disease-free survival.

Comparisons of melanoma-specific and disease-free survival were based on data from patients who underwent their assigned treatment.

The Discussion (fourth paragraph) refers to survival without saying which kind:

We did not see a significant survival advantage for all patients with intermediate-thickness melanomas.

The design changes the paper does disclose are a larger sample size ("After the second interim analysis, the sample size for the intermediate-thickness group was increased to 1200 patients") and the latent-subgroup method ("In post hoc analyses, a latent-subgroup analytic method was developed"). Both are in Methods, Statistical Analysis.

What the paper does not contain: searches of the full text as archived in PMC (abstract, body, table, figure legends) for "overall survival", "all-cause", "any cause" and "amend" find nothing. Every use of "protocol" refers to the per-protocol analysis. The one use of "registered" refers to enrollment ("Enrollment closed in 2002, after 2001 patients had been registered."). The Supplementary Appendix archived with the PMC manuscript (16 pages) has no overall-survival analysis and no statement about end points or protocol amendments. Its survival analyses are all melanoma-specific, disease-free or distant disease-free.

All quotations above were checked verbatim with src/show.py PMC4058881 --grep, and each full quotation occurs exactly once in the archived text. The Fig. 3 legend repeats the definition "survival until death from melanoma". Registry quotations were checked against the archived version records and the current API v2 record.

The discrepancy

Outcome Registered (version, date) In the paper Status
Overall survival (death from any cause), 10 years Primary in every version, v0 2006-01-10 to v9 2015-09-01 (time frame "10 years" from v4 2009-04-30; v7 2012-11-19 is the last version before publication) Not named as an end point. No estimate, hazard ratio or curve. Fig. 1B gives only crude counts of deaths by cause not reported
Melanoma-specific survival, 10 years In no version, as either primary or secondary "The primary end point was melanoma-specific survival" added (as primary)
Time frame of the primary outcome 10 years (from v4 2009-04-30) 10-year rates matched
Disease-free survival, 10 years Secondary, all versions (time frame "10 years" from v4 2009-04-30) Secondary. The abstract's conclusions cite it first as a treatment benefit matched
Incidence of sentinel-node metastases vs clinical metastases Secondary, v1 onward Secondary ("the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases") matched
Distant metastases (incidence, timing, distribution) Secondary, all versions Reported as distant disease-free survival in the node-positive subgroup (Results; Fig. S2) matched (as distant disease-free survival)
Survival with tumor-positive or tumor-negative sentinel nodes In no version Secondary added (as secondary)

The core finding is two rows: the registered primary outcome (overall survival) is not reported, and a different measure (melanoma-specific survival) is reported as the primary end point. Neither the substitution nor the reason for it is stated in the paper.

Figures

Timeline of MSLT-I registration, registry versions and publications

Figure 1. Timeline of the trial and its registry record. The trial started in 1993-11 (registry; the paper says it "commenced in 1994") and enrolled 2001 patients until 2002. It was first registered on 2006-01-10, about 12 years after the start. Every registry version therefore postdates the study start and the end of enrollment. Diamonds mark the versions that created or edited the Outcome Measures module: v0 (primary outcome overall survival), v1 (secondary wording only) and v4 (time frame "10 years" added to the unchanged primary). Open circles mark versions that changed other modules. The hatched band, bounded by the dashed line, is the period after primary completion (2012-06). The four versions in it did not touch outcomes: v6 (2012-07-30, Study Status; status set to Completed and primary completion 2012-06 recorded as actual), v7 (2012-11-19, Study Status and References; the last version before publication), and v8 (2014-05-13) and v9 (2015-09-01), Study Status only. The final report (2014-02-13) names melanoma-specific survival as its primary end point. Sources: ClinicalTrials.gov record history; the paper as archived in PMC; the registry's reference to the 2006 interim report.

Registered versus reported primary outcome

Figure 2. Registered primary outcome versus the primary end point in the paper. Left, in blue: the registry's primary outcome as of v7 (2012-11-19), the last version before publication, worded the same since v0 (2006-01-10). Right, in orange: the paper's stated primary end point. The two use the same 10-year time frame but measure different things: overall survival counts deaths from any cause, while melanoma-specific survival counts only deaths from melanoma. The paper gives no overall-survival result. The nearest data are the crude counts of deaths by cause in the Fig. 1B flow diagram, which are not time-to-event estimates and do not account for losses to follow-up. Melanoma-specific survival appears in no registry version. Its result in the primary study group is null (hazard ratio 0.84, 95% CI 0.64 to 1.09, P = 0.18). Disease-free survival, a registered secondary outcome, is reported as a secondary end point.

Why it matters

Overall survival and melanoma-specific survival differ in more than wording. Melanoma-specific survival treats deaths from other causes as censored, so a treatment effect on non-melanoma mortality cannot appear in it. It also depends on cause-of-death attribution, which may not be blinded in an open surgical trial (the registry records masking: None in every version). For a comparison of an extra surgical procedure (sentinel-node biopsy, with immediate completion lymphadenectomy when positive) against observation, all-cause mortality is the outcome the registry named, and it captures any harm or benefit regardless of cause.

What the paper reports for the registered primary outcome: no estimate. No overall-survival rate, hazard ratio, P value or Kaplan–Meier curve appears in the abstract, main text, Table 1, figure legends or the archived Supplementary Appendix. The only related information is in the Fig. 1B flow diagram (image), which lists deaths by cause among patients included in follow-up. For intermediate-thickness melanomas: biopsy group 133 died from melanoma and 47 from other causes, 180 of 805 in all (22.4%); observation group 103 and 25, 128 of 522 (24.5%). For thick melanomas: biopsy 68 and 19, 87 of 185 (47.0%); observation 43 and 9, 52 of 126 (41.3%). These are crude proportions with unequal losses to follow-up (120 and 68 in the intermediate groups; 18 and 9 in the thick groups). They are not a substitute for the registered time-to-event analysis, and this write-up draws no inference about treatment effect from them. They show that the paper did record deaths from other causes, so an overall-survival analysis was possible from the trial data.

The abstract's positive conclusions rest on outcomes other than the registered primary: disease-free survival (a registered secondary), melanoma-specific and distant disease-free survival in the node-positive subgroup (a comparison the paper itself says raises "concern about ascertainment bias"), and a post hoc latent-subgroup analysis. The paper's own primary end point is stated openly as null in the whole primary study group. A reader comparing the paper with the registry cannot tell from the paper whether the registered overall-survival comparison was done or what it showed.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The authors and journal could add a correction or a note to the article that:

  1. States that the ClinicalTrials.gov record (NCT00275496, first registered 2006-01-10) lists overall survival at 10 years as the primary outcome, and that the paper's primary end point is melanoma-specific survival.
  2. Says when and why the primary end point was set as melanoma-specific survival. If the original pre-2006 protocol named melanoma-specific survival, the correction could say so and make that protocol available.
  3. Reports the registered primary outcome: 10-year overall survival by randomized group in the intermediate-thickness (primary study) group, with the Kaplan–Meier estimates, hazard ratio, 95% CI and P value, analyzed the same way as the melanoma-specific survival results (and by intention to treat).
  4. If the registry entry does not reflect the protocol, the sponsor could update ClinicalTrials.gov with a dated note explaining the difference, so the record and the publication agree.

HTC uptake, the trial's first primary outcome, appears in no version of its registration

PLoS Med (2014) · doi:10.1371/journal.pmed.1001768 · PMC4267810 · NCT01459120

Summary

This cluster-randomized trial compared home-based HIV testing and counseling (HB-HTC) with mobile clinic HTC (MC-HTC) in rural Lesotho. The paper names uptake of HTC as its first primary outcome, states its main hypothesis in terms of uptake, bases its sample size on uptake, and leads the abstract results with uptake (92.5% vs 86.7%, aOR 2.06, p = 0.011). None of the four versions of ClinicalTrials.gov record NCT01459120 lists HTC uptake as a primary or a secondary outcome, and the registered outcomes never changed between versions. The paper has a section on deviations from the registration, but it discloses only that a registered cluster-level outcome was dropped, and it describes uptake as one of the endpoints that "were kept", which the record does not bear out.

What was registered

ClinicalTrials.gov holds four versions of NCT01459120. The record history lists them as v0 (submitted 2011-10-18, QC 2011-10-21, first posted 2011-10-25), v1 (2011-10-31; modules changed: Study Status, Contacts/Locations; the start date went from 2011-11 to 2011-10), v2 (2011-12-09; Study Status, Contacts/Locations) and v3 (2012-02-08, posted 2012-02-09; Study Status only; status COMPLETED; primary completion 2012-02, actual). The history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true and an outcomes update count of 0. I compared the outcomes module of v0, v1 and the current record (v3) field by field. They differ only in HTML character encoding (for example HIV&#x2F;AIDS for HIV/AIDS) and line breaks. Enrollment is 1800 (ESTIMATED) in every version.

The five registered primary outcomes, identical in v0 (2011-10-18) through v3 (2012-02-08), are quoted here from the current record:

  1. "Proportion of newly tested HIV-positive clients over all clients tested for HIV during the campaigns". No time frame is given.
  2. "Proportion of clients newly tested HIV-positive who enrolled into HIV/AIDS care within one month over all clients newly tested HIV-positive during the campaigns". Time frame: "4 weeks after tested positive".
  3. "Overall number newly tested HIV-positive and enrolled into chronic HIV/AIDS-care at each facility". Time frame: "4 weeks after campaign". The description says this compares the facility-level total "Irrespective if these patients were tested during one of the campaigns or during routine activities at the facility" and the total "from the 5 villages where campaigns were held as compared to the five villages where no campaigns were held."
  4. "Absolute number of newly tested HIV-positive clients". No time frame is given.
  5. "Absolute number of clients newly tested HIV-positive who enrolled into HIV/AIDS care". Time frame: "4 weeks after tested HIV-positive".

The ten registered secondary outcomes cover CD4 count, WHO stage, four tuberculosis screening and sputum outcomes with their absolute-number counterparts, "Proportion of first-time HIV-testers among all clients accessing the testing services" and "Demographic characteristics of clients accessing the voluntary counseling and testing services". No primary or secondary outcome measures acceptance of testing among the people offered it. In outcome 1, "all clients tested" serves only as the denominator for HIV positivity.

What the paper reports

Methods, "Study Hypothesis, Design, and Setting":

The study's main hypothesis was that the HB-HTC approach would result in a higher uptake of HTC than the MC-HTC approach among individuals accessing the multi-disease campaigns.

The same section dates the trial "from 17 October to 25 November 2011".

Methods, "Outcomes at the Individual Level":

The trial had three primary and five secondary outcomes. The first primary outcome was the number of persons taking up HTC.

Uptake was defined as acceptance and completion of HTC with pre-test counseling, HIV test, and post-test counseling.

The second primary outcome was the proportion of participants newly tested HIV-positive among participants who took up HTC.

The third primary outcome was linkage to care within 28 d after a positive HIV test among participants who tested HIV-positive during the campaign.

Methods, "Sample Size and Statistical Analysis":

we estimated that HTC uptake would be about 65% in the MC-HTC arm

Abstract, "Methods and Findings":

The trial was powered to test the hypothesis of higher HTC uptake in HB-HTC campaigns than in MC-HTC campaigns.

Overall HTC uptake was higher in the HB-HTC group than in the MC-HTC group (92.5% versus 86.7%; adjusted odds ratio [aOR]: 2.06; 95% CI: 1.18–3.60; p = 0. 011)

(The space in "0. 011" is in the archived text.)

Methods, "Deviations from Initial Trial Registration", which is the paper's only statement about the registration:

Aggregate data from clusters on numbers of persons newly tested HIV-positive through routine facility-based HTC during the 28 d following the study campaigns were initially registered as one of the primary outcomes. While finalizing the internal study protocol, cluster data were dropped as a primary outcome because the trial was designed and powered to measure differences in outcomes at the individual level and not at the cluster level. Only individual endpoints were kept as primary outcomes—uptake of HTC, participants newly detected HIV-positive, and subsequent enrollment in HIV care—as these endpoints represent the three major steps of the HTC cascade. Nevertheless, description of cluster data was maintained and is reported in this paper.

Table 2 ("Primary outcomes") reports, for HB-HTC vs MC-HTC: uptake of HTC overall 92.5% (1,083/1,171) vs 86.7% (1,207/1,392), aOR 2.06 (1.18–3.60), p = 0.011; participants newly detected HIV-positive 3.6% (39/1,083) vs 6.2% (75/1,207), aOR 0.64 (0.48–0.86), p = 0.002; linkage to care 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978.

The supporting information includes Text S2, "Trial protocol", an undated internal protocol for a study "to be held from October to December 2011". Its outcome table lists "Proportion and absolute number of clients accepting HTC" first among the primary outcomes, and its sample-size section reads "Expected difference in HTC-uptake between study-arms (1st primary outcome): 80% (door-to-door) versus 65% (pitso)". The protocol does not mention the registry record.

The discrepancy

Outcome Registered (version, date) In the paper Status
Uptake of HTC (acceptance and completion of testing among eligible attendees) Not registered, as primary or secondary, in v0 (2011-10-18), v1, v2 or v3 (2012-02-08) First primary outcome; main hypothesis; basis of the sample size; first result in the abstract (aOR 2.06, p = 0.011) added (not disclosed)
Proportion newly HIV-positive among those tested Primary 1, v0–v3 Second primary outcome: 3.6% vs 6.2%, aOR 0.64, p = 0.002 matched
Proportion newly HIV-positive enrolled in care within one month / 4 weeks Primary 2, v0–v3 Third primary outcome, "within 28 d": 25.6% vs 25.3%, aOR 0.99, p = 0.978 matched (4 weeks = 28 d)
Facility-level number newly HIV-positive and enrolled in care, 4 weeks after campaign Primary 3, v0–v3 Dropped as primary; reported descriptively (49 newly HIV-positive at facilities in 28 d; 7 from control villages); no arm comparison; facility-level enrolment not reported demoted (disclosed)
Absolute number newly HIV-positive Primary 4, v0–v3 Counts 39 vs 75 shown within the second primary outcome; not named as a primary outcome demoted (reported as count, not disclosed)
Absolute number newly HIV-positive enrolled in care, 4 weeks Primary 5, v0–v3 Counts 10 vs 19 shown within the third primary outcome; not named as a primary outcome demoted (reported as count, not disclosed)

Figures

Timeline of NCT01459120 registry versions and trial milestones

Figure 1. Timeline of the trial. Squares are ClinicalTrials.gov versions; circles are dates from the paper and its Text S2 protocol. Clusters were randomised at a meeting "held on October the 3rd" (Text S2; its campaign timetable runs from 17 October 2011), and campaigns ran from 17 October to 25 November 2011. The record was first submitted on 18 October 2011, one day after the campaigns began. All four versions therefore came after the study started, and v3 (8 February 2012, status COMPLETED) falls in the month of primary completion. No version changed any outcome; the versions changed only status, dates and locations. The record was not updated again before publication on 16 December 2014 (the axis is broken between March 2012 and December 2014).

Registered primary outcomes versus primary outcomes reported in the paper

Figure 2. Registered primary outcomes (left, blue; v0 of 18 October 2011, unchanged through v3 of 8 February 2012) and the primary outcomes reported in the paper (right, orange), with each registered item's fate labelled. The paper's first primary outcome, HTC uptake, has no registered counterpart in any version (ADDED). The two registered proportions match the paper's second and third primary outcomes (MATCHED), whose results are shown in the boxes, including the null linkage result. The registered absolute numbers appear only as counts (COUNT ONLY). The cluster-level outcome was dropped as primary and reported descriptively (DEMOTED), which is the only change the paper discloses.

Why it matters

Uptake is the outcome on which the paper's design and headline rest. The main hypothesis, the power calculation and the first result in the abstract all concern uptake, and the abstract's conclusion that "HB-HTC was better in reaching children, individuals who had never tested before, and men" draws on it. A reader who checks the registration to see whether the main comparison was pre-specified finds no uptake outcome at all. The deviations section says uptake was "kept" as a primary outcome, which reads as if uptake had been registered.

The uptake result is also sensitive to how it is analysed. The paper reports that "This overall difference was due to a much higher uptake in the age group <12 y in the HB-HTC group (Table 2)", where uptake was 87.5% vs 58.7% (aOR 4.91). It also reports that "Among participants ≥12 y of age, uptake tended to be lower in the HB-HTC group compared to the MC-HTC group (Table 2)": 94.7% vs 97.1% at 12–24 y (aOR 0.59, p = 0.322) and 93.9% vs 96.1% at ≥25 y (aOR 0.61, p = 0.117). Without a registered definition, a reader cannot tell from the registry which population and analysis were intended to carry the primary comparison.

For the registered primary outcomes the paper does report results, in full. The registered first primary (proportion newly HIV-positive among those tested) favoured the comparator: 3.6% (39/1,083) with HB-HTC vs 6.2% (75/1,207) with MC-HTC, aOR 0.64 (0.48–0.86), p = 0.002. The registered second primary (linkage to care within one month) was null: 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978. The abstract states both. The registered facility-level outcome was not compared between arms, which the paper discloses. So no registered primary outcome is hidden. What the registry does not support is the paper's framing of uptake as the pre-specified, powered primary outcome. On the two registered individual-level primary outcomes, the intervention arm was not better.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

A correction to the article could state that HTC uptake, reported as the first primary outcome and the basis of the sample-size calculation, was not listed as a primary or secondary outcome in any version of ClinicalTrials.gov record NCT01459120. It could say that uptake was specified as a primary outcome in the internal protocol (Text S2), and give that protocol's date relative to the registration (18 October 2011) and the start of the campaigns (17 October 2011) if it is known. The sentence in "Deviations from Initial Trial Registration" that says uptake was "kept" as a primary outcome could be revised to say that uptake was added relative to the registration. The same section could note that registered primary outcomes 4 and 5 (absolute numbers) are reported as counts rather than as separate primary outcomes. The sponsor could also add a note to the registry record, or post results there, that points readers to the outcome set actually analysed.

The paper calls store sales primary; the registered primary, women's diet quality, is reported as secondary

PLoS Med (2021) · doi:10.1371/journal.pmed.1003729 · PMC8423266 · NCT03518151

Summary

ClinicalTrials.gov NCT03518151 (WRAPPED1) has a single version, submitted on 2018-04-25 (658 days after the registered study start and 360 days after primary completion), which names one primary outcome, women's dietary quality score from a 20-item food frequency questionnaire (FFQ) at 3 months, and lists weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes. The published primary report reverses that order: it names "store sales (primary outcome)", calls women's and children's diets "secondary outcomes", and reports every outcome at 3 and 6 months. The paper cites the registration three times and never mentions that its outcome order differs from the registered one. The registered primary outcome is reported in Table 2 (+0.29 SD, P = 0.04, adjusted for the Index of Multiple Deprivation, IMD; +0.26 SD, P = 0.06, fully adjusted), but the abstract gives no numbers for it.

What was registered

The registry record history lists a single version. It was retrieved through the ClinicalTrials.gov history endpoint (/api/int/studies/NCT03518151/history/0) and checked against the current public API v2 record, which matches it.

Registry version Date Primary outcome Secondary outcomes
v0 (the only version) submitted 2018-04-25; first posted 2018-05-08; last update posted 2018-05-08 Dietary quality score, 3-month follow-up Child's dietary quality score (3 mo); women's daily fruit and vegetable portions (3 mo); women's weekly confectionery intake (3 mo); weekly sales of fresh fruit and vegetables (12 wk); weekly sales of confectionery (12 wk)

The registered primary outcome reads, verbatim:

Dietary quality score — "The primary outcome is women's dietary quality measured using a 20-item food frequency questionnaire (FFQ) to create a standardised dietary quality score (validated against 100-item FFQ and red cell folate)." Time frame: "3 month follow-up post intervention commencement"

The two registered store-sales outcomes read:

Weekly sales of fresh fruit and vegetables (secondary) — "Total weekly sales of fresh fruit and vegetables collected via electronic sales data". Time frame: "12 week period following intervention commencement"

Weekly sales of confectionery (secondary) — "Total weekly sales of confectionery collected via electronic sales data". Time frame: "12 week period following intervention commencement"

Registry dates (all marked "actual"): study start 2016-07-06; primary completion 2017-04-30; study completion 2017-11-30. Enrollment: 150 (actual). Because v0 is the only version, the registry primary outcome was never changed. The record was created after data collection for the primary outcome had ended. The paper agrees: it says the study "took place between April 2016 and March 2017" and that "Recruitment occurred in 3 waves between July and September 2016."

What the paper reports

The paper names store sales as its primary outcome in two places:

It aims to assess whether creating a healthier layout in discount supermarkets in England improves the healthiness of store sales (primary outcome) and the purchasing and dietary behaviours of women customers aged 18 to 45 years (secondary outcomes) after 3 and 6 months.

— Introduction, final paragraph

The data collected included 9 months continuous store sale transaction data (primary outcome) and participant loyalty card data (secondary outcome) provided by the collaborating supermarket to cover 3 time periods

In order to obtain an understanding of intervention effects on household members’ diets (secondary outcomes), interview-administered telephone questionnaires obtained information about participants’ diets and diets of their children aged 2 to 6 years (where applicable) at 3 time points: baseline (time 1) and 3 (time 2) and 6 months (time 3) following intervention commencement.

— Section 2.4, Outcome measures

The store-sales analysis covers four product groups: "Weekly sales per store of fresh fruits and vegetables, frozen vegetables, confectionery, and intervention checkout items" (Section 2.6). The abstract's quantitative results are store-sales and loyalty-card purchasing results. Its first concluding sentence is "Healthier supermarket layouts can improve the nutrition profile of store sales and likely improve household purchasing and dietary quality."

The paper mentions the registration in three places, each time without comment on its content:

The trial registration number is NCT03518151.

— Abstract, Methods and findings

ClinicalTrials.gov NCT03518151 (pre-results)

— Abstract, "Trial registration" line

it was registered with ClinicalTrials.gov (NCT03518151, pre-results).

— Section 2.1, Study design and setting

The paper describes itself as "a pilot study with a prospective matched controlled cluster design" (Section 2.1). The article text (excluding the reference list) contains none of the strings "retrospectiv", "amend", "changed", "protocol", "analysis plan", "prespecified" or "pre-specified". The word "deviat" occurs only in "standard deviation". The one statement about decisions made after data collection concerns analysis methods, not which outcome is primary:

The distributions of the data were unknown in advance of study commencement because this was a pilot study. The research team therefore made decisions about the statistical analyses following (i) completion of initial analyses to determine the shape of the outcome data; and (ii) consultation with the literature to identify appropriate statistical methods for the various outcome variables.

— Section 2.6, Statistical analysis

The discrepancy

Outcome Registered (v0, 2018-04-25) In the paper Status
Women's dietary quality score (20-item FFQ, standardised) Primary, 3-month follow-up Secondary ("household members’ diets (secondary outcomes)"); reported at 3 and 6 months (Results 3.4, Table 2) demoted; time point changed (6 months added)
Weekly sales of fresh fruit and vegetables Secondary, 12-week period after intervention start Primary ("store sales (primary outcome)"); interrupted time series at intervention, 3 and 6 months (Results 3.2) added as primary; time point changed
Weekly sales of confectionery Secondary, 12-week period after intervention start Primary (same); at intervention, 3 and 6 months added as primary; time point changed
Frozen-vegetable sales; intervention checkout-item sales Not registered Part of the store-sales (primary) analysis (Sections 2.6, 3.2) added
Child's dietary quality score Secondary, 3 months Secondary, 3 and 6 months (Table 2) matched; time point changed (6 months added)
Women's daily fruit and vegetable portions Secondary, 3 months Secondary, 3 and 6 months (Table 2) matched; time point changed (6 months added)
Women's weekly confectionery intake Secondary, 3 months Secondary, 3 and 6 months (Table 2) matched; time point changed (6 months added)
Loyalty-card purchasing (fruit and vegetables, confectionery, other categories) Not registered "participant loyalty card data (secondary outcome)" (Sections 2.4, 3.3) added
Economic outcomes (grocery spend, total store sales) Not registered Reported (Section 3.5) added

Every registered outcome is reported, so nothing is missing. The difference is in the outcome hierarchy. The only registered primary outcome is presented as secondary. Two registered secondary outcomes, together with two unregistered product groups, become the primary outcome, and a 6-month time point is added throughout.

Figures

Timeline of NCT03518151: study start, primary completion, first and only registration, and publication

Figure 1. Trial timeline. Blue items are registry dates; orange items come from the paper or the publisher's article page. The only registry version (v0) was submitted on 2018-04-25. That is 658 days after the registered study start (2016-07-06) and 360 days after the registered primary completion (2017-04-30). The orange bars show the study period (April 2016 to March 2017) and the recruitment window (July to September 2016) as the paper reports them. No later registry version exists, so the primary outcome was never changed in the registry. The paper (received 2020-10-15, accepted 2021-07-12, published 2021-09-07) appeared while v0 was still the only version.

Registered vs reported outcome hierarchy for NCT03518151

Figure 2. The registered outcome hierarchy (left, ClinicalTrials.gov v0, 2018-04-25) compared with the hierarchy in the paper (right). Each connector is labelled with what changed. The sole registered primary outcome, women's dietary quality score at 3 months, is reported as a secondary outcome (DEMOTED), and a 6-month time point is added. The registered secondary weekly sales outcomes become the paper's primary outcome (PROMOTED, in the table above "added as primary"), and the time frame changes from 12 weeks to 3 and 6 months. Loyalty-card purchasing, frozen-vegetable and checkout-item sales, and economic outcomes appear only in the paper (ADDED). The bottom panel gives the paper's results for the registered primary outcome (Table 2), with the 3-month headline store-sales results for comparison, and notes that the paper discloses nothing about the difference.

Why it matters

A primary outcome named in advance limits the choices that can be made after the data are seen: which outcome is the main test, which time point counts, and what the abstract leads with. Here the only registration was created after primary completion. The paper's hierarchy then differs from that registration and the paper does not say so. A reader who opens the cited NCT number finds a different primary outcome from the one the paper names, and the paper gives no explanation.

The two hierarchies lead to different headlines. The abstract's numerical results and its first conclusion are about store sales. Fruit and vegetable sales rose by +1.71 SD (95% CI 0.45, 2.96; P = 0.01) at 3 months and +2.42 SD (0.22, 4.62; P = 0.03) at 6 months. Confectionery sales fell by −1.05 SD (−1.98, −0.12; P = 0.03) at 3 months and −1.37 SD (−2.95, 0.22; P = 0.09) at 6 months (Abstract; Results 3.2). The registered primary outcome is reported in Results 3.4 and Table 2:

Women's diet quality score, intervention vs control, change from baseline Beta (SD) 95% CI P n
3 months, adjusted for IMD (registered time point) 0.29 0.01, 0.57 0.04 138
3 months, adjusted for IMD and DAG-selected confounders 0.26 −0.01, 0.54 0.06 132
6 months, adjusted for IMD (not a registered time point) 0.10 −0.16, 0.36 0.45 125
6 months, fully adjusted 0.10 −0.17, 0.37 0.48 119

Table 2 labels the IMD-only model "Unadjusted"; its footnote says "* Unadjusted = adjusted for IMD (to control for similarities between pairs of stores)." The Results text reports the 3-month IMD-adjusted estimate first: "After adjustment for IMD, diet quality score at 3 months follow-up increased from baseline by 0.29 SDs ((95% CI 0.01, 0.57), P = 0.04) among intervention women compared to those shopping in control stores."

So the registered primary outcome points the same way as the headline sales results. Its 95% CI excludes zero only in the IMD-adjusted model, and the 6-month estimate is close to zero. The abstract summarises it only as "Changes in dietary variables were predominantly in the expected direction for health benefit." The Discussion calls the diet findings "more equivocal". The paper also says it is "a pilot study, and, therefore, not powered to detect an effect" (Abstract). If the registered order had been kept, the abstract would have led with a borderline 3-month dietary effect rather than with store-sales effects. Readers and systematic reviewers should know which of these the trial was registered to test.

Explanations considered

There may be legitimate reasons for the paper's hierarchy, for example the store-level sales data being the most complete data source. This write-up does not assess them. The issue documented is only that the difference from the registered primary outcome is not disclosed.

Status

As of 2026-10-23:

Suggested correction

A correction or clarification would make the outcome hierarchy match the registry, or explain why it does not. It could do three things:

  1. State in the Methods (Sections 2.1 and 2.4) and in the abstract's Trial registration line that the trial was registered on 2018-04-25, after data collection, and that the registration names women's dietary quality score (20-item FFQ) at 3 months as the sole primary outcome, with weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes.
  2. Either present the registered primary outcome as primary, or say that store sales were designated primary in the paper, when this was decided and why. Also state that the 6-month time point and the frozen-vegetable, checkout-item, loyalty-card and economic outcomes were not in the registration.
  3. Give the registered primary result in the abstract with its estimate, CI and P value: 3 months, +0.29 SD (0.01, 0.57), P = 0.04 adjusted for IMD; +0.26 SD (−0.01, 0.54), P = 0.06 fully adjusted.

Paper says trial was registered prospectively; registry record was first submitted 33 months after study start

Circulation (2021) · doi:10.1161/CIRCULATIONAHA.121.054117 · PMC8849598 · NCT03476785

Summary

The Methods state that "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)." The ClinicalTrials.gov record gives the study start as 2015-06-01 (type ACTUAL). It also shows that the record was first submitted on 2018-03-01 and first posted on 2018-03-26, 33 months (1004 days) after that start, while its status was still "Recruiting". The record does describe this trial: its final enrollment (56, ACTUAL) equals the 56 participants the paper says were randomized. The outcome reported in the paper (the LV stiffness constant at 1 year) matches the registered primary outcome in every registry version, so the outcomes were not switched. However, no version of the record existed before enrollment began, so the registry cannot show that this outcome was fixed in advance. The paper does not mention the late registration, the registry edit of 2019-04-16 that made the primary outcome description more specific and deleted one secondary outcome, or the fact that one registered secondary outcome, pulse wave velocity, is not reported.

What was registered

The ClinicalTrials.gov record history has three versions. None of them is dated on or before the study start.

Version Date (submitted / posted) Status Primary outcome (measure; time frame; description) Secondary outcomes
v0 QC 2018-03-16; first submitted 2018-03-01; posted 2018-03-26 Recruiting; enrollment 53 (estimated) "Left ventricular stiffness"; "1 year"; "Change in end diastolic ventricular pressure and volume curve" "Aerobic fitness": "Change in peak aerobic capacity assessed by VO2"; "Vascular stiffness": "Change in pulse wave velocity"; "Diastolic function": "Change in left ventricular diastolic function measured by echocardiography" (all at "1 year")
v1 2019-04-16 / posted 2019-04-18 (modules changed: Study Status, Outcome Measures, Contacts/Locations) Active, not recruiting "Left ventricular stiffness"; "1 year"; "Change in ventricular stiffness after 1 year of exercise training measured as the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" "Aerobic fitness": "Change in peak aerobic capacity (VO2) after 1 year of exercise training"; "Vascular stiffness": "Change in pulse wave velocity after 1 year of exercise training". The "Diastolic function" outcome was deleted.
v2 (current) 2019-10-10 / posted 2019-10-14 (modules changed: Study Status, Study Design) Completed; enrollment 56 (actual) Same as v1 Same as v1

Key registry dates, all fields of the record: - Study start: 2015-06-01, ACTUAL, in every version. - Primary completion: 2019-07-02, ACTUAL (v2). In v0 and v1 it was given as 2019-06-01, ESTIMATED.

The only outcome edit, v1 on 2019-04-16, came after recruitment had closed and 77 days before primary completion. The primary outcome measure and time frame stayed the same. The new description names a specific quantity, the stiffness constant of the diastolic pressure-volume curve, where v0 described the end-diastolic pressure-volume curve in general.

What the paper reports

The PMC copy is the NIH author manuscript (NIHMS1731529).

Registration statement. Methods, "Participant Population and Study Design", second paragraph:

The trial was registered prospectively on ClinicalTrials.gov (NCT03476785).

The abstract ends with:

Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT03476785

Main hypothesis and endpoint. The paper never uses the words "primary outcome" or "primary endpoint". The abstract, "Background":

In a prospective, randomized-controlled trial, we hypothesized that 1-year prolonged exercise training would reduce LV myocardial stiffness in patients with LVH.

Methods, "Statistical Analysis", which has the only other use of "primary":

The primary analysis included all participants who completed the 1-year follow-up.

Result for the stiffness endpoint. Abstract, "Results":

LV myocardial stiffness was reduced (right and downward shift in the EDPVR; pre LV myocardial stiffness 0.062±0.020 to post 0.031±0.009), whereas there was no significant change in controls (pre 0.061±0.033 to post 0.066±0.031, interaction P=0.001).

Results, "LV pressure-volume relationship":

1-year of exercise training significantly reduced LV chamber and myocardial stiffness constants (LV chamber stiffness: from 0.060±0.031 to 0.042±0.025; LV myocardial stiffness: from 0.062±0.020 to 0.031±0.009), with no significant changes in the control group (LV chamber stiffness: from 0.041±0.016 to 0.049±0.020; LV myocardial stiffness: from 0.061±0.033 to 0.066±0.031; Figure 4A: interaction P value=0.015 and Figure 4B: interaction P value=0.023).

Fitness. Results, "Effect of Exercise Training on Oxygen Uptake and LVEDV", which matches Table 2 (V̇O2 max, mL/min/kg, interaction P 0.0004):

The exercise training program resulted in a significant increase in V̇O2 max in the exercise group (from 26.0±5.3 to 31.3±5.8 mL/min/kg, P<0.0001).

Enrollment. Methods, "Participant Population and Study Design":

Fifty-six participants with LVH and elevated biomarkers were enrolled (signed a consent form) and randomized, comprising 18 controls and 38 exercisers.

The paper gives no recruitment or enrollment dates. Searches of the full text for "2015", "2016", "2017" and "2018" outside the references return nothing. Searches for "pulse wave" and "PWV" also return nothing.

The discrepancy

Outcome Registered (version, date) In the paper Status
LV stiffness at 1 year (primary) v0 2018-03-16: "Change in end diastolic ventricular pressure and volume curve"; v1 2019-04-16 and v2 2019-10-10: "...stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" LV myocardial stiffness constant "a" from the EDPVR (transmural pressure), pre vs 1 year; the main hypothesis and the headline result matched
Same registered primary (alternative reading) as above LV chamber stiffness constant (PCWP-based EDPVR), pre vs 1 year matched (the registry wording also fits this measure)
Aerobic fitness, peak VO2, 1 year (secondary) v0, v1, v2 V̇O2max, pre vs 1 year (Table 2, Figure 2) matched
Vascular stiffness, pulse wave velocity, 1 year (secondary) v0, v1, v2 Not in the main text, tables or figure legends not reported
Diastolic function by echocardiography, 1 year (secondary) v0 only; deleted in v1 (2019-04-16) Not reported. 3D echo is used only for LVEDV. not reported (removed from the registry before primary completion)
LVEDV, Starling curves, preload-recruitable stroke work Not registered Reported (Figures 3, 5, 6), not labelled primary added
Timing of registration (not an outcome) First submitted 2018-03-01; study start 2015-06-01 (ACTUAL) "registered prospectively" contradicted by the registry; not disclosed

Figures

Timeline of NCT03476785: study start, first registration, registry versions, primary completion and publication

Figure 1. Trial timeline from the ClinicalTrials.gov record history and the paper. The study start (2015-06-01, ACTUAL) comes 1004 days before the record was first submitted (2018-03-01; posted 2018-03-26, status "Recruiting"). No registry record exists for the shaded grey interval. The only version that changed the primary outcome text (v1, 2019-04-16, bold) is dated after the study start and after recruitment closed, and 77 days before primary completion (2019-07-02, ACTUAL). Version 2 (2019-10-10, open marker) changed only the status and enrollment fields. The paper's epub (2021-09-20) describes the registration as prospective.

Registered vs reported outcomes, with connectors labelled matched, not reported, dropped and added

Figure 2. Registered outcomes by registry version (left, blue) against outcomes reported in the paper (right, orange). The label on each connector names the relation. Both wordings of the registered primary outcome, LV stiffness at 1 year, correspond to the stiffness constants the paper reports. The myocardial constant is the paper's headline result and the chamber constant also fits the registry wording. The paper's result for the registered primary outcome is shown in the diagram (exercise 0.062 → 0.031; control 0.061 → 0.066). Its group × time interaction P value is 0.001 in the Abstract and 0.023 in the Results and Figure 4B. Of the registered secondary outcomes, peak VO2 is reported, pulse wave velocity is not, and echocardiographic diastolic function was deleted from the registry in v1 and is not reported. The bottom strip states the registration-timing discrepancy, which is the main finding.

Why it matters

The paper reports the registered primary outcome, and the result is positive. In completers (exercise n=20, control n=11), the LV myocardial stiffness constant fell from 0.062±0.020 to 0.031±0.009 with exercise and changed from 0.061±0.033 to 0.066±0.031 in controls. The LV chamber stiffness constant fell from 0.060±0.031 to 0.042±0.025 with exercise and changed from 0.041±0.016 to 0.049±0.020 in controls (interaction P=0.015). Nothing here suggests that a different primary outcome was substituted.

The problem is what the word "prospectively" tells a reader. Under the ICMJE recommendations, a trial counts as prospectively registered only if it was registered at or before the first participant's enrollment. Readers and systematic reviewers use that statement to decide whether the registry entry can be taken as evidence that the primary outcome, time point and analysis were fixed before any outcome data existed. Here the registry gives a study start of 2015-06-01, and its first record dates from March 2018. Primary completion was 2019-07-02 and each participant was followed for one year, so the last participant must have started the intervention by about July 2018. Of the roughly 37-month window in which participants could have been enrolled, at least 33 months came before the record existed. v0 still listed the trial as "Recruiting", so some participants may have been enrolled after registration. The paper gives no enrollment dates, so the exact split cannot be determined.

Three further points bear on how the registry can be used to check this paper: - The primary-outcome edit. The v1 change of 2019-04-16 was made after recruitment closed and before primary completion. By then most participants' 1-year data had probably been collected. The edit made the outcome more specific without changing the construct, and its new wording matches what the paper reports. It is still a post-recruitment change to the registered primary outcome, and the paper does not mention it. - Ambiguous wording. The registry says "ventricular stiffness ... stiffness constant", which could mean the chamber constant (PCWP-based) or the myocardial constant (transmural-pressure-based). The paper features the myocardial constant. Both constants show a significant interaction, so the conclusion does not depend on which one was intended. - Inconsistent P values. The abstract gives the interaction P for myocardial stiffness as 0.001. The Results and Figure 4B give 0.023 for what reads as the same comparison. Both are below 0.05, but a reader cannot tell which is correct.

The registered secondary outcome pulse wave velocity is not reported anywhere in the main text. The registered secondary outcome echocardiographic diastolic function was removed from the registry in 2019 and is also not reported. The paper explains neither omission.

Explanations considered

Status

Checked as of 2026-10-23. - Registry history. All three versions of NCT03476785 (v0 2018-03-16, v1 2019-04-16, v2 2019-10-10) were read from the ClinicalTrials.gov history API (/api/int/studies/NCT03476785/history/{0,1,2}), retrieved 2026-10-22/23. The current record (/api/v2/studies/NCT03476785) is identical to v2. On 2026-10-23 the history endpoint returned HTTP 403 when rechecked. The first-submitted date (2018-03-01) and study start (2015-06-01, ACTUAL) also appear in the current record, so the central finding does not depend on the history endpoint. - Publisher version. The publisher page (ahajournals.org) returned HTTP 403, and a mirrored PDF on a university site refused the connection. The PMC text is the NIH author manuscript. The publisher-supplied PubMed abstract (PMID 34543068) has the same outcome values and the same abstract P value (interaction P=0.001), and ends "Registration: URL: https://www.clinicaltrials.gov; Unique identifier: NCT03476785." The Methods sentence "registered prospectively" could not be checked against the version of record. The online Data Supplement is not in PMC and was not checked. It could contain pulse wave velocity data, but the main text does not refer to any. - Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction (checked with src/known.py). A Letter to the Editor (doi:10.1161/CIRCULATIONAHA.121.058145, PMC8966638) raises left atrial size, atrial fibrillation, baseline BMI and follow-up length, not registration. The authors' published response was not read. - PubPeer. The PubPeer API was blocked by Cloudflare (HTTP 403). A web search found no PubPeer thread for the DOI or title. The absence of comments is therefore not confirmed directly. - Sources. The paper: PMC8849598, via src/show.py. The registry: ClinicalTrials.gov NCT03476785, current record and history versions 0 to 2. PubMed 34543068. Crossref, PMC and Europe PMC notices.

Suggested correction

  1. Replace "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)" with a statement such as: "The trial was registered retrospectively on ClinicalTrials.gov (NCT03476785; first submitted 1 March 2018), after enrollment began in June 2015." Give the reason for the late registration and, if known, how many participants were enrolled before it.
  2. State that the registered description of the primary outcome was revised on 16 April 2019, after recruitment closed and before primary completion, to "the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve". Also state that the secondary outcome of echocardiographic diastolic function was removed at the same time.
  3. Specify which stiffness constant, myocardial (transmural pressure) or chamber (PCWP), is the pre-specified primary outcome. Reconcile the interaction P value for LV myocardial stiffness: 0.001 in the Abstract against 0.023 in the Results and Figure 4B.
  4. Report the registered secondary outcome, pulse wave velocity at 1 year, or explain why it is not reported.

Trial report omits one of two registered primary outcomes and does not disclose the omission

Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101339 · PMC12795688 · NCT04316221

Summary

ClinicalTrials.gov NCT04316221 has listed two primary outcomes since its first version (2020-03-18): the Caregiver Reported Child Development Instruments (CREDI) and the Guide for Monitoring Child Development (GMCD), both with a 6-month time frame. The Outcome Measures section was never edited in any of the record's six versions. The published protocol lists both instruments as primary in its abstract and data-collection schedule (Figure 7), although the defining sentence of its Primary Outcomes subsection names only the CREDI. The paper, which presents itself as the trial's primary report, names a single primary outcome (the CREDI global development score). It does not mention the GMCD anywhere, and it does not say that a registered primary outcome was dropped or why. The reported CREDI result is null. No GMCD result is reported.

What was registered

The record history (ClinicalTrials.gov history API, versions 0 to 5; the web history tab numbers the same versions 1 to 6) shows these versions and changed modules:

Version (API) Date Modules changed Primary outcomes
0 2020-03-18 first version (submitted 2020-03-16, posted 2020-03-20) CREDI; GMCD
1 2021-01-29 Study Status (estimated start moved from 2020-06-01 to 2021-04-01) unchanged
2 2021-07-07 Study Status (start 2021-04-01 marked actual; recruiting), Contacts/Locations unchanged
3 2023-01-30 Study Status unchanged
4 2023-05-19 Study Status, Study Design (enrollment changed to actual), Contacts/Locations unchanged
5 2026-03-12 Study Status unchanged

The history summary reports "primaryOutcomesSame": true, "secondaryOutcomesSame": true and "outcomesUpdateCount": 0. The full records for versions 0, 1, 2, 4 and 5 were retrieved, and their primary-outcome entries are identical.

Guide for Monitoring Child Development (GMCD) · time frame: 6 months · "The GMCD has been used in low- and middle-income countries in children ages 0-24 months and focuses on seven areas: expressive language, receptive language, fine motor, gross motor, relating, play, and self-help." - Version 4 (2023-05-19), the last version before primary completion (2023-09-30) and before online publication (2025-12-20): the same two primary outcomes, with identical wording. - Version 5 (2026-03-12, after publication; current record): the same two primary outcomes. The record has no posted results (hasResults: false).

The registered secondary outcomes, all at 6 months, are "changes in infant hemoglobin", "weight-for-age", "length-for-age", "Head circumference" and "Infant diet quality" (WHO Infant and Young Child Feeding indicators).

Published protocol (J Acad Nutr Diet 2022;122:432–444, doi:10.1016/j.jand.2021.03.015, online 2021-05-04; also the paper's supplement mmc2.pdf). The protocol is internally inconsistent about the GMCD. Its abstract lists both instruments as primary:

"Primary outcomes for this study are changes in global development scores, assessed using the Guide for Monitoring Global Development and the Caregiver Reported Child Development Instruments."

The body's "Primary Outcomes" subsection, however, opens with a defining sentence that names only the CREDI:

"Primary outcomes for this study are differences in global development scores between study arms, assessed using the Caregiver Reported Early Child Development Instruments (CREDI) long form at baseline (ie, infants aged 6 to 9 months) and endline."

The same subsection then describes the GMCD: "The Guide for Monitoring Child Development is extensively used alongside the CREDI in low- and middle-income countries in children aged 0 to 24 months", and "Together the Guide for Monitoring Child Development and the CREDI ... provide complementary information about child development." Figure 7 of the protocol ("Data collection description and schedule") lists the GMCD under "Primary outcome" and schedules it at endline in both groups. Under "Protocol Amendments" the protocol says: "Any modification or addition will be communicated by updating the information on the www.clinicaltrials.gov website."

Because the protocol can be read either way, the main benchmark here is the registry, which the protocol itself names as the place where changes are recorded and which lists the CREDI and the GMCD as primary outcomes in all six versions.

What the paper reports

The paper states a single primary outcome:

"The primary outcome was the overall global development score from the Caregiver Reported Early Childhood Development Instruments (CREDI) long form, which has been validated in Guatemala and piloted previously by our team."

— Methods, "Sociodemographic variables and outcomes", paragraph 2

"We hypothesized that infants receiving a single whole egg per day and local standard nutrition care during the complementary feeding period would have improved global development (primary outcome) and growth, anemia status, and diet quality (secondary outcomes) relative to infants who received only local standard nutrition care."

— Introduction, final paragraph, which opens "We report primary findings from the Saqmolo’ Project"

"Mixed linear or logistic regression models were used to estimate between-group differences in primary (global development score) and secondary (growth, anemia status, and diet quality) outcomes."

— Abstract, Methods

Its statements about the registration, protocol and analysis plan are:

"This trial was preregistered with ClinicalTrials.gov (NCT04316221). A detailed trial protocol was published previously."

— Methods, first paragraph, which also says "The study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline."

"A detailed statistical analysis plan was made publicly available prior to data analysis."

— Methods, "Statistical analysis", first sentence (ref 51)

The archived PMC text, including the tables, figure legends and back matter, has no match for "GMCD" or "Guide for Monitoring". The supplementary tables (mmc1.docx) do not mention it either. The text contains no "amend" and no "deviat". The only departure from plan the paper describes concerns adherence monitoring: "More intensive weekly monitoring of self-reported egg consumption by telephone was planned but proved infeasible due to geographical limitations in cellular signal coverage."

The cited analysis plan (Harvard Dataverse doi:10.7910/DVN/1OSZOC, file "Saqmolo data analysis plan 1.19.2024.pdf", published 2024-01-23, after primary completion on 2023-09-30) says: "The primary endpoint for this study is differences between groups in child global development scores, as assessed using the Caregiver Reported Child Development Instruments (CREDI)." It does not mention the GMCD, and it does not say that the primary outcomes were changed from the registration or the protocol.

The discrepancy

Outcome Registered (version, date) In the paper Status
CREDI long form, global development score, 6 months Primary; v0 2020-03-18 to v5 2026-03-12, unchanged Sole primary outcome. Global raw scaled score at endline (6-month intervention), with domain scores as sensitivity analyses matched
Guide for Monitoring Child Development (GMCD), 6 months Primary; v0 2020-03-18 to v5 2026-03-12, unchanged. The published protocol (2021) lists it as primary in its abstract and data-collection schedule (Figure 7); its body's defining sentence names only the CREDI Not mentioned anywhere: not as a primary, secondary or exploratory outcome, and not in the limitations not reported
Infant hemoglobin, 6 months Secondary; v0 to v5 Secondary: altitude-adjusted hemoglobin and anemia matched
Weight-for-age, length-for-age, head circumference, 6 months Secondary; v0 to v5 Secondary: z scores plus underweight, stunting and low head circumference. Weight-for-length z, which the protocol lists, is also reported matched
Infant diet quality (WHO IYCF indicators), 6 months Secondary; v0 to v5 Secondary: minimum dietary diversity, minimum meal frequency, minimum adequate diet matched

The randomized sample (1200) matches the registered enrollment (1200, actual).

Figures

Timeline of NCT04316221 registry versions, study start, primary completion, protocol, analysis plan and publication

Figure 1. Timeline of the registration and publications. The upper lane shows the six ClinicalTrials.gov versions (blue; circles are versions before the registered study start, squares are versions after it). The darker band runs from the registered study start (2021-04-01) to primary completion (2023-09-30), and the lighter band covers the period after primary completion. No version changed the primary outcomes, so the figure has no outcome-change markers: every version lists the CREDI and the GMCD. The lower lane shows the registered start and primary-completion dates, the trial dates given in the paper (orange bar, 2021-05-28 to 2023-08-03), the published protocol (2021-05-04; CREDI and GMCD as primary in its abstract and Figure 7), the analysis plan deposited after primary completion (2024-01-23, CREDI only) and the paper (2025-12-20, CREDI only). The one version dated after primary completion, v5 (2026-03-12), also follows the paper's publication and still lists both primary outcomes.

Two-column diagram: registered primary outcomes CREDI and GMCD on the left, the paper's single CREDI primary outcome with its result on the right

Figure 2. Registered primary outcomes and the primary outcome reported. Left (blue): the two primary outcomes registered in v0 (2020-03-18), unchanged through v5 (2026-03-12). Right (orange): the primary outcome the paper reports, with its adjusted result (β −0.08 points, 95% CI −0.22 to 0.06, p = 0.26, Table 4). The CREDI connector is labelled "MATCHED" (same instrument, same 6-month time frame). The GMCD connector, labelled "NOT REPORTED", ends at a dashed box stating that the paper names the GMCD nowhere in the text, tables or supplementary tables and gives no reason. The note at the bottom summarises the protocol (both instruments primary in its abstract and Figure 7, only the CREDI in its body's defining sentence) and the analysis plan, which lists only the CREDI and states no change.

Why it matters

Explanations considered

  1. The paper is a secondary or partial report. This does not apply. The paper says "We report primary findings from the Saqmolo’ Project". It reports the registered secondary outcomes alongside the primary outcome and does not point to a separate report. A Europe PMC search for "Saqmolo" (2026-10-23) returns only this paper, the protocol (J Acad Nutr Diet 2022), a protocol abstract (Curr Dev Nutr 2021) and an egg-composition study (Food Sci Nutr 2023). None of them reports GMCD results.
  2. The change is disclosed. It is not. The paper's only statements about the registration and plans are that the trial "was preregistered", that a protocol "was published previously" and that an analysis plan "was made publicly available prior to data analysis". None of them mentions the GMCD or a change in primary outcomes. The limitations section does not address it either.
  3. A documented protocol amendment explains it. None was found. The registry's Outcome Measures module was never edited in six versions (2020-03-18 to 2026-03-12), even though the protocol commits to communicating outcome changes "by updating the information on the www.clinicaltrials.gov website". The analysis plan (version 1.19.2024) lists only the CREDI, so a narrowing appears there. But the plan does not say that it departs from the registered or protocol primary outcomes, gives no reason, and is dated after primary completion. Citing the plan therefore does not, by itself, disclose the change to the reader. Whether the GMCD was administered at all is not stated in any of these documents.
  4. The two names refer to the same outcome. They do not. The CREDI and the GMCD are separate instruments. The registry lists them as two primary outcomes with different descriptions (caregiver-report CREDI with five domains; GMCD with seven areas), and the protocol cites them separately and schedules them separately (Figure 7). The protocol abstract's "Guide for Monitoring Global Development" is a naming variant of the same GMCD: the protocol's Outcomes section and Figure 7 call it "Guide for Monitoring Child Development". The paper's single primary outcome is explicitly the CREDI score.
  5. Time-point or metric differences. These are not at issue. The CREDI is analysed at the 6-month endline, consistent with the registered time frame. The difference concerns which outcomes are reported, not when they were measured.
  6. A later correction resolves it. None was found (see Status).
  7. The protocol already made the CREDI the sole primary outcome. Partly. The defining sentence of the protocol's Primary Outcomes subsection names only the CREDI, while its abstract and Figure 7 list the GMCD as primary as well, so the protocol is ambiguous. The registry, which the protocol designates for recording any modification, lists both as primary in every version, and the paper cites the registration. The protocol's ambiguity therefore does not disclose to the paper's readers that a registered primary outcome is not reported.

The documents do not show why the GMCD is absent, and this write-up makes no inference about the reason. A change of outcomes can have legitimate reasons, such as feasibility, instrument availability or field constraints. The issue documented here is that the omission is not disclosed.

Status

As of 2026-10-23:

Suggested correction

A correction to the article that:

  1. states that ClinicalTrials.gov NCT04316221 (and the abstract and Figure 7 of the published protocol) listed two primary outcomes at 6 months, the CREDI and the Guide for Monitoring Child Development (GMCD), and that the article reports only the CREDI;
  2. says whether GMCD data were collected and, if not, when and why the decision was made;
  3. if the GMCD was collected, reports the between-group result with the same models used for the CREDI (or a reason it cannot be reported), and says when the decision to make the CREDI the sole primary outcome was taken relative to the analysis plan (version 1.19.2024) and to the start of outcome analysis, as CONSORT item 6b asks for any change to trial outcomes;
  4. qualifies the abstract's conclusion that the intervention "did not benefit development" as resting on the caregiver-reported CREDI only.

Separately, the ClinicalTrials.gov record could be updated to reflect the outcomes actually analysed, with the change and its reason noted, as the protocol's amendment section anticipates.

HIV care trial report redefines registered retention outcome and omits hair-sample adherence without disclosure

Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100307 · PMC9536254 · NCT02846350

Summary

NCT02846350 (COPA2) registered five primary outcomes on 2016-07-22, before enrolment began in November 2016. No later registry version changed them. The paper is the trial's main outcomes report. It discloses and explains two departures: viral load was analysed as a 0.50-log reduction instead of suppression, and medication persistence was not presented. It does not disclose two others. Retention in care is reported as whether the participant completed the study visit at each time point, whereas the registry and the published protocol define it as the number of missed clinic appointments in the previous 6 months. The registered primary outcome "Medication adherence: Hair sample" is not reported and is never mentioned. All the randomised comparisons that are reported are null, and the paper says so in its abstract. Its positive interpretation rests on indirect (mediation) effects that it labels exploratory.

What was registered

ClinicalTrials.gov NCT02846350 was first submitted on 2016-07-01. Version 0 passed QC on 2016-07-22 with status "Not yet recruiting" and was first posted on 2016-07-27. The registry gives the study start as 2016-07 (actual) and primary completion as 2020-11-30 (actual). The paper says baseline visits ran from November 2016 to March 2018.

Version 0 lists five primary outcomes and no secondary outcomes. The wording is quoted from the record, with escape characters removed:

# Measure Description Time frame
1 "HIV viral Load (VL)" "VL will be assessed by blood sampling. Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome; a secondary dichotomization will be made among detectable patients with VL <1000 and those with VL >1000." "Less than 6 months detectable VL"
2 "Retention in care" "Defined as the number of missed clinic appointments (i.e., "no shows" not cancelled in advance by patients or clinic staff) in the 6 months preceding assessment." "2 years"
3 "Medication persistence" "Medication persistence is the time to treatment discontinuation, with a permissible gap of < 30 days, participants will be considered to have discontinued the regimen if a therapy gap = > 30 days occurs." "2 years"
4 "Medication adherence: Hair sample" "Adherence by hair sample to estimate an average concentration of ARV medication exposure" "2 years"
5 "Medication adherence: Self-report" "Proportion of self-reported adherent participants over time" "2 years"

The record history has 13 versions, v0 (2016-07-22) to v12 (2022-06-22). None of them edits the Outcome Measures module. The history summary gives outcomesUpdateCount = 0, lastUpdateVersions.primaryOutcomes = 0 and originalData.primaryOutcomesSame = true. Version 12 was submitted the day before the paper went online (2022-06-23). Its five primary outcomes match version 0 word for word, and so does the current record (last update posted 2022-06-29). The later versions change only Study Status, Contacts/Locations, Sponsor/Collaborators, Study Design, Study Identification, Oversight, Conditions and the Document Section.

The published protocol (Trials 2018, doi:10.1186/s13063-018-2758-5, PMC6056946; reference 17 of the paper) defines the outcomes the same way. On retention:

"In this study, retention in care will be defined as the number of missed clinic appointments (i.e., "no shows" not canceled in advance by patients or clinic staff) in the 6 months preceding assessment, including that timepoint's visit with the COPA2 physician. The primary retention analysis will utilize a dichotomized indicator 0/1 of any missed visits vs. no missed visits"

On hair samples and self-report:

"Participants will provide two hair samples, at the 12-month and 24-month assessments."

"Hair sample will estimate an average concentration of ARV medication exposure over 6 weeks with cut-off values used as categorical variables (12 months and 24 months)"

"Self-report will reflect percentage adherent over the last 30 days (Visual Analog Scale; VAS)"

The protocol's analysis plan (Hypothesis 1.2) treats self-reported adherence as continuous. It "will use a linear mixed model ... to examine percentage adherence in the past 30 days as a continuous outcome".

What the paper reports

The paper states its objective (Methods, "Objectives and design"):

"The primary objective of the study was to evaluate the impact of a physician-delivered, MI patient-oriented treatment program on ART adherence, HIV viral load, retention in care, and self-efficacy among PWH disengaged from HIV care."

It cites the registration in the same paragraph ("was registered at clinicaltrials.gov, NCT02846350") and in the abstract. Its only statement about departures from plan is in Methods, "Measures of main study outcomes":

"Per the original protocol, ART adherence, HIV viral suppression, retention in care, and medication persistence were the primary outcomes of the study. At the end of the data collection period, changes to how HIV viral load was evaluated as a primary outcome had to be made given that viral suppression at baseline was a constant (see description below). Self-efficacy and patient-provider communication were the MI physician targets in the original protocol, as well as the theorised mechanisms by which the intervention was intended to work. Contrary to the study protocol, in the current study, we do not present medication persistence given issues related to pharmacy pickup data medical abstraction."

On viral load (Methods, "HIV RNA Viral Load"):

"For analyses, condition differences were analysed as a clinically significant reduction in HIV viral load from one-time point to the next, 0·50log. Though we originally intended to analyse HIV RNA viral load as the proportion of participants who were virally suppressed in each condition, the complete absence of participants who were virally suppressed at baseline precluded such analyses; that is, viral suppression at baseline was a constant at baseline."

The Discussion repeats that this "precluded the analysis of viral suppression as an outcome, as originally proposed in the protocol."

On retention, the definition is given in Methods, "Retention in HIV Care", and repeated in Results, "Retention":

"Retention in HIV care was calculated as to whether a participant completed a study clinical visit at any given time point."

"Retention was evaluated at 6 months, 12 months, 18, and 24 months as to whether the participant completed a follow-up visit."

On self-reported adherence (Methods, "Self-reported antiretroviral adherence"):

"Self-reported adherence was assessed every six months at all time points using a 10-point Visual Analogue Scale (VAS), with 0 representing none and 10 representing 100% (perfect adherence)."

The analysis covers five outcomes (Methods, "Data analyses"):

"To test the effect of the intervention on ART adherence, HIV viral load, CD4+ count, retention, and self-efficacy, mixed modelling was used"

"As such, indirect effects were examined as exploratory analyses using bivariate parallel latent growth curve (LGC) models"

A full-text search of the PMC version finds no occurrence of "hair". Hair-sample adherence is not mentioned in the text, the tables or the figure legends. No passage explains the retention definition or relates it to the registered count of missed clinic appointments. The only supplementary item listed is "Supplementary materials Image, application 1".

The discrepancy

Outcome Registered (version, date) In the paper Status
HIV viral load Primary. "Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome" (v0, 2016-07-22; unchanged to v12, 2022-06-22) Continuous viral load, with condition differences as a "clinically significant reduction ... 0·50log". Condition p = 0.757 Not reported as registered. The change is disclosed, with a reason
Retention in care Primary. "the number of missed clinic appointments ... in the 6 months preceding assessment" (v0, 2016-07-22; unchanged to v12) "whether a participant completed a study clinical visit at any given time point". Condition p = 0.999 Not reported as registered. The measure is redefined and the change is not disclosed
Medication adherence: hair sample Primary. "Adherence by hair sample to estimate an average concentration of ARV medication exposure" (v0; unchanged to v12) Absent: no result and no mention Not reported, not disclosed
Medication adherence: self-report Primary. "Proportion of self-reported adherent participants over time" (v0; unchanged to v12) Mean VAS score (0–10) in a mixed model. Condition p = 0.075 Matches the published protocol (continuous VAS). Differs from the registry wording only (minor)
Medication persistence Primary. "time to treatment discontinuation ... gap = > 30 days" (v0; unchanged to v12) Not presented ("Contrary to the study protocol ...") Not reported. Disclosed, with a reason
CD4+ count Not registered (the registry lists no secondary outcomes) Analysed alongside the main outcomes. Condition p = 0.584 Added
Self-efficacy (HIV-ASES) Not registered. The protocol describes it as an MI patient target or mediator Named in the "primary objective" and analysed as a main outcome. Condition p = 0.096 Added. The paper notes its role in the original protocol

The registry time frame for viral load, "Less than 6 months detectable VL", is unclear. The paper assesses viral load at 6, 12, 18 and 24 months, which matches the protocol, so no time-point change is claimed here.

Figures

Timeline of registration, trial conduct and publication for NCT02846350

Figure 1. Timeline of NCT02846350. The top lane shows the 13 registry versions. Only v0 (2016-07-22, filled marker) set the primary outcomes. Hollow markers are versions that left the Outcome Measures module untouched, including the three versions (v10 to v12) after primary completion on 2020-11-30 (dashed line; shaded area after it). The middle lane shows the registry study start (2016-07) and the baseline and follow-up visit periods given in the paper (November 2016 to March 2018, follow-up to May 2020). The bottom lane shows the published protocol (2018-07-24) and the paper (online 2022-06-23, issue September 2022). The registered outcomes were in place before enrolment and never changed, so the differences in the paper did not come from a registry update. Sources: the ClinicalTrials.gov history endpoint, the PMC article and Europe PMC.

Registered primary outcomes versus outcomes reported in the paper

Figure 2. The five registered primary outcomes (left, blue; v0 wording, identical through v12) and the outcomes reported in the paper (right, orange). Each connector is labelled with its status. Solid connectors join a registered outcome to the measure the paper reports in its place. Dashed connectors ending in "×" mark registered outcomes with no result in the paper. Two departures are disclosed in the paper: viral load and persistence. Two are not: retention, which is redefined, and hair-sample adherence, which is absent. The self-report adherence metric matches the published protocol and differs only from the registry wording. CD4+ count and self-efficacy are added. The p values in the orange boxes are the paper's longitudinal mixed-model tests for the main effect of condition and for time × condition. The footer gives the paper's indirect and direct LGC effects. All values are quoted from the Results section.

Why it matters

Results for the registered primary outcomes. For every registered primary outcome that the paper reports, the randomised comparison is null:

The abstract states these null findings plainly: "Using mixed models, the intervention had no effect on ART adherence over time by condition on HIV RNA viral load, CD4+ count retention, or self-efficacy." So the departures did not turn a null primary result into a positive one.

Where the positive conclusion comes from. The abstract's Interpretation reads: "These findings suggest that physician-delivered MI may enhance the patient-provider relationship, self-efficacy, and ART adherence, and reduced HIV viral load in patients disengaged from HIV care." This rests on indirect effects through the patient–provider relationship: adherence B = 0·188, p = 0·009; viral load B = -0·095, p = 0·027; self-efficacy B = 0·063, p = 0·001. The paper calls these "exploratory analyses". The published protocol did plan exploratory mediation analyses, with "Patient-level skills (self-efficacy, motivation, patient-provider relationship) as potential mediators". Those were planned for retention, adherence and viral suppression, though, not for self-efficacy as an outcome. None of these analyses appears in the registry.

Retention. The registered measure counts no-shows at clinic appointments over the 6 months before each assessment; the protocol dichotomises it as any versus none. The reported measure records whether the participant attended the study visit itself. Study visits were "combined with regularly occurring HIV clinical care visits" and participants "were compensated with US $10 per study visit for travel". As a result, the reported measure largely tracks completion of study follow-up, which the paper also reports as attrition (for example, 25·6% did not complete the 12-month assessment). It cannot show missed clinical appointments between study visits, and those are the behaviour a re-engagement intervention targets. The Discussion itself suggests the null retention result "may have been related to the fact that participants were recruited and incentivized by study staff, given that study and clinical visits were often scheduled on the same day". A reader comparing the paper with the registration or the protocol is not told that the measure differs.

Hair-sample adherence. This was the only objective (drug-level) adherence measure among the registered primary outcomes. The protocol scheduled collection at 12 and 24 months. The paper's adherence findings, including the indirect effect on adherence, rest entirely on self-report. A reader cannot tell from the paper whether hair samples were collected, analysed or are still unreported. Leaving out a registered objective measure without comment is the kind of selective outcome reporting that registration exists to reveal.

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The journal and authors may wish to consider a correction or clarification notice that does the following:

  1. States that retention was registered, and specified in the protocol, as the number of missed clinic appointments in the 6 months before each assessment (analysed as any versus none). It should say that the paper instead reports whether the study visit was completed, and why. If clinic appointment records are available, it should report the registered retention measure.
  2. States whether hair samples were collected at 12 and 24 months as planned, and whether they were analysed. It should either report the registered hair-sample adherence outcome or explain why it is not reported. If the outcome is reported elsewhere, it should cite that report.
  3. Lists the registered primary outcomes in full: five, with the two adherence measures shown separately. Against each it should give the measure actually reported or the reason none is reported, and it should mark CD4+ count and self-efficacy as outcomes not registered as primary.
  4. Optionally notes that the registry's "proportion of self-reported adherent participants" was operationalised as the protocol's continuous VAS measure.

The trial team could also post results to ClinicalTrials.gov for the registered outcomes, and update the registry entry with any outcome changes and their dates.

Trial registered five months into enrollment and safety primary added later; paper says "prospectively registered"

Circulation (2017) · doi:10.1161/CIRCULATIONAHA.116.026493 · PMC5459585 · NCT01858363

Summary

The paper is the primary 12-month report of a randomized trial comparing a low-dose paclitaxel-coated balloon (DCB) with uncoated angioplasty (PTA). It states that "The study was prospectively registered". The ClinicalTrials.gov record was first submitted on 2013-05-08, but the paper reports that randomization began in December 2012 and the registry gives the study start as 2012-11. In the first registered version the only primary outcome was 12-month primary patency. The primary safety end point, one of the paper's two primary end points, was added as a registered primary outcome on 2014-02-12, about 14 months after randomization began and before primary completion. Both primary end points reported in the paper match the registration in force at publication, and neither the late registration nor the added primary outcome is mentioned in the paper.

What was registered

Five registry versions exist. They are numbered 1-5 here, in date order (indices 0-4 in the ClinicalTrials.gov history API). Every version gives the study start as "2012-11", and every version gives the first submission date as 2013-05-08.

Version 1, 2013-05-20 (first submitted 2013-05-08, first posted 2013-05-21; status Recruiting; estimated enrollment 360). This is the earliest version, and no version exists on or before the study start. It has one primary outcome:

Primary outcome: "Primary Patency at 12 months post-procedure" (time frame: "12 months")

Secondary outcome: "Major adverse event rate" (time frame: "1, 6, 12, 24 months")

No safety outcome is registered as primary.

Version 2, 2014-02-12 (status Recruiting). The module labels for this version include "Outcome Measures". It has two primary outcomes and no secondary outcomes:

"Primary Efficacy Endpoint" (time frame "12 months"); description: "Primary Patency at 12 months post-procedure"

"Primary Safety Endpoint" (time frame "30-Days and 12-Months"); description: "Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"

The secondary outcome "Major adverse event rate" was deleted in this version.

Version 3, 2015-06-25 and version 4, 2016-10-05. The outcomes are unchanged from version 2. Version 4, which changed only the Study Status and IPDSharing modules, records primary completion as "2016-03" (actual). It is the last version before the paper was received (2016-11-21) and published online (2017-04-19).

Version 5, 2021-10-11 (results first posted 2021-10-13; after publication). The primary outcomes were reworded for results reporting. The measures and time points did not change; the patency description adds the definition used in the paper (PSVR ≤2.5 and freedom from CD-TLR), which versions 1-4 did not spell out:

"Patency at 12-months" (12 months): "The primary efficacy endpoint was patency at 12 months post-procedure defined as the absence of target lesion restenosis determined by duplex ultrasound peak systolic velocity ratio (PSVR) ≤2.5 and freedom from clinically-driven target lesion revascularization (CD-TLR)."

"Percentage of Patients With Freedom From Device and Procedure Related Target Limb Major Amputation or Death Through 30-days Post-procedure and Clinically-driven Target Lesion Revascularization at 12-month" (30-Days and 12-Months)

The posted results (83.9% vs 60.6% for patency; 94.1% vs 83.3% for safety) agree with the paper. Actual enrollment is given as 294.

What the paper reports

The primary end points, from the Abstract (Methods):

"The primary safety end point was a composite of freedom from device- and procedure-related death through 30 days after the procedure and freedom from target limb major amputation and clinically driven target lesion revascularization through 12 months after the procedure. The primary effectiveness end point was primary patency at 12 months."

And from Methods > Outcomes:

"The primary effectiveness end point was primary patency (per lesion) through 12 months, defined as the absence of target lesion restenosis (duplex ultrasound–determined peak systolic velocity ratio ≤2.5) and freedom from CD-TLR."

The registration statement (Methods > Study Design, para 1):

"The study was prospectively registered (http://www.clinicaltrials.gov; unique identifier: NCT01858363)."

The enrollment period (Results > Patient Enrollment and Follow-Up, para 1):

"Between December 2012 and April 2015, 294 patients were randomized to DCB (222 patients, 254 lesions) or PTA (72 patients, 79 lesions) at 18 centers in Germany and Austria"

The paper's statements about prespecification are in Methods > Statistical Analysis. Para 1 sets out the two primary hypotheses (noninferiority with a 5% margin, then superiority, for safety; superiority for patency) and says of the safety hypothesis:

"A priori, if noninferiority was met and the lower 95% confidence limit was >0%, superiority would be claimed."

Para 2 says:

"All analyses were prespecified in a statistical analysis plan."

Neither statement gives a date, or says how the primary safety hypothesis relates to the registry entry of 2014-02-12 that first listed the safety composite as a primary outcome. No protocol amendment, no change to the primary end points and no registration date is mentioned anywhere in the text. The word "protocol" appears only in a sentence about the antiplatelet regimen (Methods > Study Device and Procedure). The Limitations paragraph does not mention registration.

The discrepancy

Outcome Registered (version, date) In the paper Status
12-month primary patency Primary in v1 (2013-05-20) as "Primary Patency at 12 months post-procedure"; primary "Primary Efficacy Endpoint" in v2-v4 (2014-02-12 to 2016-10-05) Primary effectiveness end point: primary patency through 12 months (PSVR ≤2.5 and freedom from CD-TLR) matched
Safety composite (device/procedure-related death through 30 days; target limb major amputation and CD-TLR through 12 months) Absent from v1. Primary "Primary Safety Endpoint" from v2 (2014-02-12), during enrollment, before primary completion Primary safety end point added (in v2, about 14 months after randomization began; matches v2-v4; change not disclosed)
Major adverse event rate Secondary in v1 only (1, 6, 12, 24 months); deleted in v2 (2014-02-12) Reported as "Additional outcomes" (MAEs), not as a primary end point not a primary outcome in either source; no primary-outcome status change
Registration timing First submitted 2013-05-08 (posted 2013-05-21); study start 2012-11 in every version "prospectively registered"; randomization "Between December 2012 and April 2015" not prospective: registered roughly 5 months after the first randomization (about 6 months after the registry start date); not disclosed

The safety composite has a minor wording difference. In the registry wording for v2-v4 ("Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"), the 30-day window could be read as applying to amputation as well as death. The paper puts amputation in the 12-month window. The version 5 description ("freedom from device and procedure-related death through 30 days post-procedure and freedom from target limb major amputation and clinically-driven target lesion revascularization (CD-TLR) through 12 months") matches the paper exactly. The wording also gives the same measure and time points, so this is classed as matched.

Figures

Timeline of registration, registry changes, enrollment and publication for NCT01858363

Figure 1. Timeline of NCT01858363 and the paper. Blue squares are registry versions: filled squares mark versions that changed the primary outcomes, and open squares mark versions that did not. Grey diamonds mark the registry study start (2012-11) and primary completion (2016-03). The orange bar is the randomization period reported in the paper (December 2012 to April 2015). Its hatched part is the stretch before the record was first submitted (2013-05-08). The orange circle marks online publication (2017-04-19). The safety composite was added as a primary outcome in version 2 (2014-02-12), after the study start and during enrollment, but before primary completion. The shaded area is the period after primary completion (axis broken between 2017 and 2021). Version 4 (2016-10-05) falls in it but changed only Study Status and IPDSharing. The only outcome change in that period is version 5 (2021-10-11), which reworded the primaries for results posting. All dates come from the registry history and the paper. Source: make_figures.py.txt.

Registered versus reported primary outcomes

Figure 2. On the left, in blue, are the registered outcomes across versions 1-4 (2013-05-20 to 2016-10-05), including the registered secondary outcome and the registration date. On the right, in orange, are the outcomes as reported in the paper, with the paper's results. Each connector carries a label for its status. The 12-month patency outcome was registered as primary from the first version and matches the paper's primary effectiveness end point. The safety composite was not in version 1. It was added as a primary in version 2 (2014-02-12) and matches the paper's primary safety end point. The registered secondary "Major adverse event rate" (dashed boxes) was deleted in version 2. The paper reports MAEs as an additional outcome, so no primary outcome was dropped or demoted. The bottom row sets the registration date beside the paper's "prospectively registered" statement.

Why it matters

The paper reports a result for each registered primary outcome, and both results are positive. For primary patency at 12 months, the paper reports "83.9% (188 of 224) for DCB and 60.6% (40 of 66) for PTA, for a difference of 23.3% (95% confidence interval, 10.6%–36.1%; P<0.001)" (Results > Effectiveness Outcomes). For the primary safety composite, it reports "94.1% (193 of 205) with DCB and 83.3% (50 of 60) with PTA, for a difference of 10.8% (95% confidence interval, 0.9%–23.0%)", meeting both noninferiority and superiority (Results > Safety Outcomes). The abstract concludes: "Superiority with a low-dose DCB for femoropopliteal interventions was demonstrated over PTA for both the safety and effectiveness end points."

The registry and the paper report the same outcomes, so this is not a case of a switched primary outcome. The issue is what the registry can and cannot vouch for:

Explanations considered

Status

As of 2026-10-23:

Suggested correction

The authors and the journal could correct the registration statement in Methods > Study Design and add the registry history. Suggested wording:

"The study was registered at http://www.clinicaltrials.gov (unique identifier: NCT01858363) on May 8, 2013, after enrollment had begun (first randomization December 2012). The initial registration listed primary patency at 12 months as the primary outcome. The primary safety end point was added to the registered primary outcomes on February 12, 2014, during enrollment and before the 12-month follow-up was complete."

If a protocol amendment set up the safety end point and its noninferiority margin, the correction could also give the amendment's date and rationale and say when the statistical analysis plan was finalized. Making the protocol and the plan available, for example as supplementary material, would let readers check that both primary hypotheses were fixed before outcome data were unblinded. The reported results do not need to change.

Not written up after the deeper check (7)

These earlier-confirmed findings were dropped because the deeper check found a fixing correction, a disclosed or legitimate change, a version-of-record difference, or a failed blind re-check.